Vocabulary · Mac implementation · reference for Jot on iPhone

Jot Vocabulary on Mac

What the Mac app now does with a user's vocabulary, why, and what we measured. Jot for iPhone shares the rules through the jot-shared package, but several pieces live in the app, and those are listed at the end as work for iOS.

Jot 1919006jot-shared 6c53328FluidAudio fork f941d646 (branch jot/bias-rewind)26 Sep 2026
Why it changed

Three rules

Real evidence only

A correction applies only when the speech model itself chose the word, a real acoustic score supports it, or a measured spelling distance does. The old path passed invented values (confidence 0.85, margin 0) to the gate; those are gone.

Nothing silent and invisible

Every applied correction is recorded, can be undone, and can be asked about. Spelling-only matches apply under the full guard set, never as an unseen patch.

Never learn from common words

No rule is learned when the heard word is an everyday word. Rules like “not → Jot” or “fine → Flink” were being learned this way; they are refused now, and removed once from existing stores.


The pipeline

What runs, per speech model

The vocabulary step runs right after decoding, before text cleanup (fillers, numbers) and before any AI rewrite. There is no time cutoff. On a 30-minute dictation it takes about 4.5 s in a debug build.

decode→vocabulary→artifact scrub→cleanup (fillers, numbers)→AI transform (optional)→ask, then paste
ModelWhile transcribingAfter decoding
Nemotron multilingualDecode-time bias toward the user's terms (strength 3.5). The CTC keyword spotter listens to the same audio in parallel.VocabularyPass: spelling matches first, then spotter hits placed on the word Nemotron wrote at that moment.
Nemotron EnglishCTC spotter in parallel. No bias: this model has no hook for it.VocabularyPass, same as above.
Parakeet v2 / v3Nothing.CTC spot + ctcTokenRescore, then VocabularyGate.apply with Parakeet's real token confidences. No spelling corrector on this path.
JapaneseNothing.Alias substitution, unchanged.

Sources: DualPipelineTranscriber.swift (Nemotron), Transcriber.swift (Parakeet), VocabularyRescorerHolder.swift (gateDetections, finishRescore), VoiceInputPipeline.swift (streaming spotter), NemotronMultilingualStreamingTranscriber.swift (bias).

Evidence

Three kinds of evidence, three rules

.decodeBias

The model itself produced the term under bias. Nothing to patch afterwards: the word is already in the transcript. Mac, Nemotron multilingual only.

.acoustic

A CTC spotter detection with its real log-prob score. Placed on the transcript word the decoder wrote within 1.0 s of the detection, using the decoder's own token timings (WordTimeline). When word timings can't be aligned to the transcript, the detection is dropped; it is never placed by proportional position.

.textual

A spelling match from the model-free corrector (word n-grams up to 8, OSA Damerau-Levenshtein, per-language maximum distance, ambiguous ties dropped). Applies only through the gate's full guard set, and only for languages with a common-word list. Visible and undoable.

In VocabularyPass, spelling matches claim their words first. On the owner's 40 recordings that order got 35 of 48 mentions right with one wrong apply in 4 of 5 runs, against 34–35 with one or two wrong the other way round. The corrector names an exact word window; spotter time spans are coarse and its scores uniformly weak.

Decode-time bias

The model listens for the terms

In the FluidAudio fork, the Nemotron greedy decoder adds a bonus to the tokens of a user's term. A term may only start where the unbiased decoder would itself start a word. The decoder snapshots its state when bias first flips a choice; if the term doesn't complete, it rewinds to the snapshot, bans that start, and continues. This works across chunk edges and is resolved at finish(), so a term cut off at stop never leaves half a word. Bias turns off for the rest of an utterance after 64 rewinds. The vocabulary is set only between utterances, with canonical spellings (no aliases).

StrengthRare-word WEROther words WEROwner: names caughtOwner: wrong place
off14.1%2.08%14 / 480
2.58.4%2.02%··
3.07.9%2.00%··
3.57.3%1.98%31 / 481chosen
4.06.8%1.98%35 / 483rejected
4.56.3%1.99%37 / 485rejected

WER columns: LibriSpeech test-clean, 2,620 clips, with the standard 100-word rare-word biasing lists (Le et al., 2021). With 1,000-word lists, 3.5 gives 8.3% and 2.03%. The owner's recordings were used only to reject settings, never to tune one: above 3.5 the bias starts writing terms over words that were actually said.

4.5one hundred fifty flink oh lines
3.5 (open)My name is Vinit Firam Vineet Flink · should be Vineet Sriram

Safety checks that pass at 3.5: a vocabulary of made-up words leaves 40 recordings byte-identical to bias off; live and one-shot decoding agree with bias and blank rescue on or off; stopping inside a term at 262 different points never leaves a fragment.

Learning

What Jot learns, and what it refuses

Asking the user

How the Mac asks “Did you mean…?”

The paste is held and the pill asks one question at a time, then delivers once (AppDelegate.handleDeliveryBridge).

The ranking and limit mirror AskPolicy in jot-shared, but the Mac applies them in its own code (MacVocabGate in VocabCoreAdapters.swift), because the Mac pill works on its own correction type.

Settings

What the user sees on Mac


Checking the iOS summary

Where the iOS summary is right, and where it describes Windows

1. Matching engine. Windows nudges Parakeet while transcribing and always runs the spelling corrector; iOS runs the corrector only when its acoustic model finds nothing; jot-shared adds one combined pass but can't nudge.

right
  • The nudge lives in the speech engine, not in jot-shared. On Mac it's in the FluidAudio fork and only for Nemotron multilingual. Jot for iPhone runs Parakeet, so it has no nudge today; Parakeet bias is planned as shared work for Mac and iPhone.
  • The Mac's Nemotron paths run both kinds of evidence on every recording through VocabularyPass, like Windows. iOS running the corrector only as a fallback is the part to change.
  • One gap on Mac: its Parakeet path doesn't use the spelling corrector yet.

2. What you see. Windows has no on/off switch and no “Sounds like” field, learns alternate spellings only from corrections, caps the list at 200, and warns on bad entries; iOS still has the switch and an editable “Sounds like”.

Windows, not Mac
  • The Mac also still has the switch, keeps aliases, and has no cap. Those removals are Windows decisions and weren't made on Mac.
  • The bad-entry warnings are shared and on Mac. iOS should show them where terms are added.

3. What jot-shared doesn't cover. Vocabulary as the very last step with a 4-second cutoff; Auto-detect and unsupported languages always off; asking only about applied corrections (at most 3); the UI removals.

mostly Windows
  • Last step, 4 s cutoff: not on Mac. The Mac runs vocabulary right after decoding, before cleanup and AI transform, with no cutoff. The ask then checks the final text, so a correction the transform reworded away is dropped.
  • Languages: on Mac the spelling corrector runs only for the 17 languages that ship a common-word list, and fails closed for Slovenian and Croatian. Other languages get acoustic matches only. The Mac has no auto-detect language setting.
  • Asks: at most 3, yes. But the Mac also asks about blocked near-misses whose heard word isn't common, and offers the merge teach once, so asks aren't limited to applied corrections.
For Jot on iPhone

What iOS should do

  1. Run VocabularyPass on every recording, with spelling matches and acoustic matches together, instead of the corrector only as a fallback.jot-shared: VocabularyPass.run
  2. Give it word timings. Build them from Parakeet's token timings with WordTimeline.wordTimes. Without them, acoustic detections are dropped by design.jot-shared: WordTimeline
  3. Mark real spotter hits as acoustic, with their real scores. Never pass a synthesized confidence or margin.VocabularyGate.Detection evidence
  4. Run the one-time learned-rule cleanup at launch and mark it done only on a non-nil result. Use isCommonOriginal for ask admission too.CorrectionStore.dropCommonOriginalRules · VocabularyGate.isCommonOriginal
  5. Show the hygiene warning wherever a term is added.VocabularyHygiene.warning
  6. Keep asks to 3, weakest evidence first, with the same suppression and one-time merge teach.AskPolicy
  7. Don't pull the FluidAudio fork. Its change is Nemotron-only and iPhone runs Parakeet.
  8. Don't edit the existing golden fixtures in jot-shared; Windows is locked to them. Add new fixtures only, and run swift test (38 vocabulary + 8 text-pipeline tests pass today).
Open

Known limits