Why “n-gram” is being discussed
The most recent Hacker News stories and comments contributing to this topic's mentions.
32GB dedicated to an N-gram table instead of a draft model is an unusual choice for speculative decoding — what made it win over the more common draft-model approach here?
by siris9476 · Sep 1, 2026
Oh, cool! Yeah that seems like a good application. The current Storyteller alignment algorithm actually does do just that! We use Whisper to transcribe the audio to text, and th…
by smoores · Aug 31, 2026
It should be fine keeping the n-gram embeddings on SSD which lets you run at least the Q1 and Q2 models on 64GB
by tveita · Aug 29, 2026
It isn't the only marker, for sure, but Claude talks like a thesaurus, and n-gram test like these highlight that.
by swiftcoder · Aug 28, 2026
I suspect we will see optimizations where the various vectors of the n-gram you actually use are hot in vram, the rest are warm in system memory and then cold storage on nvme. S…
by hadlock · Aug 27, 2026
Probably. I've spent zero time with optimization at this point. Code is all new this morning. Curious if llama.cpp is doing full BF16 for the n-gram embeddings table, or if that…
by cmrdporcupine · Aug 26, 2026
Interest
Proportion of Hacker News items mentioning "n-gram" over time.
Mentions
Total number of Hacker News items mentioning "n-gram" over time.
An n-gram is a sequence of n adjacent symbols in a particular order. The symbols may be n adjacent letters, syllables, or rarely whole words found in a language dataset; or adjacent phonemes extracted from a speech-recording dataset, or adjacent base pairs extracted from a genome. They are collected from a text corpus or speech corpus. Read more on Wikipedia