Writing
Essays
Shipping AI from R&D and PoC to production at scale.
Spaces
Explore side quests
Essay-adjacent feeds that can be shared independently now and split into their own domains later.
Aug 26, 2026 · Embeddings · 11 min
Embeddings from scratch
A model is handed integer ids, not meaning. Embeddings turn those ids into vectors that a network learns by predicting each token's neighbors - so tokens that share contexts end up close together, and geometry emerges. Here Skip-gram and CBOW are built and trained from scratch, with an honest look at what they produce on a tiny corpus.
embeddings · word2vec · nlp · from-scratch
Jul 16, 2026 · Tokenization · 7 min
Building BPE from scratch
Byte-Pair Encoding fixes word-level tokenization by working with subword pieces: it starts from single-byte characters - so text in its range is never out-of-vocabulary - and repeatedly merges the most frequent adjacent pair. Here it is built and trained from scratch, with a live playground.
tokenization · bpe · nlp · from-scratch
Jul 16, 2026 · Tokenization · 6 min
Why word-level tokenizers break
A model never sees text - it sees integer ids from a fixed vocabulary. Building the simplest possible tokenizer makes the encode/decode contract concrete, and shows exactly where word-level tokenization loses information.
tokenization · nlp · from-scratch