How I built a semantic recommendation engine

When you finish reading an article, the most important thing for engagement is what you read next.

The way I used to handle “Related Posts” was: match a few tags, maybe check the category, and hope for the best. The problem is that tags are manual, prone to human error, and don’t actually capture the meaning of the content.

So in January I built a semantic recommendation engine, and wrote here that blending embeddings with tags and categories made the picks “strictly better”. In October I took the tags and categories out. They weren’t adding structure. They were covering for an extractor that had been feeding the model broken text.

This is the engine as it works now, and what I got wrong the first time.

The original idea: hybrid scoring

The first version embedded every post and scored each pair with a weighted blend:

1. Semantic similarity (85%): cosine similarity between the two posts’ embeddings.
2. Tag overlap (10%): a Jaccard index of shared tags.
3. Category bonus (5%): a small bump if both were Posts, or both were Works.

const semantic = cosineSimilarity(aVec, bVec);
const tagSim = jaccard(a.meta.tags, b.meta.tags);
const sameCategory = a.category === b.category ? 1 : 0;

const score = 0.85 * semantic + 0.1 * tagSim + 0.05 * sameCategory;

Cosine similarity compares two embeddings by the angle between them; vectors pointing the same way mean something similar. The reasoning was that meaning should drive the pick, and structure should break ties.

What I found when I read the output

The recommendations looked fine for clusters of obviously related posts, like my Astro search posts. They were odd everywhere else, so I read what the script actually sent to the model.

The chunking never ran. The script stripped Markdown, collapsed all whitespace, then split the text into chunks on blank lines. By then there were no blank lines left except the one after the metadata header. Every post became two chunks: a few hundred characters of title, description and tags, and the entire body as one block. Averaging them gave the header as much weight as the whole article. The 1,200-character chunk size in the config did nothing.

The text was damaged before it was embedded. A regular expression removed Markdown links, including their visible words. A sentence from my TanStack Start post reached the model as “A few days after I , PageSpeed Insights scored the home page 73”. Meanwhile an MDX comment I’d left in another post, a private note tagged [PERSONAL EXPERIENCE], went straight through.

Tags rewarded my copy-pasting. Two of my project pages, Codemap and Drool, share exactly the same three tags. That gave them a perfect tag score, so Codemap recommended Drool first even though the embeddings rated Learnway as more similar.

The category bonus overruled better matches. My Next Filing is a tax app I built at a govtech hackathon. Its best embedding match was a post about open-source government software. The 5% bump for staying among projects pushed an unrelated productivity app above it.

None of the weights were wrong in isolation. They were tuned on top of text the model couldn’t read properly, and they made the output look more stable than it was.

Embed the whole document

Extraction now walks the MDX syntax tree instead of running regular expressions over it. It keeps headings, paragraphs, link text, inline code, image captions and FAQ answers, in reading order. It drops code blocks, imports and comments. Internal linking uses the same extractor, so both features see the same version of a post.

There’s no chunking or averaging anymore. The embedding model is voyage-4-large through Vercel’s AI Gateway, and it reads up to 32,000 tokens at once. My longest post is about 21,000 characters, so every post is embedded whole.

Shortlist, then rerank

Embeddings are good at finding candidates and less good at judging between close ones. So the script uses them only to shortlist the twelve nearest posts or projects, then hands that list to a reranker.

A reranker reads a query and a document together and scores how well the document matches. The query here describes the post being read: its title, description, excerpt and section headings. Each document is a candidate’s full text.

const candidates = shortlist(entry.key, vectors, policy.shortlist);
const scores = await rerankScores(
  policy.rerankModel,
  recommendationQuery(entry),
  candidates.map((candidate) => lookup(candidate.key).input),
);

The top three candidates scoring at least 0.35 become recommendations. Tags and categories play no part.

Two things surprised me while tuning this.

First, I started with an instruction at the top of the query: score how much a reader who just finished this piece would want to read the document next. It made things worse. My posts about recommendations and internal linking started appearing under unrelated essays, because the instruction matched their subject better than the essays matched each other. Without it, the scores separate cleanly.

Second, some posts have nothing related on the site. Two essays, one on imitation versus inspiration and one on whether AI inference is profitable, have no candidate above the floor. The old engine always filled three slots, so those essays recommended whatever happened to be nearest. Now they show no related section. An empty section is better than three picks I’d have to apologise for.

The reranker also found connections tags never would. My essay “The Greater Fool” walks through the ventures I tried, and it’s now the top recommendation on several of those project pages, Weaveu among them. Before, Weaveu pointed to other project pages that happened to sit nearby.

Cache by exact input

The fastest way to burn money on AI APIs is to send the same text twice.

Every embedding is cached under a hash of the model name and the exact text sent. Every rerank score is cached under the reranker, the query and the document. Both are stored per item, so a new post costs its own embedding and the handful of comparisons it takes part in. Everything else is read from disk.

A full rebuild of all 49 posts and projects comes to roughly fifteen cents, based on the gateway’s published per-token prices. A normal edit costs a fraction of that.

Where it runs

The script used to run before every build. Now it’s an authoring command:

pnpm run recommend

It’s also part of pnpm run content:prepare, which refreshes all the site’s AI-generated inputs. It writes src/data/editorial/recommendations.json, with a fingerprint of the content it ranked. The build checks that fingerprint and warns when a post changed after the last run, but it doesn’t call any model.

The site now runs on TanStack Start, and every page imports a small related.json generated from that file. It holds just the top three IDs per page, with no scores, so it adds almost nothing to the bundle. No client-side fetching, no API latency, no database connection strings.

Fewer knobs, better picks

The hybrid score felt clever because every weight had a story behind it. But I was tuning weights on output I hadn’t read, built from input I’d never checked. Once the text was clean, the extra structure had nothing left to correct and plenty to distort.

The engine is simpler now. It extracts the text properly, embeds it, shortlists with cosine similarity, and lets a model that reads both texts make the final call. When nothing clears the bar, it says nothing.