Classifying 29,559 local news items with AI on a laptop

pontecesures.net is a personal project I’ve been running for years: the news portal of Pontecesures, my home town, a Galician municipality of about 3,000 people. The idea has always been a simple one — keeping the information about what happens there alive. Twenty-one years of archive, from 2005 to 2026, 29,559 entries.

The problem is that most of them are uncategorised or categorised inconsistently. When you’ve been publishing for two decades, taxonomies decay on their own: I ended up with 56 categories that no longer meant anything. An archive like that stops being browsable, and at that point much of its value is lost. It’s exactly the kind of task you keep putting off, and one I had wanted to tackle for a long time.

This weekend I finally did. The idea was to assign categories and tags automatically, with human review on top, and push the result back to WordPress. What came out is a proof of concept that works, and above all a collection of surprises I wasn’t expecting.

Worth saying up front: this means writing to a production site that is mine. That explains several decisions that would otherwise look like paranoia, and also why one of the failures I describe below genuinely made me sweat.

I set myself three constraints from the start:

1. Everything local. No cloud services. A MacBook Pro M2 Max with 32 GB and nothing else.
2. No Python. A personal quirk, I admit it.
3. Bilingual content. Spanish and Galician mixed together, sometimes within the same entry.

Those three conditions shaped everything that followed.

The architecture

flowchart LR
    WP[("WordPress REST API")] -->|ingest| DB[("SQLite + sqlite-vec")]
    DB <--> EMB["bge-m3 · embeddings"]
    DB <--> LLM["qwen3-4b · classifier"]
    DB --> UI["Astro + React UI · human review"]
    UI -->|sync| WP

    subgraph LMS["LM Studio (local)"]
        EMB
        LLM
    end

Bun + TypeScript, SQLite with the sqlite-vec extension, LM Studio for the models, and Astro with React and Tailwind for the review interface.

The obvious question is why SQLite and not a dedicated vector database. The answer is that 29,559 vectors of 1,024 dimensions take up around 120 MB. That fits in a single file. Qdrant or Chroma would force me to run a separate server process for a volume SQLite handles without breaking a sweat, and on top of that I’d lose the advantage of having metadata and vectors in the same SQL. At this size, setting up infrastructure would be doing extra work for nothing.

The corpus, and an unpleasant surprise

That last figure deserves a mention of its own. 17.5% of the archive has characters lost at source: Éxito shows up as ??xito, PABELLÓN as PABELL??N. And this isn’t recoverable mojibake — there’s no Ã or â€ anywhere: the original bytes are simply gone from what the WordPress API serves. They were lost in some migration years ago.

The curious part is that the embeddings tolerate it surprisingly well. The clustering comes out coherent regardless. It looks ugly in any interface, but it broke nothing.

Language stopped being a problem

The first real obstacle was Galician. The embedding model I had installed, nomic-embed-text, is English-only. Useless here.

I picked bge-m3 (multilingual, 1,024 dimensions, 8k token context) and forced myself to validate it before building anything on top:

The margin is enormous. And I confirmed it with real data from the archive: starting from a Galician entry, “Botellón na estación”, the nearest neighbour was a news item in Spanish about the same incident, “Personal de ADIF evitó un nuevo botellón en la estación”.

The conclusion is reusable well beyond this project: with a decent multilingual embedding, language stops being a classification problem. There’s no need to split corpora by language or translate anything. It’s one of the things that surprised me most by how well it works.

Clustering doesn’t give you topics, it gives you places

Here’s the finding I least expected.

I discarded the 56 categories that already existed in WordPress — chaos accumulated over years — and decided to build the taxonomy from scratch. The plan was to run k-means over the vectors and let the topics emerge on their own.

It didn’t work. With k=10 and k=14, the largest clusters were these:

Those are places, not topics. Territory weighs so heavily in the embeddings that the algorithm groups by municipality before it groups by subject: a sports item from Valga lands next to a festival item from Valga, because they share place names, mayors’ names and streets. Only Esquelas (obituaries), Emprego (jobs), Deportes (sports) and Sucesos (incidents) emerged cleanly, because their vocabulary is distinctive enough to beat the vocabulary of place.

What clustering was good for was discovering topics that weren’t on my initial list: obituaries (690 entries), jobs (around 1,500), the commuter train, the Ulla lamprey, the Camiño de Santiago, the orchestras at the Chanteclair hall. Four of them ended up as categories I hadn’t anticipated.

The final taxonomy is 14 categories in Galician, deliberately few and broad:

Política · Obras · Sucesos · Deportes · Cultura · Festas · Sociedade · Avisos · Novidades · Esquelas · Emprego · Turismo · Medio ambiente · Resumos

And one decision that turned out to be key: territory goes in tags, not in categories. A news item about roadworks in Valga is Obras with a Valga tag, not an “Other municipalities” category. That way “all Obras” includes items from outside the town, and topic and place can be crossed freely.

The k-NN + LLM hybrid

The LLM classifier takes around 2.8 seconds per entry. For 29,559 that would be about 23 hours of laptop fan noise.

The obvious alternative: use k-NN over the embeddings for entries whose already-classified neighbours agree, and reserve the LLM for the doubtful ones. If your five nearest neighbours are all Deportes, you probably are too.

I calibrated the k-NN with leave-one-out over 305 hand-labelled entries:

I chose K=5 with unanimous consensus. With partial consensus accuracy drops to 87%, too much error to accept without review.

Here I made a methodological mistake worth avoiding: leave-one-out underestimates real coverage. It gave 13%, but in production the first pass gave 28% and the second 51%. The reason is that leave-one-out compares each entry only against the others in the seed set, whereas in production an entry can have as its neighbour any of the already-classified ones — including those the k-NN itself has just assigned in that same pass. The effect compounds within a single run.

The per-round performance tells the rest of the story:

Performance decays between rounds rather than growing. What’s left at the end are the genuinely ambiguous entries, the ones whose neighbours disagree with each other. Seen that way, k-NN does a very specific job: it separates the easy from the hard. And the hard part is exactly the LLM’s job.

The saving was real: 21,289 entries resolved by k-NN at around 25 ms each. With the LLM that would have been roughly 17 hours.

The experiment that failed, which is the most interesting part

An initial experiment suggested something very promising: alternating blocks of LLM work with k-NN passes multiplied throughput. 34 entries classified by the LLM unlocked 419 via k-NN, a factor of ×12. The logic seemed impeccable: every entry the LLM resolves becomes an available neighbour for many others.

I implemented an automatic cycle alternating the two. And it didn’t hold up:

The ×12 was luck. Those 34 entries happened to land in dense regions of the corpus.

The decisive measurement came at the end: 1,400 entries classified by the LLM in one go unlocked 103 via k-NN. A ×0.07, with 1.9% coverage. With a volume 40 times larger than the initial experiment, throughput was 170 times worse. There’s no room for ambiguity: k-NN was exhausted in this corpus.

I also tried a heuristic to pick the block better, prioritising pending entries with more unclassified neighbours, on the hypothesis that they’d unlock more. That didn’t work either: ×0.8 against the ×1.0 of random choice. The problem wasn’t missing neighbours, it was that they disagree with each other.

The final comparison, over 6,900 pending entries:

Six minutes of difference. All the complexity of the cycle contributed absolutely nothing.

The lesson seems to apply in many contexts: an experiment with a spectacular result and a small sample deserves to be replicated before you build on it. And it’s worth measuring the total cost of the simple alternative before starting to optimise. I built the entire cycle before checking how long the brute-force approach would take.

Where a small LLM fails

qwen3-4b classifies topics well, but it has two blind spots I documented in detail.

It doesn’t detect aggregates. Entries that bundle several distinct news items (“Noticias de Padrón e Valga – 8 de xuño”) get the category of the first topic instead of being recognised as a roundup. 0 out of 70 detected, even after adding an explicit rule at the very start of the prompt. The model understands what a text is about, but it doesn’t recognise the shape of the text.

I solved it with a pattern rule: unmistakable markers like several consecutive “Ler máis” links or a “Noticias recollidas de Google News” footer. Ten lines of code get it right where the model fails 100% of the time.

Confidence is compressed. It almost never drops below 0.85. But the interesting thing is that the model does discriminate:

Its “I’m unsure” is 0.85, not 0.5. Adding explicit anchors to the prompt didn’t change the scale. To set an operational threshold you have to put it where the real signal is, not where intuition would say.

The silent failure that survived the whole process

This one is my favourite, because it’s the kind of bug that’s genuinely frightening.

With 96.8% of the archive already classified, one particular entry still had no category. It had 3,754 characters, it was unmistakably political — the PP spokesperson criticising the local government — and the system reported it as “could not be classified: usually due to lack of text”.

The diagnosis was false. When I asked for the model’s raw response, this appeared:

{"categorias": ["Política"], "confianza": 0.9, "razon": "O texto trata da
análise crítica do goberno local... a mención de "análise global do goberno"
indican un enquadramento político claro."}

The model had got it right. With 0.9 confidence and impeccable reasoning. What was failing was my parser: the LLM writes unescaped double quotes inside the reasoning field, JSON.parse throws, and my code treated that as “the model didn’t know”.

Two things kept it hidden throughout the whole project:

1. The error message lied. It said “too little text” because that was the usual reason classify() returned null. Nobody suspects a parser when the system blames the data.
2. The failure rate was plausible. 1-2% of entries left unclassified raises no eyebrows in a corpus with one-line entries. And the failure only triggered when the model elaborated enough to quote something — that is, in the best-reasoned responses.

The measured impact: of the 30 entries still pending, the old parser resolved 6 and the fixed one resolves 20. The failure rate went from 80% to 13%. I’ll never know how many of the ~7,000 classified by LLM were lost to this over the course of the process, because they got mixed in with the genuinely impossible ones.

The lesson: an error message that blames the data is the best hiding place for a bug in the code. When a catch turns any exception into a null with a generic explanation, the system stops distinguishing “can’t be done” from “I couldn’t read it”.

A forgotten column that deleted categories in production

And this is the one that made me sweat.

Categories and tags live in the same table, taxonomy_terms, distinguished by a kind column. It’s a normal, compact design. It’s also a minefield: any query that forgets to filter by kind mixes the two, and the result almost never fails loudly. It just returns too much.

The omission showed up in seven places. Six were cosmetic: the front page listed Valga and Lamprea among the categories, the counts were inflated. No test caught it — I caught it myself, looking at the interface and thinking «in categories I’m seeing a lot that I think are tags».

The seventh wrote to production. The function that sends categories to WordPress selected terms like this:

SELECT c.post_id, group_concat(t.wp_term_id) ids
  FROM classifications c
  JOIN taxonomy_terms t ON t.id = c.term_id
 WHERE c.taxonomy_version_id = ?
   AND (c.reviewed_at IS NOT NULL OR c.source = 'llm'
        OR (c.source = 'knn' AND c.confidence >= 0.75))

No AND t.kind = 'category'. And three circumstances came together there:

1. Tags are assigned by rule, with source='llm' and confidence 1.0: they passed every quality filter, every time.
2. Tag IDs and category IDs are both integers. Nothing in the type distinguishes a 341 from a 368.
3. WordPress doesn’t complain. It receives categories: [219, 368, 352], silently discards the ones that aren’t categories and, if none valid remain, empties the field.

The result: entries that had a category in WordPress lost it. A news item titled «O Día da Bandeira», with its correct Festas in the local database, ended up as “Uncategorised” on the public site. The request returned 200 OK and the system reported “500 updated, 0 failures”.

It’s worth pausing here: I was deleting, live and without noticing, categories from a twenty-one-year archive that I maintain myself. No staging environment or recent backup would have helped, because the site is the site.

Scope: 143 entries out of the 224 written before I caught it, 64%. The other 81 had at least one valid category among the IDs sent and survived by chance.

It was repairable because every write is logged in wp_writes with the previous state:

{"post_id": 921, "payload": "{\"categories\":[219,368,352]}",
 "previous": "[20,23,2]", "applied_at": "2026-08-10 14:55:37"}

That previous was a requirement I set myself on day one precisely because this is a production site. Without it there would have been no way to know even which entries to touch: nothing in WordPress distinguishes an entry that never had a category from one whose category you just deleted. The 143 were repaired in a minute.

What gave it away wasn’t a test, it was looking at the result. The system reported success. Only by opening a specific entry on the site and comparing it against what the database said did the discrepancy appear. The verification that worked was the dumbest one: go and check whether it’s actually there.

Three things I’m keeping:

• A discriminator in a column is an invitation to forget it. No type backs it up: SELECT ... FROM taxonomy_terms compiles just as happily with or without the filter. If I rebuilt it, two SQL views (categories, tags) over the same table would make the error impossible.
• The highest-confidence data is the most dangerous. The tags got through precisely because they had confidence 1.0. A quality filter is not a substitute for an identity filter: it checked how reliable the term was, never what kind of term it was.
• “0 failures” measures what the code knows how to check. The loop counted successful HTTP responses, and every one of them was.

Tags: when a regex beats the model

Categories answer “what is this about”. Tags do something else: they relate entries to each other. Places (Estación, N-550), entities (ADIF, Nestlé, BNG), recurring topics (Lamprea, Entroido).

The criterion I set myself: a good tag is one that, when clicked, returns a set of news items that makes sense to read together.

And here clustering did work. With k=30 sharp groups emerged — commuter train, Ulla lamprey, basketball, the rowing club, Nestlé, unemployment — and the LLM, when naming them, proposed tags I hadn’t anticipated: N-550, Alto de Cordeiro, Valeiros, Mancomunidade Ulla-Umia, Cerámica Celta, Iria Flavia.

It’s the exact reverse of what happened with categories. The same clustering that only gave places there produces exactly what’s wanted here.

There was plenty of LLM noise to filter out, mind you: tags in Spanish despite asking for Galician (Huelga de basura), generic ones that are categories in disguise (Emprego, Accidente), ones with no value as tags (Decembro, Fin de semana) and duplicates (Ría de Arousa / Mar de Arousa). I also discarded Concello, which appears in 10,153 entries — a third of the archive: a tag that marks a third of everything relates nothing.

But the important decision was another one: assignment is by regular expression, not by LLM. These are named entities: either the pattern appears in the text or it doesn’t. A rule is more accurate than the model and costs three orders of magnitude less.

The result: 41 tags, 43,745 assignments, 1.5 per entry on average, 75.2% of the archive tagged. With no dominant tag, from Valga (6,563) down to Alto de Cordeiro (36).

An example of what it achieves:

“The new street furniture isn’t being installed at the Station” → Estación · Padrón · Catoira · ADIF · Tren de proximidade

The lesson here is that not every classification problem needs a model. For named entities, a well-chosen list of patterns wins on accuracy, cost and predictability. The LLM was useful for discovering which tags should exist; assigning them is a regular expression’s job.

Looking back, that division of labour is what best sums up the whole project. I started out assuming that embeddings plus an LLM would solve the problem end to end, and the archive corrected me three times:

In all three cases the mistake was the same: asking the model to decide rather than to propose. Clustering is excellent at showing you what’s in a corpus you hadn’t anticipated, and rather poor at telling you how to organise it. The LLM understands what a text is about, but not its shape, and for “does ADIF appear here?” a one-line rule is more accurate, cheaper and — this counts for more than it seems — predictable: I can read it and know exactly what it will match.

What the model did solve end to end was the thematic classification of 12,154 entries, which is genuine judgement work and where nothing else would have served. The question isn’t whether to use AI, but which part of the problem is really about judgement and which is about recognising a pattern. Confusing the two costs three orders of magnitude.

Where it stands now

The distribution by category came out like this:

Avisos with 151 entries is still anomalous: there are far more public notices and announcements in the archive. It’s the same blind spot as with Resumos: the model prefers thematic categories over ones that depend on the shape of the text. It probably needs a pattern rule too.

And there’s a distinction that took me a while to see: classified is not the same as published. Of the 28,637 entries with a category, only 20,553 make it to WordPress. The rest are k-NN assignments below the 0.75 confidence threshold, deliberately left out because that’s where the errors concentrate:

That distinction wasn’t visible in the interface. An entry with a k-NN category at 0.68 looked the same as a hand-reviewed one, and there was no way to find the ones that weren’t going to be published. I added an explicit filter and a per-row marker. It’s the same number seen from two sides: what the database knows and what the site shows don’t match, and the interface has to say which of the two it’s displaying.

Reference timings

In case they help anyone size up something similar:

The models I used, and why

Two models, both running locally on LM Studio and chosen on very different criteria.

bge-m3, for the embeddings

BAAI/bge-m3 (text-embedding-bge-m3), 1,024 dimensions, 8k token context.

The main reason was Galician. My first choice was nomic-embed-text, which I already had installed, but it’s English-only and was of no use here. bge-m3 is trained on more than 100 languages and, crucially for this case, places the same concept at the same point in vector space regardless of language. That 0.96 similarity between Spanish and Galician against 0.37 between different topics is what allowed me to treat the whole archive as a single corpus.

The other two reasons were context and size. The 8k tokens cover the 99th percentile of my entries (~1,650 tokens) with plenty of room to spare, so I never had to chunk any text. And 1,024 dimensions is a sensible middle ground: the 29,543 vectors take up around 120 MB, which fits in a SQLite file without needing separate infrastructure.

qwen3-4b, for classification

Qwen3-4B (qwen/qwen3-4b-2507).

Here the criterion was different: I wanted the smallest model that would do the job well. With 29,559 entries to process, every tenth of a second per entry adds up to hours. A 4B model fits comfortably in 32 GB of RAM alongside the embedding model, leaves the laptop usable while it works, and classifies at ~2.8 s per entry. A 30B one would have multiplied that time without a matching improvement, because the task — read a local news item and say whether it’s Deportes or Festas — isn’t especially hard.

Being multilingual mattered as much as it did for the embeddings: it has to understand Galician text and return categories in Galician.

The 2507 version is the July 2025 refresh, with better instruction following than the original, which is exactly what a classifier needs when you’re asking it for JSON in a specific schema.

And it delivered: 12,154 entries classified with sound judgement. Its two limits — it doesn’t detect aggregates, and it compresses confidence around 0.85 — are the kind you work around with a rule, not the kind you fix with a bigger model. In fact I tried explaining the aggregate problem in the prompt and it still failed all 70 times, which suggests it’s a limitation of the task type rather than of scale.

What I didn’t use

No paid API. Not because of price — classifying 29,559 entries with a small commercial model would cost a few euros — but because it wasn’t necessary: a local model handles the task, and this way the whole archive stays on my machine. I also ruled out fine-tuning a model of my own; 305 hand-labelled entries aren’t enough material, and the k-NN over the embeddings already filled that role far more cheaply.

What I’m taking away

Twenty-one years of archive, 29,559 entries, two models running locally and zero euros of API spend. All on a laptop, over a weekend. That part — that this can be done today with no infrastructure and no budget — still strikes me as remarkable.

And the result is the one I wanted: twenty-one years of Pontecesures news that can now be browsed by topic and by place. That someone can click Lamprea and see everything published about the Ulla lamprey since 2005 is exactly what I built the site for. The archive was always there, but an archive you can’t browse is halfway to not existing.

But I started out with an assumption that turned out to be false: that embeddings and an LLM would solve the whole problem for me. They didn’t — not because they failed, the model classified 12,154 entries by topic with good judgement, but because a good part of the work wasn’t judgement, it was pattern recognition. That’s where a regular expression wins: more accurate, 8 seconds instead of 23 hours, and I can read it and know what it will match. The models shone at proposing — discovering topics I hadn’t anticipated, suggesting tags like Alto de Cordeiro; the decisions were made by me or by a rule.

The rest of the lessons aren’t about models either. The multilingual embedding made the language problem disappear effortlessly. An experiment showing ×12 led me to build an entire cycle that didn’t save even six minutes. An overly generic catch hid a bug for the whole project by blaming the data. And a forgotten column in a WHERE deleted categories on a production site without anything returning an error.

The models did their part well. What broke was everything around them, which is where it always breaks.