Where we started
In May 2025, OpenAI publicly disclosed that it had banned dozens of Apple employees because they had been making massive numbers of requests through internal OpenAI accounts — requests that allegedly served Apple's own AI development. That included suspicions that training data, architecture details, and internal strategy documents ended up in the prompt history. OpenAI has argued since that Apple bears partial responsibility, because those accounts were created in such large numbers in the first place.
The point isn't that Apple is "evil". The point is that a company with its own legal department, with NDAs, with SOC 2 audits failed to prevent data exfiltration over a publicly accessible API. That's the template we need to look at for our own setup.
Specifically, there are three points that often get lost in the discussion: first, LLMs are designed to retain history by default. Every prompt lands in the token stream, and every token stream is stored for at least 30 days for abuse detection. Second, with moderate effort you can reconstruct what you had in mind when you sent a prompt from the prompt history. Third, the API provider — not you — controls the infrastructure on which all of this happens.
And that's exactly the point where my personal watch-&-alert setup had a similar weak spot. I have about 38 RSS feeds running in my main account. Among them: two mailing list archives that stand in for internal discussions from my day job, a few GitHub watch lists on repositories covered by NDAs, and a Telegram channel where I collect research material for client projects. I dutifully had all of these sources classified by GPT-4o at first. "Is this a security advisory, a privacy topic, or routine?" — done.
Until I noticed that the classification response occasionally contained original text from the feeds. Not the entire post, but individual sentences, because the model returned them as "helpful context". That's functionally identical to what happened to Apple engineers at OpenAI: sensitive data lands in a system you don't operate, and you have no contractual leverage to get it back out.
Requirements
Out of that mess, I derived three hard requirements for our pipeline. Without them, I won't touch the next refactor.
Requirement 1: Sensitive content never leaves the machine. Anything that might contain trade secrets — mailing list archives, NDA-relevant GitHub repos, private research channels — gets classified locally. Period. No discussion, no "but GPT-4o is just better".
Requirement 2: The system stays maintainable for one person. I don't have a security engineer in the background helping me set up a seven-layer audit. The solution has to fit in 200 lines of Python, otherwise it won't work at my weekend pace.
Requirement 3: The heartbeat must keep running. The watch-&-alert setup depends on the system sending a heartbeat every day at 06:30 saying: "I'm alive, here's my stats." If the new architecture breaks that heartbeat, I'll fall back to the old variant. Pragmatism beats purist ideology.
These three requirements were also the litmus test for whether the refactor was worth it at all. I knew that if I ended up without a workable solution, the time spent would be wasted.
Three options, side by side
We played through three variants before we built anything. Each has its merits, but only one fully met our requirements.
Option 1: Keep everything on GPT-4o
The simplest variant: keep the status quo and hope that OpenAI doesn't repeat the Apple fiasco with our data. It works technically, but fails Requirement 1. If a provider can't guarantee data hygiene for a company with 160,000 employees, it certainly won't guarantee it for a one-person project. Then there's the cost: at about 1,200 alerts per month and an average of 800 input tokens per classification, we land at roughly 12 US dollars per month. Not the end of the world, but not zero either — and that money funds a problem, not a solution.
Option 2: Fully local, Ollama only
Variant two is the other extreme: classify everything through a local Ollama model, no cloud fallback. Advantages: no API costs, no data leaves the house, the model can be fine-tuned specifically to my feed texts. Disadvantages: the hardware. On my decommissioned NUC with an 8th-gen i5 and 16 GB of RAM, a 7B quantized model runs at about 2.3 tokens per second. A classification takes between four and eight seconds. At 1,200 alerts per month, that doesn't matter. But once I start wanting to classify larger content — complete PDFs or thread histories, for instance — the pipeline gets sluggish. Also: a 7B model is enough for Heise and Tagesschau. As soon as I get into specialized topics like Vulkan API changes or legal nuances, the hit rate drops noticeably.
Option 3: Hybrid — local for sensitive content, cloud for the rest
Variant three splits the data stream. A small routing layer checks each feed for a sensitive flag. Sensitive feeds go through a local Ollama model, uncritical feeds can keep going through GPT-4o. That's the architecture we built. It's honestly not the most elegant, but it's the most pragmatic, because it combines the benefits of both worlds without betraying the requirements.
The solution
The core of the solution is a routing filter that decides per feed where the content goes. We wrote it in Python 3.12, with type hints throughout, so that half a year from now I still understand what I did.
from __future__ import annotations
import logging
import sqlite3
from dataclasses import dataclass
from pathlib import Path
logger = logging.getLogger(__name__)
@dataclass(frozen=True, slots=True)
class FeedPolicy:
"""Steuert, wie ein einzelner Feed klassifiziert wird."""
feed_id: int
title: str
sensitive: bool
def load_feed_policies(db_path: Path) -> list[FeedPolicy]:
"""Lädt die Feed-Policies aus der SQLite-Datenbank."""
with sqlite3.connect(db_path) as conn:
rows = conn.execute(
"""
SELECT id, title, sensitive
FROM feeds
WHERE active = 1
ORDER BY id
"""
).fetchall()
return [FeedPolicy(*row) for row in rows]
def classify(content: str, policy: FeedPolicy) -> dict[str, str]:
"""Wählt anhand der Policy das passende Backend."""
if policy.sensitive:
logger.info("lokales ollama für feed=%s", policy.title)
return _classify_local(content, model="llama3.1:8b")
logger.debug("cloud-gpt-4o-mini für feed=%s", policy.title)
return _classify_cloud(content, model="gpt-4o-mini")
def _classify_local(content: str, model: str) -> dict[str, str]:
"""Stub: real ruft hier ollama via HTTP auf."""
return {"backend": "local", "model": model, "label": "TODO"}
def _classify_cloud(content: str, model: str) -> dict[str, str]:
"""Stub: real ruft hier die OpenAI-API auf."""
return {"backend": "cloud", "model": model, "label": "TODO"}
Here's what's happening: load_feed_policies reads from the same SQLite database that also holds the RSS feed items. The sensitive flag per feed is a column I set once and that drives the routing logic. The actual classification step happens in classify, which decides based on the policy. I've left out the stub functions _classify_local and _classify_cloud here because they're trivial — the local variant talks to Ollama via http://localhost:11434/api/generate, the cloud variant uses the openai Python package.
The SQLite extension is intentionally kept small. A sensitive INTEGER NOT NULL DEFAULT 0 column, an index on active, done. We don't do this with an external policy file, because keeping the policy file and the data source in sync keeps causing surprises. Better to have a single source of truth.
The classification output lands back in the SQLite database, each with backend and model as columns. That gives me a per-alert audit trail of which model produced the result. That's important not just for me, but also for the morning heartbeat: if the rate of "unclassifiable" results suddenly climbs, I see it immediately in the report.
What we learned
Five lessons stuck with me from the refactor, and I think they're useful beyond this specific case.
Lesson 1: The architecture must enforce data hygiene, not merely permit it. A policy column in the database is enough, because it can't be bypassed without changing the code. A Markdown list in a Confluence document doesn't work — eventually someone forgets it.
Lesson 2: Local LLMs are no longer exotic in 2026. A llama3.1:8b quantized model fits in 5 GB of RAM, runs on a CPU, and delivers sufficient quality for RSS classification. If you haven't tried it in the past two years, catch up now.
Lesson 3: Cloud LLMs remain useful when the data stream contains no secrets. Heise, Tagesschau, public GitHub releases — that's all fine to send out. It saves hardware and often delivers better results, because the models there are larger.
Lesson 4: The heartbeat is the most honest quality indicator. When the pipeline posts its report every morning at 06:31 and the report shows 38 feeds read, 1,247 items classified, 19 alerts triggered — then the thing works. If the heartbeat misses one day, something is broken, and exactly where I didn't expect.
Lesson 5: The effort was less than feared. Roughly four hours for the database extension, three hours for the routing logic, one hour for the heartbeat report. That's about one working day, and I have a system that no longer violates Requirement 1. That's a day that pays off.
How this fits into the book
The book "Watch & Alert: From RSS feed to your own alert pipeline" covers exactly this local-versus-cloud split in several places. Chapter 6 under "Local models with Ollama" walks through how to set up Ollama on a 32 GB RAM mini PC, including the question of which model size still makes sense for which kind of classification. The section "Privacy as an architectural principle" spells out the rule we applied live here: sensitive data doesn't leave the machine, because otherwise the machine is no longer yours.
Chapter 9 under "Security and secrets" — starting at the section "Keeping API keys and tokens cleanly separated" addresses the other side of the problem in parallel: even with the routing logic in place, you need to separate cloud and local credentials. The book presents a dual-key system that prevents you from accidentally using the OpenAI key for a local model (or vice versa). That separation is intentionally not shown in the code snippet above — it belongs in its own configuration layer, and that's described in detail in the book starting at page 142.
The book uses a slightly different variant for the same decision: instead of a sensitive column in SQLite, it recommends a YAML file with feed groups, because that's easier for beginners to edit. In our setup here, we chose the database variant because it sits closer to the pipeline and makes migrations easier. Both end up at the same place — two paths to the same goal.
More on routing sensitive content can be found in Chapter 6 under "Local models with Ollama" (starting at the section "Privacy as an architectural principle"). More on secure key management in Chapter 9 under "Security and secrets" (starting at the section "Keeping API keys and tokens cleanly separated").
When you should rebuild this — and when you shouldn't
It's worth rebuilding if your watch-&-alert pipeline processes content that isn't meant for outside eyes — NDA-relevant repositories, internal mailing lists, client material, research notes. In that case, the split between local and cloud isn't an overreaction, just necessary.
It's not worth rebuilding if your pipeline consists exclusively of public sources and the classification cost is a real burden. If you're fine with 15 news feeds and pay 4 euros a month to OpenAI, the hybrid setup is overkill. Stay on the cloud and call it a day.
If you're somewhere in between, I'd recommend experimenting with the local model before reworking the architecture. Install Ollama, pull llama3.1:8b, have it classify 200 of your items on a trial basis, and compare the results to GPT-4o. If the difference hurts in day-to-day use, you need the hybrid. If not, save yourself the effort.
What you should definitely not do: use the architecture "because Apple has the problem" as an argument against any cloud use. Apple and OpenAI have a specific organizational problem. Your hobby project has a different one. Stay honest, weigh your options, document your decision — and sleep well.