Jeremy WangProject page · updated 2 Nov 2026LinkedIn

Language models write plausible text. Code checks it.

A language model predicts likely text; it does not check facts. So in the data pipeline I designed, the model has to quote the posting for every requirement it reads, and code checks those quotes against the posting. The pipeline runs every morning at 05:00: collect, load into SQL, read, check, report.

It started as my own job search, and I still use it every day, so when the model misreads a posting I see it the next morning.

Python
SQL · SQLite
pandas
language-model APIs
Claude Code
pytest
Git

job postings in the SQL database: 43,648
new postings on a typical morning: ≈370
model calls logged with prompt, response and cost: 71,821
automated tests before a change ships: 3,200+

As of 2 November 2026. Each figure is traced to its source at the end of this page.

What runs at 05:00

Six steps, unattended, before the working day starts

A scheduler on a Mac starts the run at 05:00. It has run on 166 mornings since 16 May 2026; a typical run takes about 90 minutes.

Collect
Career sites of 50+ employers (banks, insurers and public bodies in Canada, employers in Hong Kong) and job boards.
Load
One row per posting in SQLite, deduplicated. Each run records a database integrity check before and after.
Filter
Rules set aside duplicates and postings that are clearly out of scope before any model is paid to read them.
Read
A language model returns structured fields for each posting: requirements, years, pay. Each requirement comes with a quote from the posting.
Check
Code validates every answer against its schema. Where a quote decides something, code looks for it in the posting first.
Report
A ranked list on a local dashboard, led by what needs action that day.

Why code checks the model

A plausible quote is not the same as a real one

A language model is very good at producing the sentence a posting probably contains. It has no step where it goes back to the posting to confirm. So every requirement it reports must carry a quote, and a few lines of Python check that those exact words are in the posting. Here is one real posting, with the check run over the model’s quotes.

Operations Analyst · Canada

read 29 Oct 2026 · 5 of 6 quotes found

✓IT-related degree or diploma“Completion of a recognized degree or diploma program in an IT related discipline”
✓Class 5 driver’s licence and vehicle“Have and maintain a valid Manitoba Class 5 Drivers License and access to a vehicle”
✓ITIL process methods“Using prescribed Information Technology Infrastructure Library (ITIL) process methods”
✓Tier II service desk support“Action and close all Tier II support calls”
✓Hardware and software installation“Install hardware, operating system, and application software”
✗Customer service Model: “customer service and decision making and problem solving” The posting says“…communication, customer service, decision making and problem solving, integrity…”

The model added one word. Customer service is in the posting, but this quote is not, so it does not count as evidence.

The code makes no judgement about meaning. It checks one fact: are these words in the posting? Meaning stays with the model, and the model has to show where it found it.

Where a quote decides something, such as which résumé a posting gets or whether a years requirement rules it out, a reading with a missing quote does not count. The posting stays in the queue and the doubt is written down for a person to look at.

The whole check · pipeline/base_queue.pyPython

def _norm(s: str) -> str:
    s = (s or "").replace("\u2019", "'").replace("\u2018", "'").replace("\u201c", '"').replace("\u201d", '"')
    s = re.sub(r"\\([^\w\s])", r"\1", s).replace("\u2013", "-").replace("\u2014", "-")   # markdown escapes: 3\+
    return re.sub(r"[\s*_#>]+", " ", s).strip().lower()


def quote_found(quote: str | None, jd_text: str) -> bool:
    """A model quote is in the posting: every piece around an ellipsis appears verbatim (whitespace, quotes, markdown
    and case normalised)."""
    pieces = [p for p in re.split(r"\s*(?:\.\.\.|…)\s*", _norm(quote or "")) if len(p) >= 8]
    jd = _norm(jd_text)
    return bool(pieces) and all(p in jd for p in pieces)

129,656requirement quotes from 20,860 postings, each run through this function on 2 Nov 2026

99.58%found word for word in the posting

539not found, in 339 postings. Most are tidy sentences the posting never contained. At this volume, 0.4% is still hundreds of claims without evidence.

Checking what a number means

A score that looked precise and ranked nothing

The board used to show a 0–100 fit score on every posting. In August I audited it: 41 of the 61 postings in the top tier sat at exactly 96 or 97. The number restated the tier instead of ranking within it, so I took it out of the ranking.

A model attends to everything it is given, so each call asks one question. One reads the posting, one judges whether it is a real, specific job, one scores the fit, and for the strongest matches a second reader goes through the requirements as a sceptical screener.

How it is built

I design it and decide what ships

Coding
Built with Claude Code, Anthropic’s AI coding agent, under rules I wrote. I set what each change must do, read what changed, and decide what ships.
Tests
3,200+ automated tests (pytest) run before a change reaches the 05:00 run. Hooks stop the coding agent before actions that cannot be undone, such as deleting data, force-pushing or overwriting a file already sent.
Models
The daily reading runs on a low-cost model (DeepSeek). Every call is logged with its prompt, response, tokens and cost: 71,821 calls since 1 July 2026, US$50 in total.
Runs
Each run writes a manifest: the stages it ran, rows before and after, errors, and the database integrity check. Any score on the board can be traced to the model call behind it.

Where the numbers come from

Every figure on this page, with its source

All counts are read from the pipeline’s own database and run logs on 2 November 2026 by one read-only script.

Figure	Value	Source
Job postings in the database	43,648	Row count of the `jobs` table (SQLite). The live database starts 1 July 2026; earlier rows are archived.
Morning runs	166	One run manifest (JSON) per run, 16 May – 2 Nov 2026.
New postings, typical morning	372	Median of the last 31 mornings (range 90–1,170), rows added that day.
Read and scored, typical morning	233	Median of the same 31 mornings.
Run length, typical morning	92 min	Median of the same 31 mornings.
Model calls · cost	71,821 · US$50.42	The `llm_calls` table, 1 July – 2 Nov 2026.
Employers collected directly	59	Distinct employers in rows from career-site collectors.
Requirement quotes found	99.58%	`quote_found` over 129,656 stored quotes; 539 not found.
Automated tests	3,216	Passing in a full `pytest` run, 2 Nov 2026.
Fit-score audit	41 of 61	Top-tier postings at exactly 96 or 97, scoring audit of 1 Aug 2026.

Six steps, unattended, before the working day starts

Collect

Load

Filter

Read

Check

Report

A plausible quote is not the same as a real one

A score that looked precise and ranked nothing

I design it and decide what ships

Every figure on this page, with its source