Case study · a weekly data pipeline, rebuilt to survive

STATFINDER.

A weekly data pack for one reader. Every week it assembles NFL and college-football schedules, Sagarin ratings, betting odds, team stats, and against-the-spread records into pages a private client reads to make his picks. A paid engagement, delivered every week of the season. In July 2026 I rebuilt the whole thing, in four days, so it could survive a second season.

In production · private client deployment 242 tests, all passing 4 days, 5 phases
01

What season one left behind

Season one delivered its packs every week. It also shipped as a single-league codebase hastily forked into two, held together by a scheduled job that committed data into the repository twice a day. The audit, fact-checked claim by claim, wrote down the bill.

DEBT / 01

24 confirmed bugs

The count came from an adversarial pass over the season-one tree, trusted over the code's own comments. Several were not theoretical. They had been quietly costing the prototype output all season.

DEBT / 02

Two copies of one pipeline

The NFL and college paths were divergent copy-paste twins of the same logic, drifted apart by a season of hotfixes. Nearly every module existed twice.

DEBT / 03

The repo was the database

The data model was "commit the output directory twice a day." Roughly 1,450 tracked data files were auto-committed on a cron, bloating history and making the repository its own storage layer.

DEBT / 04

A secret in plaintext

A live Discord webhook token sat in a tracked file, and in git history. Extraction to a clean repository later, with none of that history carried over, was the only real fix.

DEBT / 05

Config backwards

The .env file silently overrode real environment variables, the reverse of every convention, so the process trusted the wrong source without saying so.

DEBT / 06

Tests that never ran

Twelve tests covered forty-three modules, and CI never ran them. Observability was grepping standard output for a NOTIFY: string.

02

One pipeline, not two

The plan was a strangler rebuild: build a new package beside the untouched old one, prove it produces the same output, then swap it in. The design corrections were deliberate reversals of the season-one sins.

A single league-parameterized package replaced the two forked copies. One League dataclass drives both sports through one game-key builder, one output layout, and one rating-versus-odds formula. Config became typed, with real environment variables beating .env instead of the other way around. Errors fail loud. Every stage writes a receipt, and each run emits a JSON summary instead of printing a line for a human to grep.

Phase one built that package in a single day. Four parallel agents took the pieces while the foundation and orchestration stayed in the main loop. By the end of the day it was passing 177 tests.

03

Proving it made the same thing

A rebuild is only trustworthy if it provably produces what the old system produced. So the next phase was a parity gate, and the standard was zero unexplained differences.

An offline replay harness rebuilt four real season-one weeks through the new pipeline (NFL 2025 weeks 16 and 17, college weeks 13 and 14) and diffed every artifact against the archived season-one output. Every accepted difference was written down. The gate passed on July 3: four weeks, zero unexplained deltas.

The gate paid for itself in what it caught. It proved that NFL odds promotion had been silently dead for the entire 2025 season, a pinned-key mismatch that dropped the paid odds feed on the floor. College promotion was dead for every multi-word team name; only 5 of 60 college week-13 games had ever carried odds. The pages render blanks without complaint, so season one never announced what it was missing; the parity gate is what finally did. The same gate caught three bugs in the new code before they could ship.

WEEK VIEW / NFL 2025 ● LIVE TOOL
Statfinder Week View for NFL 2025: a dense table of games with betting odds, over/under, win-loss records, computed power ratings, a rating-versus-odds edge column, and schedule strength
The Week View, once promotion works. Each game carries its odds line, the over/under, records, the computed power rating, and the rating-versus-odds column that is the whole point: where the computed number and the market's number disagree. Every visible row exports to CSV.
04

The live tool

With the package proven, the season-two tree moved to a fresh private repository with no webhook token anywhere in its history, and the data-in-git model died with it.

It went live as a private web tool: the client's access is now a page he opens in the browser, not a published artifact of committed data. SQLite writes alongside the flat files, and an export command reproduces any week's files byte-for-byte from the database.

GAME VIEW / MATCHUP ● LIVE TOOL
Statfinder Game View for a single matchup: favorite and spread, the market line against the computed power rating, a full team statistical snapshot, and both teams' full season schedules with scores
The Game View for one matchup: favorite and spread, the market line set against the computed power rating, a full team snapshot, and both teams' entire seasons with scores and per-week ratings. This is the page the client opens when a line looks wrong.
The tool is private, client access only. The UI is plain on purpose; the views are the product.
05

Made to run itself

The last phase made the system safe to leave alone through a seven-month offseason and a season it reports on twice a day.

THE OFFSEASON GUARD

Skips cleanly instead of rebuilding forever

A guard reads each league's real kickoff windows and classifies every scheduled run as in-season, preseason, or offseason. Out of season it skips everything, costs nothing, and posts nothing, so the schedule is safe year-round. Rebuilding the final week forever was a season-one bug.

THE NOTIFIER

Reports shaped so nobody learns to ignore them

Every run reports to Discord, outcome-differentiated on purpose: a failure is loud and red with the failing stage's error up front, a success is one quiet green line, and a fully-skipped offseason run says nothing at all.

THE SCHEDULE

Pinned, gated, and on a real timer

Dependencies are pinned and a CI test gate runs on every push. A Windows scheduled task, statfinder-refresh at 06:15 and 18:15, replaced the old scheduled-job cron. The README was rewritten as an operations runbook.

06

What's proven, and what waits

Honest state, not a victory lap.

Proven offline

Four weeks of parity with zero unexplained differences, and a suite that grew from 12 tests to 242, all running in CI. The rebuild's correctness is measured, not asserted.

Waiting on the season

The first live-fire run against real feeds can't happen until a game week opens, because the odds API only returns upcoming events. The three pages still need a human visual pass beyond the machine checks.

Documented, not solved

The commercialization questions are written down and left open on purpose: the ratings sources carry no license to resell, and the pipeline still assumes a single tenant. Those are business decisions, not rebuild tasks.

07

The point

Statfinder is a small system with one paying reader, rebuilt to the standard of something much larger: audited, parity-proven against its own past output, self-scheduling, and honest about what it hasn't proven yet. The most valuable thing the rebuild produced was the discovery that the old version had been quietly broken all along. If you have a pipeline that fails without telling you, this is the work that finds it.