How I Caught a Multi-Client Blog Quality Drift in My Own AI Pipeline (and Closed It in One Afternoon)

TL;DR

A client emailed me a screenshot of their weekly SEO Health report. It said "36 days of no autoblog." That was wrong — my autopublish cron had been firing twice a week and emailing success notices the whole time. So I went hunting for the measurement bug. I found two problems instead of one:

1. **A measurement bug.** The audit was reading the wrong table. The fix took 15 minutes plus a 79-row backfill from Sanity. 2. **A real content quality drift.** While fixing #1, I audited every blog post my pipeline shipped in the last 30 days. **35 out of 48 posts across 6 clients failed the quality floor.** Thin word counts, missing hero images, zero internal links, no TL;DR or FAQ headings.

The fix shipped the same afternoon: a quality gate in the publish path that refuses to ship a post unless it has a hero image, 1,500+ words, a TL;DR heading, an FAQ heading, and at least 3 internal links. I also wrote a cross-client audit script that scans every client's Sanity project nightly and emails me a failure list.

This post is the postmortem. If you run a content pipeline across multiple clients, the same drift is probably happening to you. Here is how I caught it and what I changed.

What I was running

For background — I run an AI-driven content pipeline that publishes long-form SEO blog posts to six client sites. Each client has their own Sanity CMS project. Posts are published twice a week by a cron job that:

1. Pulls Search Console "strike distance" keywords for the client (positions 5 to 20 with real impression volume). 2. Writes a 2,500-word article using Claude with a specific copywriter voice (Halbert, Hopkins, Wiebe, Ogilvy, etc.). 3. Runs an E-E-A-T scoring pass and a banned-word filter. If the post scores below 7 of 10 it goes back for a rewrite, up to 2 retries. 4. Fetches a hero image and 2 inline images, watermarks them, uploads to Sanity. 5. Publishes the document to Sanity. The client's Next.js site picks it up on the next page request. 6. Fires social teasers to Facebook, Instagram, Threads, and LinkedIn.

That has been my pipeline for about 4 months. Roughly 90 posts shipped through it in the last 45 days. The whole thing runs unattended on Windows Task Scheduler. I get a summary email after each run. More on the framework behind it on the services page.

The first complaint

The first thing the client flagged was the Monday SEO Health report. The report's headline number said "36 days of no autoblog" — but they had received my success email less than a week ago. They asked, in essence: which one is lying?

When two pieces of evidence disagree, the right move is to find the table both of them are reading and check what it actually contains. So I traced the report code:

const pubRes = await pool.query(` SELECT domain, MAX(completed_at) AS last_pub FROM content_tasks WHERE status = 'published' AND completed_at IS NOT NULL GROUP BY domain `);

It reads `content_tasks`. I then traced the autopublish cron. The autopublish cron writes to `seo_alerts` and `distribution_tasks`. It does **not** write to `content_tasks`. The success emails were real. The audit was reading a table that the cron never touched.

That gap was 36 days old because that was the last time anyone published through the dashboard UI, which uses a different path that DOES write to `content_tasks`. The dashboard path had been quiet for over a month because all publishing had shifted to the cron.

**Fix #1:** I added a single `INSERT INTO content_tasks (...)` call to the autopublish script at the success branch, so future runs are tracked. Then I wrote a one-shot script to backfill the last 45 days from each client's Sanity project. 79 rows landed. Audit metrics now match reality.

This took about 25 minutes and three SQL queries.

The deeper problem

While I was inspecting Sanity to build the backfill, the client sent a second message. They had clicked through to a recent blog post that was thin — under 500 words, no hero image, no internal links, no TL;DR, no FAQ. They said: "this is the quality I keep seeing drift between clients. Why?"

Two things were now clear:

• The post existed in Sanity.
• The autopublish cron's quality gates would have rejected it. Every gate would have caught it: word count under 2,500, no image, no internal links, no FAQ heading.

So how did it get published?

I queried Sanity directly to look at the document IDs. The autopublish cron uses a deterministic prefix: `blog-autopub-{timestamp}`. The thin post had a different prefix: `blog-{domain-with-dashes}-{slug}`. That naming pattern came from a different file — `sanity-publish.ts` in the dashboard's library, used by the dashboard's "Publish to Sanity" button.

There were **two parallel publish paths into the same Sanity projects**, and only one of them enforced any quality gate:

Path Doc ID prefix E-E-A-T Image Word floor TL;DR + FAQ

------ --------------- --------- ------- ------------ -------------

Cron `blog-autopublish.cjs` `blog-autopub-...` yes yes yes (2,500) yes

Dashboard `sanity-publish.ts` `blog-{domain}-{slug}` no no no no

The dashboard path was an artifact from an older content workflow that pre-dated the cron. It still worked. Anyone hitting the "Publish to Sanity" button in the dashboard could ship a 200-word stub straight to a live client domain with no checks. The same architectural blind spot probably exists in most multi-client setups — see my AI automation services page for how I think about pipeline gates in general.

The audit that surprised me

Before patching the gate, I wanted to know how widespread the drift was. So I wrote a 200-line audit script that queries every client's Sanity project, scores every post in the last 30 days against the quality floor, and writes a markdown report. The first run pulled 48 posts across 6 clients.

**35 of them failed the floor.** Not just the obvious bypass-path ones. The breakdown:

• **3 bypass-path posts** — thin, no image, no headings. Exactly the pattern the client called out.
• **32 cron-path posts** — most failures were "internal_links < 3" and "no TL;DR heading." A few were "no hero image" (especially every CinCin post for 30 days straight).

The cron had been writing TL;DR sections in its markdown output, but the markdown-to-portable-text conversion was stripping out the heading style. Same story for internal links — they were in the markdown, but the conversion was dropping the `markDef` entries that Sanity needs to render them as actual links. And one client specifically uses a `heroImage` field on its `blogPost` schema (other clients use `mainImage` on the `post` schema) — when the image upload step failed silently, the publish proceeded with no image at all.

So the quality gates in the cron were scoring the source markdown, but the document that landed in Sanity didn't always carry the markdown's quality forward. The gate said "good content"; the published doc was "less than good."

What I changed

Three things, in order.

1. A quality gate in the shared library

I added a `validateContentMeetsProtocol()` function to the dashboard's `sanity-publish.ts` library. It runs before any Sanity mutate POST. If the content does not have a hero image, fewer than 1,500 words, fewer than 3 internal links, no TL;DR heading, or no FAQ heading, the publish call returns `success: false, error: 'blocked_quality_gate: ...'` and the database row is marked `status = 'blocked_quality_gate'`. The post never reaches Sanity.

The floor is intentionally lower than my cron's target of 2,500 words, because hand-curated content sometimes legitimately sits at 1,600 to 2,000 words. The point is to catch thin garbage, not punish edge cases.

An environment-variable bypass exists for genuine emergency hotfixes — but every bypass is logged. For the broader pipeline design I run for clients, the same gate philosophy applies across every service we deliver.

2. A cross-client nightly audit

I wrote `blog-quality-audit-all-clients.cjs`. It scans every client's Sanity project nightly, scores every post against the same protocol, writes a markdown report, and emails me a summary if any client has failures. If everything passes, no email. Quiet by default.

The script also tags every failure with its source path (`AUTOPUB`, `BYPASS`, or `OTHER`) so I can tell which pipeline is misbehaving at a glance. The first run today caught the 32 cron-path failures I would never have noticed otherwise. Details on the broader AI automation system here.

3. A backlog of cron-side fixes

I have a small list of follow-ups for the cron itself: fix the markdown-to-portable-text conversion to preserve link `markDefs` and heading styles; make the image upload step fatal on failure for the `heroImage` field instead of silently shipping an image-less post; force a default brand fallback image if no live source returns one. These will land over the next few days, and the audit script will tell me when each client's failure count drops to zero.

Why this matters if you run an AI content pipeline

If you batch-publish AI-generated content across multiple sites, here is what I learned the expensive way:

1. **Every publish path needs the same gate.** If you have a cron AND a dashboard button AND a CLI script, all three need the same minimum quality validation. Otherwise the slop ships through the unguarded one. Centralize the gate in a shared library that all three call. 2. **A scoring gate on the source is not a scoring gate on the destination.** If your gate scores markdown but your transform-to-CMS step quietly drops content, your gate is decorative. Score the destination doc too — that is what the audit script does for me. 3. **The audit telemetry has to be cross-client.** Per-client dashboards hide cross-cutting regressions. The "every post for 30 days has no image" pattern was invisible until I queried all clients at once. 4. **Measurement bugs and quality bugs look identical from the outside.** "The dashboard says zero posts" and "the posts are all bad" both look like "the pipeline is broken." The only way to tell which one you have is to read the actual table the dashboard reads, then sample the actual docs the pipeline produces. 5. **Build the audit before you trust the gate.** Gates fail open. Audits catch what gates miss. Always have both.

The whole stack — gate, audit, backfill, postmortem — took about 4 hours of focused work. Most of it was reading code, not writing it. The audit script will catch the next drift in the first 24 hours instead of after 6 weeks.

FAQ

How do you keep AI-generated blog content from drifting in quality over time?

Run two independent layers. Layer one is a gate at every publish path — a function that refuses to ship a post unless it meets a clear quality floor (word count, hero image, internal links, FAQ heading). Layer two is a cross-client audit that scans your live CMS daily and emails you when any post fails the floor. Gates fail silently; audits notice.

What is the difference between a measurement bug and a quality drift in a content pipeline?

A measurement bug means your dashboard or audit is reading the wrong source, so it reports failures that are not real or hides failures that are. A quality drift means the content itself is degrading — even though your dashboard says everything is fine. Both look the same from the outside ("the pipeline seems wrong"). To tell them apart, sample the actual artifacts the pipeline produces and compare them to what the dashboard claims.

How can I tell if my AI content pipeline has multiple publishing paths?

Check the document IDs in your CMS. Most automation scripts use deterministic prefixes (timestamps, slugs, doc-type tags). If you see multiple distinct prefix patterns for the same content type, you probably have multiple publish paths, and at least one of them is not enforcing the same gates as the others.

What is the minimum quality floor I should enforce on AI-generated blog content?

For SEO-targeted content, my floor is 1,500 words minimum, a hero image, a TL;DR or Key Takeaways heading, an FAQ heading, and at least 3 internal links to other pages on the same domain. That is the floor — not the target. My cron target is 2,500 words. The floor exists to stop the worst drift, not to define what good looks like.

How often should I audit live AI-generated content?

Nightly is the right cadence for a multi-client setup. The cost of a nightly Sanity query against six projects is essentially free, and a missed publish for a week is far cheaper to catch on day 2 than on day 30. If you run a single site, weekly is fine. If you run more than six clients, you probably need a real monitoring system, not a shell script.

What should the audit email contain?

Just enough to know whether to act. I include a table per client showing total posts, pass count, and fail count, plus a sample of failing slugs. If everything passes, the script does not send the email at all. Silence is the success signal. The full per-post detail goes into a markdown report file that I only open when the summary tells me to.

How should I remediate live posts that already failed the quality floor?

Do not delete them — that loses the data and the link equity. Patch the document's slug in your CMS to an `archived-{original-slug}-{date}` prefix so the post no longer serves at its live URL but is recoverable. The URL goes 404 (or, where a gate-compliant twin shares the slug, the twin serves uncontested). For posts with measurable traffic or external links, add a 301 to a relevant hub page or the gated version. I archived 6 live posts this afternoon using this approach in a single one-shot script that hit the Sanity mutate API per project. Total time: about 5 minutes.

Next step

If you want this audit script for your own pipeline, drop me a line through the contact page and I will share the source. It is about 200 lines of Node.js with no dependencies beyond the Sanity REST API and an optional Resend email key.

The bigger lesson — and the one I will be tightening across every client this week — is that automation buys you reach but costs you visibility. If you cannot see what your pipeline shipped overnight, you will not notice the drift until a client points to a thin post and asks why. The audit script is now what sees for me.