Done is a claim. I check it.

Detail

• 6,967 non-merge commits in the main product repository, 2016 to 2024, range 644 to 929 a year. Counted 11/1/26, merges excluded.
• The 774-a-year figure above counts 2016 to 2024 only. R9's "about 9,850" and section 3's summary count a longer, different window: June 2014 (when the git record starts) to May 2025 (when AI tooling first appears in the repository).
• No 2025 commit carries an AI co-author mark. AI tooling first appears in the repository in May 2025; two July 2025 messages mention Cursor.

I moved over 1,000 managed domains myself and guided hundreds of client-managed domains through their own changes. When the next swarm pushed CPU near 100%, a block at the edge brought it back to about 20%. Second, auditing my own agent system. I asked whether a job an agent called done could be proved done. In a real week, 280 of 637 finished jobs (44%) had real evidence behind them; 8 more had a check recorded that didn't meet the rule; 349 had no check at all. That result is why I'm rebuilding how my system verifies finished work.

0 = none, 1 = introductory, 2 = working, 3 = advanced, 4 = expert.

• JavaScript

  August 2021: fixed a season rollover bug that duplicated football stat rows, by changing the server-side JavaScript that archives players and adding a 240-line staff debug page to find which rows to delete.

• SQL

  May 2019: wrote a 1,425-line reporting page whose queries count distinct customers, sum sales and left join several tables, to show sales by day, month and site.

• HTML and CSS

  April 2020: started the merged staff screen with 3,066 lines of new CSS, including 22 media-query blocks, replacing two separate screens.

• Production web servers and databases

  I built the production database behind the platform, including its tables, keys and indexes, and I've managed it ever since. I also wrote its original backup procedure, a date-based rotation that keeps backups closer together for recent dates and further apart for older ones.

• Python

  My Python is 2026 work, the measurement kit and agent tooling, written by AI agents under my direction. That's why I rate it 2.

• Recent stacks, built with AI under my direction

  TypeScript, React, Next.js, Node and Express, PostgreSQL. In 2026 I led the build of a multi-tenant business app with row-level database security, a tested ledger, and deploys that roll back if a health check fails. I also moved a legacy staff system to a React and Express API one slice at a time, while the old system kept running.

Same evidenceQ19

I call the Claude API directly in a staff assistant (Q20) and use Codex and Manus as second-opinion reviewers. My system connects to Anthropic, OpenAI, Gemini, DeepSeek, Groq and OpenRouter. It spends from prepaid plans first and touches metered credits only with my approval.

Detail

• Cursor appears in the product repository's commit messages and rules files from July 2025 onward. Claude Code is how the agent system and the measurement kit were built.

I ran it on a real week of my own agent system (Q27). I also run multi-model review: the same work goes to several AI reviewers and I judge their findings. It isn't a formal benchmark, but it shows me where each model is strong and where it misses. I haven't done red-teaming, dataset QA or benchmark design as formal work. Their core is what I already do: define what counts as correct, check results against evidence, and record where a method fell short.

Detail

• The kit's verify command runs its rules against 20 labeled cases: 17 of 20 agree, 2 lenient misses, 1 strict miss. Rerun: python -m agent_scorecard verify, 11/1/26.

Same evidenceQ27

Examples: finance, audit, trading, accounting, operations, law, engineering, healthcare, science, logistics.

And fluent Brazilian Portuguese.

Read more

• DXdev on GitHub
• agent-scorecard
• agent-guardrails
• agent-memory
• Verify, do not assume
• All green does not mean all tested
• Verify the AI reviewer's checkable claims
• A rubric verifier for autonomous workers

Written review is valuable to me when it serves a real purpose.

AI drafted it from my repositories and notes, a separate AI model reviewed each answer, and I made the final call on the wording.

In another class, I programmed a remote control car to follow a line of tape using LEDs and sensors, adjusting motor speed and direction as the sensor readings changed.

Third, rubrics for agent output: I define the rules and tests my agents are measured against. Fourth, payments for a small SaaS: I've run my platform's payment side, including PayPal and Stripe. Fifth, releases and incidents on a live site, which I've handled on the same platform.

Second, I built an AI agent system that multiplies my own development throughput. It runs my real development work, using Claude Code, Jira and GitHub, with hooks and required tests as the controls. My median went from about 5 to 7 resolved tickets a week up to about 28 over the last 26 weeks.

I built in controls: hooks that block known mistakes, required tests before anything gets committed, and a rule that metered AI spending needs my approval first. My median went from about 5 to 7 resolved tickets a week to about 28 over the last 26 weeks.

In my agent system specifically, a commit gate (tests that must pass before code can be committed) also runs tests on exactly the files being committed, every time.

Then a few layers of client review flag any skipped rows and say why.

I track cost per call against a price table I re-verify myself, plus daily token caps. On context limits, my agents checkpoint and flag themselves at set token levels, and an idle sweep compacts background agents and resumes them. Before I move work to a new model, it goes through a price and benchmark check first.

I fixed it in stages so I could tell what moved the number. First I added the two missing indexes on the production database: load dropped from about 20 seconds to about 1.5 seconds with no code change. Then I rewrote the query to use them, moved both sections to load after the page, and confirmed the same improvement on the test environment.

Detail

• A ticket with monitoring data and before-and-after load times, and one commit whose verification note records the staged measurements.

Mention unit tests, integration tests, edge cases, fixtures, reproducibility, or comparison against trusted outputs where applicable.

From the commit history: the 11/1/26 blog publishing change says 8 mutations proved red. The lane-manager state machine shipped with 33 tests and 5 mutations red. A verifier hardening change records 44 tests failing before the fix, 261 passing after, and 19 of 19 guard mutations red. Process and git code also gets separate integration tests run against real temporary git repositories.

Detail

• Hooks stop known mistakes at the moment an agent tries them, such as printing a credentials file into the conversation, a wait loop with no deadline, or a database query likely to scan a whole table. Four of them are public, with both blocked and allowed cases tested; 292 tests pass today.
• My measurement kit has nine test files; I reran them on 11/2/26 and 124 passed. Its fixtures are a seeded synthetic log, so every run is reproducible, plus 20 labeled cases used as trusted outputs for the verification rules. The privacy scan is tested by planting leaks. An AI wrote the 20 labels, so I had a second AI reader judge the same 20 cases without seeing any label: it agreed on 18 of 20.

Same evidenceQ16

Read more

• agent-guardrails
• agent-scorecard

A missing figure names the source it would need instead of guessing. Every percentage shows its count too, like 45 of 69, so nobody wonders what it's a share of. In my measurement kit, the input log carries a fingerprint of its contents so you can tell if it changed, timestamps without a time zone are rejected, and every version is in a dated changelog.

Read more

• agent-scorecard

Detail

• Client material stays out of public artifacts. My public tools run on synthetic data, and the measurement kit has a privacy scan that checks its files, file names and full git history. Two secret scanners run over that public repository found nothing.
• For AI tool uploads I have a written spending and data policy. Nothing uses a paid, per-call AI service without my explicit yes in that conversation, and background jobs never touch one. The one free-tier tool, whose provider may train on prompts, is barred from my product code, customer data and partner material. On a client job I would follow the client's own rules on which AI tools may see their files before using any of them.

Read more

• agent-guardrails

Detail

• The kit's method states it: a unit is trusted when an independent check recorded evidence of the outcome, and the verifier must not be the executor or the requester.
• My scorecard reports two numbers, and the strict one gives no credit for a finish that is only claimed.

Read more

• Related post: verify, do not assume

The rule is a plain text match against evidence fields, no model call. That's why I'm now rebuilding how my system verifies finished work, and part of that is already live.

Detail

• A second one: an AI judge scores my blog posts before they publish. I had it blindly re-score 10 posts three times each. Two of its four checks agreed every time. The other two didn't, and 3 of the 10 posts flipped between pass and fail across runs. Now each post gets three runs, the majority wins, and a check that still splits is marked unresolved.

Read more

• agent-scorecard

That is how my measurement kit works: six labels with written definitions, and 20 test cases written before the rules ran. An AI wrote those cases, so I had a second AI reader judge them without seeing any label, and it agreed on 18 of 20.

Read more

• A rubric verifier for autonomous workers

Detail

• For leaks into instructions, I separate two cases. My kit's checks are plain rules with no model call anywhere, so there is no prompt for an expected answer to leak into. For a model that reviews or grades, the practice is to hand it facts to check, never the verdict I expect. The same goes for the second reader on the kit's 20 cases: it saw each case's unit, event and evidence, shuffled and renamed, and never the label.

Read more

• agent-scorecard

A task that never passes is usually broken or out of reach. Those need different fixes, so I keep them apart. I also fix the seed and the as-of date, so a rerun differs only because of the model, and I have a second reader score a subset and compare their agreement with what chance would give. I learned this the hard way with my blog's AI judge: re-scoring 10 posts three times each flipped the verdict on 3 of them, so scoring now takes the majority of three runs. My measurement kit works the same way. It prints an interval on every sampled share and counts a finish as unstable if it was reopened or reverted within 14 days.

Read more

• agent-scorecard

That's how I set up the second reader on my kit's 20 labeled cases. It saw each case's unit, event and evidence, shuffled and renamed, and never the label, and it agreed with the labels on 18 of 20.

I score the two separately, so a style problem can't hide a missing outcome and a tidy answer can't hide a wrong one. The tricky cases are the ones that look cosmetic. An outside review of my kit found that a verifier name made only of spaces counted as a trusted pass. That looks like a whitespace nit, but it let an unverified finish count as verified, so I treated it as a real defect and fixed it in release 0.3.0.

Same evidenceQ28

Read more

• agent-scorecard

For dated records, I also check the date against the claim's timeline. You can see the habit on this page. The 774 a year in Q02 is 6,967 non-merge commits over nine years, recounted from git. The 44% in Q27 is 280 of 637, and I redid that division too. I do this with git, tickets and live pages every day, and PDFs and workbooks get the same steps.

A later correction replaces the original only when it points at what it replaces, and "cannot tell" is allowed to be the answer. I've run this on a real case (Q39): a ticket said STAGING/LIVE, but the live site still showed the old behavior, so the live site outranked the ticket until a merge and a second check showed the fix live.

Read more

• The Hotfix That Lied

I do the same to the numbers on this page. Before I put the kit's test count here, I reran its tests myself, and 124 passed on 11/2/26. For anything other people read, I edit the words myself. I cut what isn't relevant and replace complex wording with plain words.

Read more

• Verify the AI reviewer's checkable claims

My sampling audit has a "cannot tell" label for a request where the evidence doesn't show either way, and it never counts as finished. For a defect I haven't confirmed, I write down what I saw, what I haven't checked, and the check that would confirm it.

Read more

• agent-scorecard

I also keep a few items on purpose where a grader and a sensible human disagree, because they show where the grader is wrong. My kit does that: three of its 20 labeled cases are kept as known disagreements between the grader and the label, two lenient misses and one strict miss.

Read more

• agent-scorecard

Audit 1 and compliance 1: I designed an audit that scores agent work against dated evidence, and I run written controls on secrets and spending, which is the same evidence discipline but not a financial audit or a regulatory program. Treasury, investment operations, market data and legal documentation 0. Those are fields where I'd work alongside a specialist (SME06).

The main line had no trace of it because the merge, the one step that puts a hotfix into production, had never run. I didn't trust the ticket or my own checkout. I merged, then checked the live site from outside. The first check still showed the old behavior, and a check minutes later showed the fix live. Three dated facts settled it: when the commit landed, which branch it landed on, and what the live site served before and after the merge.

Same evidenceQ34

Read more

• The Hotfix That Lied

For example: spreadsheets, scripts, technical specifications, decision memos, source citations, and review reports.

My measurement kit is a good example of a full handoff set, and it's all public: a README, a method document, a schema specification, a glossary, a first-pilot guide and a dated changelog.

Read more

• agent-scorecard

If I think they're wrong, I say what I checked. When I raise a quality issue myself, I say what I saw, what it affects, how I know, and what I'd change, and I turn it into a changelog entry plus a fix. My kit has two cases where the reviewer was right. A reviewer pointed out that the README said there were no confidence intervals while the method depended on one, so I corrected the README in release 0.2.0. Another outside review found that a verifier name made of only spaces counted as trusted, and I fixed it in release 0.3.0 with a regression test.

Read more

• agent-scorecard

My measurement kit shows the handoff end to end: three dated releases, a tagged first release, and a "what this breaks" note on each so the next person knows what to change.

Read more

• agent-scorecard

I handled that in small steps the team could review, and at one point I sent a written list of what was still pending. It came to 275 commits, all mine, with roughly 89,000 lines added and 52,000 removed across source files, not counting built copies of the stylesheets. Recounted from git on 11/2/26.

Writing rubrics is third, because rubrics are how I decide what counts as done. After those come evaluating outputs, designing complex scenarios and leading QA.

I don't need ceremony, but I do like a clear written standard to work to.

My public measurement kit uses only synthetic data, and a privacy scan checks its files and full history before anything is published. I reran that scan on 11/2/26 and it came back clean, and two separate secret scanners found nothing on 11/1/26. I welcome independent review too. The whole kit exists to check work nobody has checked yet.

Same evidenceQ25

Read more

• agent-scorecard

A request submitted at 14:21 can only be approved at or after 14:21, 18 minutes after the revocation, so unless something changes the picture, the approver had no access when they approved. Before saying it firmly, I'd check five things. First, the approval event's own timestamp, clock and time zone, and whether it was a standing pre-approval given before 14:03. Second, whether the revocation was backdated, meaning its recorded time differs from its effective time. Third, whether that access screen is a snapshot, and as of when. Fourth, any later reinstatement or correction. Fifth, the audit log entry for the approver's action.

Here are four checks, each of which can pass or fail on its own. One, each request gets its own conclusion that states its submission time. Two, each conclusion cites a dated record of the approver's access at that time, not today's status. Three, the approval event's timestamp falls inside the approver's active window. Four, a request with no dated evidence is marked "cannot determine", not "approved".

Consider event status, effective time, date/time zone, and the relevant cutoff if useful.

Excluding only "rejected" still counts pending, voided, cancelled and duplicate events. Reversals and corrections get counted as new events instead of being tied to the entries they fix. And with no stated cutoff, events after the reporting date can slip in. To validate it more safely, I'd sort by effective timestamp in UTC, with an event sequence number to break ties. I'd accept only a named list of final statuses, remove duplicates by event id, link each reversal or correction to the entry it fixes, and state whether the cutoff is inclusive. Then I'd reconcile the result to a trusted opening and closing balance and write one test case for each failure above.