Catch it, then teach it
Derived from: the corrections corpus I maintain for my own build — one file per rule, each carrying the reasoning behind it, all loaded into every session automatically.
Two motions. Nothing in the default workflow makes you do either.
First: never accept an AI claim that touches your expertise without checking it. It will be fluent and confident when it's right, and fluent and confident when it's wrong. Your expertise is the only test that distinguishes them. Early on, a tool I was building produced a worked answer to an engineering problem and then graded its own answer "correct" — the number was wrong, and one of the forces was pointing the wrong direction. I caught it in about four seconds, not because I'm good at AI, but because I'd spent years on that exact kind of problem. The real question that moment raises isn't "how did it get that wrong." It's: what about every answer it's given me in the areas where I couldn't have caught it?
Second, and this is the half people skip: institutionalize the correction. Don't take the fix and move on — write the rule down so the system can't make that mistake again. Otherwise you re-argue the same correction every week, and you're the only thing standing between the AI and the same error, forever.
The format below is the one that made the difference for me: one file per rule, and every rule carries its why. A rule without its reasoning gets misapplied at the first edge case — the AI follows the letter and misses the point. A rule with its reasoning gets applied correctly to situations you never anticipated.
Have the conversation
This is the one to use. Fill in the four lines at the top, paste it into whatever AI you use, and it will open by naming three things about your idea it needs to know. It names the structure to the AI so nothing gets skipped, and tells it that every question has to be about your idea — if a question could be asked of any project, it's the wrong question.
I'm building something with an AI assistant, and it keeps getting certain things wrong — things I'd catch, but only if I'm looking. I want to build a corrections file: the rules it reads every session, so I stop re-explaining the same thing and the mistakes that matter in my field get caught. Here's what I know so far: My field: [YOUR FIELD] What I'm building: [WHAT YOU'RE BUILDING] Who it's for: [WHO IT'S FOR] What going wrong looks like: [WHAT GOING WRONG LOOKS LIKE] Don't ask me those again. Build on them. How I want to work together: - I'm the expert in my field. You're the expert in where AI assistants go wrong. Use that to ask sharper questions — but don't invent corrections you think I probably have. Ask. - Work through it in this order: where an AI is most likely to be confidently wrong in my field in a way a non-expert wouldn't catch; the corrections I've already had to make, and for each one WHY it was wrong in the terms my field uses; then the rules themselves; then how I feed the file back in every session. Use that structure so nothing gets skipped, but don't explain it to me. - Every question has to be about THIS project in THIS field. If a question could be asked of any project, it's the wrong question. - One question at a time. If my answer is vague — "it made a mistake" is vague — say so and ask again, until the reasoning is written down well enough that someone outside my field could apply the rule to a new situation. - Write each rule in this format: a short name, a one-line description, the rule itself, **Why:** with the specific incident and what it cost, and **How to apply:** concretely. One rule per entry, under 120 words. - When we're done, give me the file as one block, plus the two-line note I paste at the top of every session so the AI knows what it is. Then tell me which of my rules should become an automatic check instead of a rule, and what that check would be. Start by telling me the three things about my field you'd need to know before you could predict where an AI gets it wrong. Then ask me the first one.
What mine looks like
This is my version, genericized: the actual working document behind this rule. It's here so you can see what you're aiming at. You don't need to copy it — the prompt above does the work — but it's a real file from a real build, and if you want the raw material, take it.
-------------------------------------------------------------------------- ONE FILE PER RULE. Keep them short. Load all of them into every session. -------------------------------------------------------------------------- name: short-hyphenated-slug description: one line — enough to know whether this rule applies right now [THE RULE, stated as a rule. One or two sentences. Say what to do, not what happened.] **Why:** [THE SPECIFIC INCIDENT, and what it cost. Name what was actually wrong, not "it made a mistake." This is the field that makes the rule survive contact with a situation you didn't anticipate.] **How to apply:** [WHAT TO DO DIFFERENTLY, concretely enough to follow without judgment calls. If there's an anti-pattern, name it.] -------------------------------------------------------------------------- WORKED EXAMPLES — three of my real ones, generalized -------------------------------------------------------------------------- name: no-speculation-as-fact description: Don't state an unverified cause as fact — verify first When explaining what changed and why, verify against the actual source before asserting a cause. Mark genuine speculation AS speculation. **Why:** I was told a change in the output "was almost certainly" caused by a specific restructure. It was stated as fact. It was wrong — nothing had changed there; the real cause was something else entirely. I spent time reasoning from a false premise before catching it. **How to apply:** For any claim about cause, check the actual record first. "I don't know yet, let me check" is always acceptable. "Probably X" is acceptable if labeled as a guess. A confident wrong cause is not. --- name: check-the-source-not-your-memory description: Read the actual definition before using a name or value Before using a name, constant, field, or value from the system, read its real definition. Do not write it from memory or infer it from context. **Why:** Plausible-looking names were invented that didn't exist anywhere in the system. Everything looked right and nothing worked, and the invented names then propagated into a planning document, so the error outlived the original mistake. **How to apply:** Look it up first, every time. The five seconds of checking is always cheaper than the debugging. --- name: push-back-when-you-have-reasons description: Don't fold the moment I disagree — I need the expertise When I push back on a recommendation, don't automatically accept it. Re-evaluate honestly. If my pushback contains information you lacked, update and say which part changed your mind. If it doesn't, make the case again. **Why:** I'm the domain expert; I rely on you for the expertise I don't have. Folding the instant I disagree removes the exact value I'm here for. Mutual challenge is the point, not deference. **How to apply:** Never open with "you're right" and no reasoning. Either "reversing — the piece I missed was X" or "I still think Y, and here's the tradeoff you'd be accepting."
The graduation rule. A correction starts in this file. It graduates out when it becomes something stronger: a line in the permanent documentation, a check the system runs automatically, or a test that fails if the mistake recurs. When it graduates, delete it here and note where it went — the file is a working set, not an archive. If a rule keeps getting violated despite being written down, that's the signal it needs to become a check rather than a sentence.
The hard part
The hard part is writing the why, not the fix. The fix takes ten seconds and only ever solves the case you just hit; the reasoning is what lets the rule apply correctly to a situation you haven't met yet. If you can't articulate why it was wrong in the terms your field uses, you haven't finished the correction.
The first motion — verifying — only protects you where you happen to look. The second — writing the rule down with its reasoning — is what makes the system get better instead of just not getting worse. For a company whose entire institution is one person, that written record is the institutional memory, and it's the only asset that compounds while you sleep.