All 439 faults from priorities.demo, grouped by the one change that would fix them together. Each fault sits in exactly one group. Groups run in pipeline order, since a mistake upstream corrupts everything after it. Within a stage they are sorted by combined importance. Open a group to see its faults; each links to its card with the live evidence.
The V5 engine's job. Testable in your lab today at localhost:8081.
6 groups · 118 faultsRead the paper as a tree (section > question > sub-part) keyed by the PRINTED label, not the ordinal; one question = one row.
Where this started: the p7 probe scored 3 of 9 checks.
Fixed 20 Sep — 20 fixes in the V5 reader, measured 29/29 papers live (27/29 on frozen readings), 170 tests. Not yet merged.
A mark is read with provenance or left empty (never defaulted), fractions allowed, and every total reconciled against the printed maximum.
Next group up, on the bench the tree group left behind. (The p7 mark faults quoted here were fixed with the tree work; what remains is the reader attaching a number to the wrong line.)
One representation for 'OR' / 'attempt any N of M' / section choice, carried from the paper through the key to the total and the denominator.
Mridul's V5 fork targets this group (F70, M010). Coordinate before touching it.
Crop and attach every figure to its question (and to the key), instead of describing it in words.
Classify every block before it becomes a question: instructions, answer keys, booklets, branding, watermarks, cover forms, rough-work pages, margin codes.
Validate and normalise the uploaded paper first: encrypted PDFs, phone photos, bad text layers, symbol fonts, rotation, 2-up scans.
Testable in the lab once the OpenRouter key is in.
3 groups · 55 faultsGround keys in a source for every subject, use an independent judge, and never let a low-confidence key mark a class unreviewed.
No grading without an approved key; key edits re-grade; every mark records the key version that produced it.
Schemes that allow alternative methods, units, synonyms, orderings and tolerances; the grader credits a valid route it recognises.
The grading pipeline. Not yet exercised in the lab.
9 groups · 187 faultsDeterministic, bias-tested grading that marks the student's answer (not the model's), and never turns a failure into a zero.
Check each upload: is it an answer sheet, for this paper, for this student, complete, in order, not a duplicate.
One source of truth for each mark and total, with bounds, rounding rules, and recompute-on-edit.
Store a transcription for every answer, and harden notation (maths, chemistry, units, ledgers).
Flags that are right, a queue ordered by risk, per-answer review, and teacher corrections that nothing overwrites.
Locate answers by the student's own labels with a real 'not found' state, never a zero.
Normalise pages (orientation, resolution, spreads, supplements) and link an answer across page breaks.
Feedback that matches the mark, sits under the right question, gives one next step, and has no LaTeX leaks.
Worked 20 Sep — a guard now routes feedback that contradicts its own mark to a teacher. Measured on only 24 real comments, so most of this group is still unmeasured. M153 was found to be WRONG: students see rendered maths; LaTeX leaks only to the teacher's box, the parent PDF and WhatsApp.
Test and support non-Latin scripts end to end: question paper, answers, digits, key and feedback.
Backend and infrastructure; mostly Mridul/Ashish territory.
4 groups · 60 faultsIdempotent re-grades, versioned outputs, per-tenant rollout, and alerts when something stalls or drifts.
A regression suite and real-teacher comparison, end to end and per subject, run before every release.
Authenticated access to scripts, consent, safeguarding escalation, and cheating/tamper detection.
Worked 20 Sep — signed expiring links for children's scripts, parent-report links, a who-changed-this-mark trail, delete now clearing what a script produced. The production bucket is still public: that is an AWS change, and it must follow the code, not precede it.
Nothing reaches a parent or an analytics chart until the mark is final, and a correction follows it everywhere.
Not code. Written policy the product needs before it can promise anything.
1 groups · 19 faultsFounder decisions to write down: who owns the final mark, review promises, disputes, retention, grace marks, disclosure to parents.
Written up 20 Sep — all 19 as decisions with options and a recommendation (POLICY-DECISIONS.md). Five block a school rollout.