Human oversight by design
Segregation of duties with AI in the mix, choosing review-all versus sample-with-teeth, and the anti-rubber-stamp mechanics that keep oversight real as volumes grow.
'Human in the loop' is a phrase that decays into a checkbox unless it's designed: which human, reviewing what, with what attention economics, caught by what metric when the review goes hollow. Finance has run this problem for a century under the name segregation of duties — the discipline transfers directly, with AI as a new actor in the grid:
- AI as preparer (the default): extraction, drafts, matches, flags. Then the classic rule holds unmodified — preparer ≠ approver — with AI in the preparer seat and a named human approving. The subtle violation to watch: AI both prepares the payment file and generates the exception report on it. Same system, both sides of a control = one system agreeing with itself (the Module 2 independence rule, now as formal SoD).
- Review-all vs. sample-with-teeth — the honest capacity decision: below certain volumes, review everything (JE drafts, external sends, anything material). Above them, sampling is legitimate if it has teeth: risk-weighted selection (100% of high-score flags and material amounts; random N% of the rest), documented method, and consequences that scale — one caught miss triggers expanded review of that category. Sampling without the escalation rule is hoping with a methodology section.
- Thresholds move with evidence, not with fatigue. The auto-release limit ($X payments below which no human release is needed), the auto-match tier, the auto-disposition of trivial flags — each threshold started conservative and may loosen only on measured performance ('4,000 auto-matches, sampled monthly, zero errors → raise tier'). Write the threshold history down; 'why is auto-release at $2,500?' deserves an answer with dates and data, because an auditor will ask and so will you in two years.
The rubber-stamp problem, measured
Every review step trends toward theater as trust grows — approval rates drift to ~100%, review times drop to seconds, and the control still exists while no longer operating. You've met the metric twice (automation gates, strategy-course governance); in finance it becomes a formal control-health check: track approval rate and median review time per reviewer per control. Sustained ~100% approval means either the AI has earned a narrower gate (good — move the threshold, keep sampling, document it) or the review has hollowed (bad — and fixable: rotate the duty, inject known-bad seeded items and see if they're caught, shrink the queue so attention fits it). The seeded-item test — slipping a deliberate error into the review stream quarterly — is the single most honest measure of whether your human-in-the-loop is a control or a costume. Auditors love it; reviewers respect it once it's framed as testing the system, not the person.
A reviewer with 400 items in the queue and an hour to clear it is not a control — arithmetic already decided the outcome. When volumes grow, the honest options are: more reviewer capacity, higher automation tier (with the evidence to justify it), or narrower scope. The dishonest option — same queue, same hour, quieter conscience — is how organizations discover post-incident that their oversight was load-shedding for a year. Review capacity gets planned like any other control resource, and the queue-depth metric belongs on the same dashboard as the approval rate.