What an AI system must show before it’s relied on, and what happens when it’s wrong
A framework for deciding when an AI system may be used, who can challenge its decisions, and how errors get corrected.
The products are built to it: NextConsensus keeps track of what the evidence currently says, Ambit of what an agent has actually been approved to do, and Refract of what has changed underneath. Each one writes down something AI systems in hospitals and similar settings usually leave in people’s heads.
The method
Every AI system in a setting where an error can hurt someone makes claims it does not state: that a guideline is current, a drug protocol is safe, or a patient needs immediate prioritization.
A claim can be checked when you can see what backs it, when you can challenge it, and when a correction reaches the systems that relied on it.
Most AI governance work starts with the operating model — who approves what, under what conditions, for how long. This framework starts earlier: how do you know what you know, and how is an error repaired?
NextConsensus freezes each forecast before the outcome is known. Refract keeps source changes verifiable, and Fast Harm, Slow Repair applies the same discipline to recovery evaluation.
Ambit builds the operating model into the software agents run on: what an agent is approved to do is kept apart from what it can reach, every change it makes carries its own undo, results are checked before anyone trusts them, and each tool has a fixed scope.
Ethotechnics publishes ten proposed standards, each numbered and versioned. Every one maps itself to NIST AI RMF, ISO/IEC 42001 and the EU AI Act in a section of its own — the mapping is mine, not an endorsement by those bodies. No outside institution has adopted one yet, so they remain proposals. The one the projects on this site share is STD-07, the Revisable Delegation Record, which has a conformance checker.
What the standards bind
The twelve lawsEach law is a rule the standards enforce: capability does not imply authority, authority decays unless renewed, every delegation creates a correction obligation, and nine more.
The one rule underneathOne sentence the laws come down to: no system should gain the power to act on people faster than the institution gains the ability to inspect, challenge, change, and survive its decisions.
Core axiomsThe five principles the laws come from, and why each is owed to people rather than offered to them.
TheoryEssays on why the laws hold. Motivation only: no standard cites one as a requirement, and a reader can adopt a clause without the argument.
Principles
What I require from the teams I lead. The beliefs these implement live on /about/; each principle links to the work it came out of.
01Human override is a product feature.
Override, escalation, and return-for-review decide whether a system that can hurt someone can be deployed safely.
Recurs in two deployments
AndwiseAutomated clause analysis surfaced and explained each issue. A named reviewer checked every analysis before a physician could act on it.
EthotechnicsThe stages of approval written down as an open standard: who may override a system, and what they have to record when they do.
02Provenance beats confidence scores.
Knowing where a recommendation came from and what evidence backed it matters more than a confidence score nobody can audit.
Recurs in two deployments
RefractEvery change replayed into a verifiable event carrying its provenance, with the judgment left to the caller.
NextConsensusA public ledger of forecasts registered and frozen before the outcome is known, scored against the record afterwards.
03The decision the AI feeds into matters more than the AI’s output.
When an automated system causes harm, ask who owned the decision the model fed into, not only whether the model was wrong.
Recurs in two deployments
EpicInstalled, signed off, and live — and still not doing the job, because nobody owned what happened after the go-live.
The Crumple ZoneEssays on the gap between an automated recommendation and the person who has to carry it out.
04Governance belongs in the product.
Evaluation, monitoring, escalation, and correction work best built into the product’s daily workflows, not handed to a committee outside them.
Recurs in two deployments
EthotechnicsApproval, correction, and escalation published as open, versioned standards.
AndwiseCompliance review was a routed step in the flow, signed off by a named reviewer.
05A deployment is finished when corrections reach every record it touched.
An error corrected at the source remains active everywhere it already propagated.
Recurs in two deployments
RefractDownstream systems receive what changed and when.
Fast Harm, Slow RepairA protocol that measures how far a wrong output travels before the correction catches up with it.
06Evaluate workflows before models.
A model that performs well in evaluation can still fail at the integration point where busy clinicians have to use it.
One instance so far
TranscarentFour programs shipped on one shared decision architecture because routing was the binding constraint.
07You can get an approval later; you cannot undo a harm.
When the risk is clinical, launching early costs more later than it saves now.
One instance so far
EpicThe escalation route that had to exist before the software could honestly be called live.
08Hospital IT trusts the system around the model.
Hospital IT, security, and legal teams review the system around a model — access boundaries, audit trails — as closely as the interface.
One instance so far
DoximityHospital security reviews cleared the product on documented policies and procedures.
The operating model
AI deployment requires continuing authorization: when a system may influence decisions, what conditions limit that authority, how people can challenge it, and what evidence or behavior reopens review.
A model recommends a treatment change. At every state, the authorization is a dated record that can be superseded — never a permission granted once.
State transitions for the treatment-change recommendation. Rows are in order; a state is entered only when the row above it exits. The last row shows
three of the six review steps, one per column; the inspection guide has all six.
State
Trigger
Who acts
Evidence required
What propagates
State 0Not authorized
A use is proposed.
Named deployment and escalation owners
Defined clinical or operational decision
Named deployment and escalation owners
Specified affected population and exclusions
Nothing. The proposed use cannot influence care or workflow.
State 1ObservedThe recommendation is observed but cannot influence care.
State 0 exits when A bounded use case, accountable owner, and evaluation plan are approved.
—
Silent or retrospective evaluation
Error taxonomy and exception review
Baseline comparison against current practice
Nothing to care or workflow. Outputs are captured for evaluation.
State 2Constrained useAfter a silent evaluation, it may inform a narrow workflow under human review.
State 1 exits when Observed performance and failure modes justify a limited prospective deployment.
Explicit human review on each recommendation
Prospective workflow validation
Documented override and escalation paths
Monitored safety, equity, and operational indicators
Recommendations into one narrow workflow, within limited population scope and predefined stop conditions.
State 3Routine use
State 2 exits when The deployment performs acceptably inside its stated scope and the institution can pause, correct, or roll it back.
Independent review, with challenge rights preserved
Stable prospective performance
Independent evaluation sized to the risk of the use
Operational readiness for correction and rollback
Routine use within the defined scope, watched for change and reviewed on a schedule.
Any stateReopenedA new safety signal suspends the authorization immediately, from whichever state it had reached.
01 Detect Notice a change that could matter — in the evidence, the model’s behavior, policy, the data, the workflow, the people it is used on, or the outcomes being observed.
The clinical safety lead opens the review and makes the decision.
04 Review → Case for review Give the people allowed to decide the change (traced to its source), the original reasoning, how the system has actually performed, any objections, and what is still unknown.
06 Propagate → Everything that relied on it, corrected Update the approval record, and every workflow, instruction, screen, monitoring rule and person that relied on the old decision.
The loop opens on a change, not on a review cycle. A system that can only be reviewed on a schedule cannot be corrected in between.
Decision-readiness inspection guide
What an automated decision must demonstrate before an organization can rely on its outputs. Derived from the operating model above: the three tests it must pass, the eight lines of the authorization record, and the route when facts change.
1 · Three tests a deployment must pass
Explainable
The people relying on a deployment can inspect the basis for its current authorization: intended use, evidence, assumptions, known limits, and accountable owner.
Challengeable
A clinician, operator, patient advocate, or reviewer can contest a result or deployment condition through a defined route that does not depend on informal access or personal discretion.
Correctable
When evidence, policy, performance, or operating conditions change, the institution can review the deployment, make a new decision, and carry the correction to every workflow it affects.
2 · The authorization record
Eight lines a reviewer should be able to read off one record before authorization. If any line is unanswerable, the deployment is at State 0 (Not authorized), regardless of model performance.
The decision or workflow being supported
Authorized users, population, setting, and exclusions
Current model, prompt, policy, data, and integration versions
Evidence and assumptions that justify using it
Known limitations and required human checks
Named deployment, clinical, and escalation owners
Effective date, review date, and superseded authorization
Stop conditions and rollback route
3 · When the facts change
Before authorizing, confirm each step of the review loop has a named owner and propagation route, not just an assumption that someone will notice.
Detect — Change signal
Notice a change that could matter — in the evidence, the model’s behavior, policy, the data, the workflow, the people it is used on, or the outcomes being observed.
Connect — Map of everything that depends on it
Trace which of the approval’s assumptions, which people, which workflows and which later decisions the change could affect.
Triage — Review priority and interim controls
Judge how much it matters, how urgent it is, whether it can be undone, how many people it reaches, and whether the system can keep running while the review happens.
Review — Case for review
Give the people allowed to decide the change (traced to its source), the original reasoning, how the system has actually performed, any objections, and what is still unknown.
Decide — Recorded decision
Make the decision and write it down: keep, caveat, narrow, expand, monitor, escalate, suspend, or retire.
Propagate — Everything that relied on it, corrected
Update the approval record, and every workflow, instruction, screen, monitoring rule and person that relied on the old decision.
How the principles developed
Accountable routing and escalation
First observed 2012
Escalation paths enforced in software, so no clinical or operational error stalls without an owner.
2012Epic
Co-led CMS PQRS and built its federal quality escalation path, unblocking stalled reporting.
View record →
2021Transcarent
Built one exception queue across specialties, with cases routed by clinical role.
View record →
2025Essay
Argued in “Toothless Ethics” that escalation has to be built into the software, not left to good intentions.
View record →
2025Ethotechnics & Ambit
Wrote the review loop down as a state machine, and made the limits on what an agent may do something a program can check.
View record →
When the evidence, a policy or a model changes, making sure everything that relied on the old answer is updated and corrected — not only the system that changed.
2011Georgia Tech RNA
Modeled thermodynamics and barrier kinetics deciding whether a molecular process completes or stalls.
View record →
2024Refract & NextConsensus
Built deterministic change-detection bots and claim-trajectory scoring against public records.
View record →
2025Fast Harm, Slow Repair
Designed a draft test for how long harm lasts and how far a correction reaches; it covers one of twelve planned cases.
View record →
What changes when a decision that used to end with a person ends in software instead: where errors travel, who absorbs the friction, and whether the system can repair itself when the information under it changes.
Three cases, and which of the questions each one asks
Case
Mechanism
Who absorbs the cost
Which questions
Emergency department loudspeaker
Patients are called for triage by name over a loudspeaker. For a Deaf patient the mechanism deciding who waits is inaudible.
The patient; triage delays compound while the patient has no way to detect or contest the queue.
Who waits?
What cannot be allowed to fail?
Epic: a misrouted clinical alert
A misrouted alert could bury a critical lab result.
Whichever clinician trusted the queue; nobody in the routing system answered for it.
Who absorbs error?
Who covers for the system?
Andwise: monetization
The monetization paths most likely to fund growth would have made employers or financial institutions the customer.
Field observation: how recommendations shift clinician judgment
In 2012, working on clinical decision support for type 2 diabetes at Georgia Tech and Emory, we ran an early pilot measuring how clinicians revised their recommendations after seeing an algorithm’s output. What stood out was not whether the recommendation was right, but what happened to the clinician’s own judgment once they had seen it.
Internists n=262%→92%change +30 pts
Familiar with the rules n=464%→86%change +22 pts
Endocrinologists n=468%→76%change +8 pts
Unfamiliar with the rules n=271%→71%change +0 pts
Agreement with the algorithm, before and after the clinician saw its recommendation. Pilot validation with six clinicians across twenty patient data sets (Georgia Tech / Emory, 2012). Percentages are rounded; the subgroups nest rather than partition, so the reviewer counts do not sum to six. Rows are ordered by how far each group moved.
Internists’ agreement rose more than endocrinologists’ agreement. Reviewers familiar with the rules shifted more than those unfamiliar with them. These are observations from six clinicians, with overlapping subgroups.
In this pilot, clinicians’ recommendations shifted toward the algorithm after they saw its output. The framework on this page requires accountable human review before automated outputs become institutional policy.
The rule the measurement above argues for. Software can update what the evidence says; only a
named person can change what the institution does, and that change is recorded. “No action” is one of the choices, because a system where
declining to act is not a recordable choice will drift into acting by default.
Jain, K., Patel, K., Rowland, J., Yong, C. DiaMonD (DIAbetes MONitoring and Dosing) System. Wallace H. Coulter Department of Biomedical Engineering, Georgia Tech / Emory University. Advised by Dr. Lawrence Phillips, MD.
Go deeper
Case studiesWhere this came from: five deployments through hospital security review, federal quality reporting, and clinical sign-off, with the decisions and contemporaneous records behind them.
PrinciplesWhat 14 years of clinical products left me unwilling to ship without.
Current workNextConsensus, Ambit, and Refract: the living implementations of these mechanisms in active clinical and agent infrastructure.