How Independent Auditors Catch AI Bias Companies Miss
Why companies miss AI bias and how independent auditors catch it, from Workday to a 4-million-application Stanford study and Mount Sinai's AI triage test.
Companies miss AI bias because the builder tests its own system, once, on its own data, against the metric it trained for. Independent auditors catch it by testing on the people the system actually serves and splitting results by group and by context. Then they test again after launch. The cases on the record right now point the same way. The people who find the bias rarely work for the company that built it.
Why do internal teams miss it?
The applicant runs the first real test. Derek Mobley says he was rejected from more than 100 jobs screened through Workday before he sued in 2023. In July 2026, the court pushed class certification to March 2027 after "Workday recently produced three bias evaluation reports." No court has ruled that bias occurred. The vendor's internal testing now sits in the evidence file.
Every bias test you run today is a document someone may subpoena tomorrow.
One company's data hides the pattern. In May 2026, Stanford researchers analyzed 4 million applications to about 150 employers, all screened by one vendor's AI. Under the EEOC's 4/5’s rule, 26% of Black applicants applied to positions where the system discriminated against their group. And 10% of people who applied four times were rejected everywhere. That pattern only exists across employers, so no single employer could see it.
The test never varied the context. Mount Sinai researchers stress-tested ChatGPT Health with 60 clinician-written cases under 16 conditions, 960 responses in all. It under triaged 52% of true emergencies. Race and sex showed no significant effect. When a family member minimized the symptoms, recommendations shifted sharply (odds ratio 11.7). A demographic-only bias test would have passed it.
Bias hides in the context your test never changed.
Neutral-looking inputs carry the bias. In July 2025, student lender Earnest paid $2.5 million to settle Massachusetts claims that a variable built on colleges' loan default rates, plus a green-card knockout rule, disadvantaged Black and Hispanic borrowers. Earnest denied wrongdoing. Nothing in the model was labeled race.
The builder picks the grade. A FAccT 2024 study of 391 New York City employers found only 18 had posted their required bias audit, and 96% of those posted showed passing results.
How does the right auditor catch it?
I laid out four checks in my response to Stanford's white paper for Congress. They apply to any organization.
- Separate the builder from the checker. The reviewer earns nothing when the system passes and did not design the controls. Michigan Medicine did this with a widely used sepsis model and found it caught 33% of actual sepsis cases among its own patients.
- Test on the population you serve. Compute selection rates by group and by intersections such as age with race. Federal guidelines treat a rate below four-fifths of the top group's as evidence of adverse impact, and smaller gaps can count when statistically significant. Pair impact ratios with significance tests. Trace inputs such as zip codes and default rates for correlation with protected groups, then vary context the way Mount Sinai did. Language models answer the same prompt differently, so our Sentinel Index runs every variation 10 times.
- Put an expiration date on every result. Learning systems drift. Retest on a fixed cadence and after every model or vendor update, with continuous monitoring such as Fusion Sentinel between audits.
- Name who answers for it. A rule without an accountable party is a suggestion. Put a name on every model and every finding.
What should you do from where you are today?
- Exploring AI. Write down who each decision will affect and what fair looks like for them before you pick a tool.
- Buying a tool. Ask the vendor for its bias testing method, the population it validated on and the date. If that population doesn't match yours, the results don't transfer.
- Already deployed. Pull 90 days of your own outcomes and compute impact ratios by group this month. It is the fastest first look at whether you have a problem.
- Scaling or regulated. Commission an independent audit against ISO/IEC 42001 with technical bias testing and set a re-audit date before you sign off.
Checkbox theater ends where independent verification begins. Behind every percentage point is a person.
Share this article
Related Articles
The Reskilling Illusion: When AI Transformation Means "You're Fired"
Oct 03, 2025
