Treat prepayment review like a diagnostic test. An edit that holds a claim, or a fraud model that scores it, is doing what a screening test does. It sorts a large population into cases that are probably fine and cases that deserve a closer look. Medicine has a well-developed way of judging screening tests. Beyond asking how often a test catches what it is looking for, it asks what a positive result actually means given how common the condition is.
Payment integrity programs often report only one side of that ledger, the dollars stopped. That number matters. It is also incomplete, because it says nothing about how many honest claims were held to find the problem ones, or what holding them cost.
Base rates decide what a flag means
The key idea from diagnostic testing is that a positive result means different things depending on how common the condition is. A test can be accurate and still produce mostly false positives when the thing it looks for is rare.
Consider a prepayment model applied to 100,000 claims. Suppose it catches 90% of improper claims, its sensitivity, and correctly passes 95% of proper ones, its specificity. Those are good numbers for a screening tool.
| Scenario | Claims flagged | Improper claims among them | Share of flags that are improper |
|---|---|---|---|
| 1% of claims improper, 95% specificity | 5,850 | 900 | 15.4% |
| 0.2% of claims improper, 95% specificity | 5,170 | 180 | 3.5% |
| 1% of claims improper, 99% specificity | 1,890 | 900 | 47.6% |
This is a Redlake illustration that assumes 100,000 claims and 90% sensitivity in each scenario.
When 1% of claims are improper, the model flags 5,850 claims to find 900, so about 85% of the flags are false alarms. When the problem is rarer, at 0.2%, the share of flags that are real falls to 3.5%. Only a large improvement in specificity, to 99%, brings the share of real findings near half.
Base rates in payment integrity are often low. CMS’s improper payment measurements find that most improper payments reflect documentation problems rather than fraud, as we described in February. Deliberate fraud is rarer still. A model tuned to find fraud in a population where fraud is uncommon will, by arithmetic, flag mostly claims that are not fraudulent.
Most programs do not know their base rates precisely, and they can find out. Reviewing a random sample of claims, flagged and unflagged alike, estimates how common the problem really is and how many problems the screen misses. CMS’s own improper payment program works this way, measuring error rates from random samples rather than from the claims it chose to review.
False positives have costs
A false positive in payment integrity is not free. Each one means a held claim and a records request for a provider who did nothing wrong.
For providers, held claims mean slower cash flow and staff time spent answering documentation requests. Small practices, with the least margin and the fewest staff, feel it most. For plans, every flag consumes reviewer time. A queue full of false alarms crowds out the cases that deserve attention and teaches reviewers to treat flags as routine, the same alert fatigue that undermines clinical decision support.
A flag that is usually wrong is not a finding. It is a referral for a closer look.
Screening, then confirmation
Medicine handles the base rate problem by pairing screening with confirmation. A positive screening test leads to a more specific test before anyone acts on the result. Payment integrity can do the same.
- Never deny on a flag alone. A flag should route a claim to confirmation, usually review of the medical record by a qualified person, before any adverse action. The flag is the screening test. The record review is the diagnosis.
- Measure each edit’s yield. For every edit or model, track how many flagged claims confirmation finds improper, the equivalent of positive predictive value. An edit whose flags are rarely confirmed is consuming review time and provider goodwill for little return.
- Check what the screen misses. A small random audit of claims that were not flagged estimates the screen’s sensitivity. A screen that looks precise but misses most problems is not doing its job either.
- Use prior information. A flag means more for a provider with a history of confirmed problems than for one with a clean record, in the same way a test result means more for a patient with symptoms. Risk-based thresholds focus review where the prior probability is higher.
- Report the burden, not only the savings. Alongside dollars stopped, report the number of claims held and later paid in full, and the days of delay they experienced. Those numbers describe the program’s cost to honest providers.
- Target where base rates are high. Screening works best in populations where the problem is common. The surge in skin substitute billing that we described in March, with its newly enrolled single-product providers, was such a population, and targeted review there would produce far more real findings per flag than a general screen.
What we build for
We build tools health plans use in utilization review, and prior authorization faces the same arithmetic. Criteria screening sorts requests into those that clearly meet coverage criteria and those that need a clinician’s review. Our rule is that a machine can say yes but only a person can say no. That is the clinical approach to screening, where a negative screening result is never treated as a diagnosis on its own.
We also think every screen should be measured by what it finds. When a request is routed to a clinician, the record should show what the screen flagged and what the reviewer concluded. Over time, those records show which screening rules identify real problems and which mostly add work. Every recommendation shows its reasons, and every decision can be reconstructed, which is what makes that measurement possible.
A payment integrity program that knows the positive predictive value of each of its flags can make an honest trade between catching problems and burdening honest providers. A program that knows only how much it stopped cannot.