Every implementation guide should ship with test data. A FHIR implementation guide tells software how to exchange a kind of health information. It defines the resources and the profiles, and it describes the steps each side takes. What it rarely includes is a shared set of patients and expected answers that every implementer can run.
That gap shows up at the worst time. Payers must run prior authorization APIs built on the Da Vinci guides by January 2027. Providers in a February survey named coordinating tests with vendors and health plans as their second biggest barrier, according to healthsystemCIO. Each pair of partners is building its own test cases, one partnership at a time.
What test kits check and what they miss
Test tools exist. ASTP released an open-source test kit for the Prior Authorization Support guide in 2024, built on its Inferno framework, according to ASTP. It tests both the provider client and the payer server against the guide’s required operations and profiles. Our piece on HTI-4 described how EHR certification now covers the provider side of the exchange.
Guides do include example resources. An example shows what a message looks like, not what answer it should produce. A test like that checks form. It confirms that a request has the right structure and that a server answers in the right shape. It cannot confirm that a payer’s rules produce the right answer for a given patient. Nor can it confirm that an EHR fills in a documentation form the way the payer expects.
Those are the failures that matter in production. A request can pass every structural test and still be denied because the EHR filled a field from the wrong part of the chart. A coverage check can return a valid response that says approval is needed when the payer’s own policy says it is not.
The guides also move. CMS proposed in April that payers support specific versions of the Da Vinci guides from October 2027, according to the CMS-0062-P fact sheet. Some payers will run 1 version while some EHRs run another. Without shared test cases, nobody can show quickly whether 2 versions give the same answer for the same patient.
A guide that ships without test data leaves every pair of partners to write the same tests from scratch.
What a test pack would hold
A test pack is a set of synthetic cases published with the guide, under the same version number. Each case pairs an input with the answer the guide expects. For prior authorization, a useful pack would hold a few kinds of material.
- Synthetic patients. Records for invented patients that carry the conditions the test cases need. None of the data describes a real person.
- Sample rules. A small set of invented coverage policies written in the guide’s own format, so every implementer runs the same logic.
- Expected results. For each patient and rule, the answer the exchange should produce, down to the fields a form should fill.
- Edge cases. Patients designed to trip common mistakes, such as a lab result just outside a time window.
A case might pair an invented patient who had 4 weeks of physical therapy with a sample imaging policy that requires 6. The expected result would be a request for more documentation, with the therapy dates already filled in.
A payer could run the pack against its server and compare its answers to the expected ones. An EHR vendor could do the same with its client. Each side could point to a published case when the 2 disagree in production. The case would show which behavior the guide intended.
The pack would also make version changes legible. If a new version changes an expected answer, the pack would show exactly which case changed. Implementers would know what to retest instead of retesting everything.
None of this replaces testing between real partners. A payer’s real policies and a health system’s real charts will always hold surprises. But a shared pack would catch the common errors before partners spend weeks finding them together.
Who should build it
The people who write the guide know what the test cases should be. They wrote the steps, and they argued over the edge cases while they wrote them. Capturing those cases as data costs far less than every implementer rediscovering them.
Standards bodies could make a test pack part of what it means to publish a guide for trial use. Packs should be public and versioned, with a change log that explains each altered case. A guide that reaches implementers with a pack would get better feedback, because implementers could report failures against a shared case instead of describing them in prose.
Regulators could help too. When CMS names a guide version that payers must support, it could favor versions that come with a test pack. ASTP already builds test kits for certification. A test pack would give those kits content to check beyond structure.
Synthetic data has a privacy advantage as well. Real records cannot be shared freely for testing, and they should not be. Invented patients can be published and argued over without exposing anyone. That freedom is what makes a shared pack possible.
Redlake builds tools that health plans use in utilization review. Every recommendation our tools make shows its reasons, and every decision can be reconstructed. A test pack applies the same idea to the standards themselves. It shows the reasoning a guide intends, in cases anyone can rerun. Our piece on the prior authorization API deadline argued that plans should test with real partners. A shared pack would let them start from the same page.