28 Aug
A regional health system once signed off on a FHIR readiness assessment that fell apart in week two of production. The EHR had a clean CapabilityStatement, a green checklist from the vendor, and a project manager pushing to lock the go-live date. Two weeks after cutover, prior authorization requests started bouncing because Coverage resources referenced Patient IDs that didn’t resolve on the payer side. Nobody caught it beforehand. Not the vendor, not the client’s integration team, and not the consultant who certified readiness, because none of them had actually audited the data model. They audited the documentation.
That kind of failure is why readiness reviews need to go further than a checklist.
A CapabilityStatement is not a readiness certificate
A CapabilityStatement tells a consultant which resources a server exposes and which interactions it supports. It does not tell you whether the data behind those resources is any good. A system can declare full support for Coverage, Claim, and QuestionnaireResponse and still fail every real transaction, because support and correctness are separate questions.
Vendors have every incentive to show you the CapabilityStatement first. It’s the easy artifact. It looks thorough. It is also, on its own, close to worthless for a sign-off decision. The readiness question a consultant actually needs answered is narrower and harder: does this client’s data, as it exists today, produce a valid FHIR resource that a receiving system can consume without a human fixing it first?
You only answer that by looking at the data model, not the specification the vendor claims to follow.
The seven-layer data model audit
Run these seven checks against a sample of real or production-representative data before certifying anything. This sequence applies to any Da Vinci-related engagement, whether it’s a PAS implementation for a mid-level hospital or a DTR rollout for a CCBHC network.
| Layer | What to check | Common failure point |
|---|---|---|
| Resource mapping | Does each source field map to the correct FHIR element, not just a plausible one | Local “status” fields mapped to the wrong FHIR status code, silently accepted |
| Terminology binding | Do local codes resolve to the required ValueSet, not a default or free-text fallback | Diagnosis or procedure codes stored as local strings never mapped to SNOMED or LOINC |
| Cardinality and required fields | Are 1..1 and 1..* elements actually populated, not defaulted to null-flavor | Coverage.beneficiary or Claim.insurer left empty because source system treats it as optional |
| Reference integrity | Do resource references resolve to an existing, correctly typed resource | Practitioner reference pointing to a deactivated or merged provider record |
| Extension and profile conformance | Do US Core or Da Vinci profile extensions carry the values a receiving system expects | Missing race or ethnicity extensions required by US Core but never populated upstream |
| Transaction-level testing | Does a full CRD, DTR, or PAS exchange complete without manual correction | QuestionnaireResponse submitted without required linkId answers, rejected downstream |
| Provenance and update handling | Are updates to source data reflected correctly, without orphaning old references | Patient merge in the EHR breaks existing Coverage and Claim references |
Most readiness assessments stop at layer one or two. The failures that actually take down a go-live live in layers four through seven, and those are the layers nobody tests until the deadline is already tight.
Where resource mapping quietly breaks
Resource mapping looks like the simplest layer and causes some of the ugliest downstream problems, mostly because errors here pass every automated schema validator. A field can be perfectly valid FHIR and still be wrong.
Watch for these patterns during mapping review:
- Status field collapsing. Source systems often have five or six internal status values that all get mapped to a single FHIR status code, losing information a downstream system needs to route the resource correctly.
- Free-text leaking into coded fields. A “notes” or “comments” field gets stuffed into a coded element because nobody built a proper terminology lookup, and it passes validation because FHIR allows text-only CodeableConcepts.
- Date and time zone drift. Source timestamps stored in local time get mapped without a time zone offset, which corrupts anything time-sensitive, including prior authorization expiration windows.
- Identifier system mismatches. A member ID gets mapped without the correct identifier.system value, so a receiving payer can’t match it to their own records even though the number itself is correct.
- Silent truncation. Long-form clinical notes or lengthy code descriptions get cut off at a field length limit inherited from a legacy system, and the truncation happens before mapping, so the FHIR resource is technically valid and factually incomplete.
- One-to-many collapsed to one-to-one. A patient with three active coverages gets mapped as if they have one, because the source schema was built around a single-payer assumption that FHIR’s Coverage resource doesn’t share.
None of these show up in a CapabilityStatement. All of them show up the first time a real transaction runs against a real payer or exchange partner.
Terminology binding: the gap nobody budgets time for
Terminology work is where most project timelines quietly slip, and it’s rarely because the mapping is hard. It’s because nobody assigned an owner to it. A local code table sits in a legacy system, half-documented, and the vendor’s default integration engine ships with a generic terminology binding that was built for a different client’s code sets.
Before sign-off, pull a sample of actual coded values from the source system and trace each one through to its bound ValueSet. Not the ValueSet the implementation guide says should apply. The one the system is actually using. Production systems have been found binding diagnosis codes to a 2019 snapshot of a ValueSet that had since been updated, which meant every new code added to the standard set silently failed validation on the receiving end.
This check takes an afternoon. Skipping it costs weeks after go-live.
Cardinality and reference integrity traps
Cardinality violations are the easiest failure to fix and the easiest one to miss, because a null-flavored or omitted required field often still passes a basic validator. The validator checks structure. It doesn’t check whether omitting that field breaks the business process downstream.
Reference integrity is worse, because it depends on the state of other resources at the moment of the transaction. A Practitioner reference that resolved correctly during testing can break in production the day that provider record gets merged, deactivated, or re-keyed during a routine EHR maintenance cycle. Test reference integrity against a live, moving dataset, not a static extract taken once and reused for the whole engagement. A static test dataset gives you false confidence about a system whose real data changes daily.
What actually belongs in a sign-off report
A readiness sign-off should read like an audit finding, not a vendor endorsement. Every claim in it needs to trace back to something you actually tested against real data.
Include these elements in every report before you attach your name to it:
- Sample size and source. State exactly how many records you tested, from which system, and how recent the data was.
- Layer-by-layer results. Report findings against each of the seven layers above, including layers that passed cleanly, not just the ones with problems.
- Specific failure examples. Name the actual field, resource, and value that failed, not a generic “mapping issues found” line.
- Remediation owner and deadline. Every open finding needs a named owner and a date, or it becomes background noise the client forgets about.
- Re-test plan. State how and when each remediated item will be re-verified before the go-live date, not assumed fixed because someone said so.
- Residual risk statement. If anything remains unresolved at sign-off, say so explicitly and get written acknowledgment from the client, rather than letting silence imply everything is clean.
A sign-off without these six elements isn’t a technical assessment. It’s a liability with a signature on it.
Running the audit before you sign anything
The data model audit takes longer than reading a CapabilityStatement, and that’s the entire point. A client under deadline pressure will always prefer the faster answer. Your job is to make sure the faster answer isn’t the wrong one with your name attached to it.
Pull a real data sample, walk it through all seven layers, and write down what actually breaks instead of what the documentation promises. That’s the difference between a readiness assessment that holds up in production and one that turns into a support call three weeks after go-live.
FAQs
A FHIR data model audit checks whether a client's actual data, mapped resources, and terminology bindings hold up under real transactions, not just whether their system claims FHIR support. It goes past a vendor capability statement and tests resource mapping accuracy, cardinality rules, reference integrity, and code system alignment against live or representative data.
A CapabilityStatement tells you which resources and interactions a server supports in theory. It says nothing about whether the underlying data was mapped correctly, whether required fields are populated with valid values, or whether the terminology bindings match the ValueSets a payer or exchange partner actually expects. Readiness on paper and readiness in production are different claims.
Coverage, Claim, and QuestionnaireResponse resources cause the most failures in Da Vinci PAS and DTR workflows, usually because of missing reference chains between Patient, Coverage, and Practitioner, or because local code values were never mapped to the required terminology bindings. These gaps rarely show up until a real prior authorization request runs end to end.
For a mid-size hospital or health system, a thorough audit across the seven layers, including sample transaction testing, typically takes one to two weeks with access to a data analyst and a copy of production-like data. Smaller CCBHC or rural health environments often take less time, but only if someone already has clean data dictionaries to work from.
The sign-off should carry both names, but the audit itself needs to be independent of whoever built the mapping. If the same team that configured the FHIR Facade or integration engine is also the one certifying it works, errors that were invisible during build tend to stay invisible during review.
ISO 27001:2022 Certified
Aigilx health specializes in developing Interoperability solutions to create a healthcare ecosystem and aids in the delivery of efficient, patient-centric and population-focused healthcare.
