26 Aug
A prior auth startup once shipped a feature that read clinical notes and auto-suggested the diagnosis code for the claim. It worked in the demo. It worked for the first fifty claims. Then it produced an ICD-10 code that looked completely legitimate, right format, right number of characters, right prefix for the specialty, and didn’t exist in the current code set. The claim bounced. Then the next one did too, because the model had learned the pattern of that fabricated code from its own recent context.
That’s the failure mode nobody puts in the pitch deck.
Structured FHIR data stops this because it removes the guessing step entirely. When a coded field is bound to a real value set, the model isn’t generating a code from scratch. It’s selecting from a list that’s already been validated. This piece walks through why that distinction matters more than any prompt engineering trick, and what it actually looks like to build on top of it.
The code that didn't exist
Medical coding systems are enormous and specific on purpose. ICD-10-CM has over 70,000 codes. CPT has thousands more. A model trained on general text, or even on a pile of clinical notes, learns the shape of these codes without learning the exact, current, valid list. It knows E11.9 looks like a diabetes code. It doesn’t know whether E11.65 or E11.649 is the correct level of specificity for hyperglycemia with diabetes, and it will pick confidently either way.
This isn’t a rare edge case. It’s the default behavior of a model asked to fill in a coded field from unstructured input. The model has no ground truth to check against, so it produces the most probable-looking token sequence. Sometimes that’s correct. Sometimes it’s a code that was retired years ago. Sometimes it’s a code that never existed.
For a startup, that failure surfaces downstream: a denied claim, a rejected prior auth, or a structured data field that a downstream clinical system trusts without question.
Prompting harder doesn't fix a hallucination problem
The instinct is to fix this with better prompting. Add few-shot examples. Tell the model to “only use valid ICD-10 codes.” Add a system instruction that says “double-check your work.” None of this works reliably, because the model still doesn’t have access to the actual list of valid codes at inference time. It’s still predicting, just with slightly better-calibrated confidence.
You can reduce the error rate this way. You can’t eliminate it. And in healthcare, a code that’s wrong 2% of the time isn’t a rounding error. It’s a compliance problem, a revenue cycle problem, and eventually a trust problem with the providers using your product.
The fix isn’t a better prompt. It’s a different data layer.
What CodeableConcept actually locks down
| Element | What it holds | Why it matters for AI output |
|---|---|---|
| System | The terminology the code belongs to (ICD-10-CM, SNOMED CT, LOINC, RxNorm) | Tells you which rulebook the code has to obey |
| Code | The exact value | This is the part a model would otherwise be free to invent |
| Display | The human-readable text tied to that code | Lets a person sanity-check what the code actually means |
| ValueSet binding | The finite, versioned list the code must come from | This is the checkpoint that rejects anything not on the list |
A “required” binding means the code has to come from that exact list. Nothing else validates. This is the mechanism that turns “the model guessed a plausible-looking code” into “the system checked the code against a real terminology server and either accepted or rejected it.”
An AI model working against this structure isn’t writing the code field directly. It’s mapping clinical language to candidate codes, and each candidate gets checked against the bound ValueSet before it’s accepted anywhere downstream. The check happens outside the model, against a source that doesn’t hallucinate.
Da Vinci turns "probably right" into "verified right"
This matters even more once prior authorization enters the picture. The CMS interoperability mandates push payers toward Da Vinci-based FHIR APIs, and three of them do most of the work here:
| Da Vinci API | What it does | What it replaces |
|---|---|---|
| CRD (Coverage Requirements Discovery) | Tells a requesting system what a payer requires, in real time | Guessing from a static policy PDF |
| DTR (Documentation Templates and Rules) | Serves the exact documentation template tied to that requirement | A generic form that may not match what the payer wants |
| PAS (Prior Authorization Support) | Submits the request using the codes and structured data the payer expects | Manual entry against remembered or outdated rules |
An AI model layered on top of this workflow isn’t inferring what a payer wants from a policy summary it half-remembers from training. It’s querying a live source and getting back the actual current requirement. The hallucination surface shrinks because there’s less left for the model to guess.
What changes in your stack once FHIR is the source of truth
For teams building on top of unstructured EHR exports or scraped clinical text, the fix isn’t just “add FHIR.” It’s rethinking where the model sits relative to the data.
- The model extracts, it doesn’t originate. Its job is mapping clinical language to candidate codes, not producing final codes.
- A terminology server does the validation. Every candidate code gets checked against the bound ValueSet before it moves forward.
- CodeableConcept becomes your contract. Any field that carries a code should carry it in this structure, not as a plain string the model can freely rewrite.
- Version pinning matters. Code sets update. A ValueSet binding needs a version, or you’ll validate against last year’s list.
- Rejections are visible, not silent. A code that fails validation should surface as an error state, not get quietly passed through with the model’s best guess.
None of this eliminates the need for a good model. It changes what the model is trusted to do.
Before you ship a clinical AI feature, check this
If your product touches diagnosis codes, procedure codes, medication codes, or anything tied to a claim or a prior auth, run this check before launch: pick ten outputs your model generated for coded fields, and verify each one against the actual current value set for that terminology, not against what looks plausible. If any of them fail, you don’t have a prompting problem. You have a missing validation layer, and FHIR’s structured resources are the fastest way to build one.
The startups getting this right aren’t the ones with the cleverest prompts. They’re the ones who stopped asking the model to be the source of truth and started asking it to work inside one.
FAQs
Large language models predict the next plausible token based on patterns in training text. When a model reads unstructured clinical notes and has to produce a diagnosis or procedure code, it generates something that looks statistically like a real code, not one verified against an actual coding system. Without a structured source that constrains the output to valid values, the model has no way to check its own answer.
FHIR resources use a data type called CodeableConcept, which binds a coded field to a specific value set from a recognized terminology such as ICD-10-CM, SNOMED CT, LOINC, or RxNorm. Instead of an AI model generating free text and guessing at a code, the system pulls from a finite, validated list. The model can only select from what's actually permitted for that field.
A value set binding restricts a coded field to a specific list of allowed codes from one or more terminology systems. FHIR defines these bindings as required, extensible, preferred, or example, with "required" meaning only codes from that exact list are valid. This is what turns an open-ended text field into a constrained, checkable data point.
Yes. Da Vinci implementation guides like CRD, DTR, and PAS define exactly which structured data elements and code systems a payer expects for a given service. An AI model built against these APIs pulls coverage rules and documentation requirements from a live, validated source instead of inferring them from a policy PDF, which cuts down on invented or outdated codes in submitted requests.
They can, but the model's output needs a validation layer sitting between the generated text and anything that reaches a claim, an EHR, or a clinician. That layer should check every code against the applicable value set before it's accepted. Without it, an LLM working from unstructured notes will eventually produce a code that looks right and isn't.
ISO 27001:2022 Certified
Aigilx health specializes in developing Interoperability solutions to create a healthcare ecosystem and aids in the delivery of efficient, patient-centric and population-focused healthcare.
