In short
- The safest AI uses in a patient portal help patients navigate, handle administrative questions, understand advice a clinician has already given and draft messages that a clinician reviews before anything clinical is sent back.
- AI that interprets symptoms or test results for an individual patient, or recommends starting, stopping or changing treatment, is high risk: FDA treats decision support aimed at patients as meeting the device definition, and Anthropic's usage policy requires review by a qualified person and disclosure that AI was used.
- Whatever the feature, the portal needs a BAA with the model provider, answers tied to the patient's own records, a clear AI label and a fast path to a human.
AI in patient portals is safe when it helps patients do what they already came to do: find a document, reschedule a visit, understand a bill, make sense of instructions their clinician already gave, or write a clear message to their care team. It becomes high risk when it interprets symptoms or test results for an individual patient, or recommends starting, stopping or changing a treatment. FDA treats decision support aimed at patients as meeting the device definition, and Anthropic's usage policy requires that a qualified person review AI recommendations in healthcare before they reach the patient. Most product decisions in a portal AI project follow from which side of that line each feature falls on.
This guide is for founders, product leads and digital health teams adding AI features to a patient portal or app. It is not legal advice.
Why is patient-facing AI held to a different standard?
Because the reader is not a clinician. FDA's Clinical Decision Support Software guidance, issued January 29, 2026, explains which decision support functions fall outside the device definition, and one of the criteria is that the software supports a health care professional's decision. The guidance says directly that software functions that support or provide recommendations to patients or caregivers, rather than professionals, meet the definition of a device. FDA says it supports access to safe and effective decision support for non-professionals and intends to follow its existing policies, including those on device software functions, mobile medical applications and general wellness products, when regulating it.
NIST's Generative AI Profile (NIST AI 600-1, July 2024) names two risks that matter more when the user is a patient. Confabulation is a model confidently presenting false content, and NIST's own example is a confabulated summary of patient information leading to wrong diagnoses or treatment. Human-AI configuration covers people anthropomorphizing AI systems and experiencing automation bias, over-reliance or emotional entanglement. Both risks apply to a worried patient reading a fluent answer late at night.
Which AI features carry lower risk in a portal?
Anthropic's usage policy, effective November 12, 2026, is a useful sorting tool even if you use another model, because it spells out what it treats as high risk in healthcare and what it excludes. Its exclusions include general educational content about how a condition or treatment works that is not applied to the individual's circumstances, wellness content about sleep, stress, nutrition or exercise, and helping someone understand advice or a diagnosis a qualified person has already given them. Portal features that fit those exclusions:
- Navigation: finding a form, a past visit summary or the right page, in plain language.
- Administrative help: rescheduling, preparing for a visit, explaining a line on a bill, checking what to bring.
- Explaining what the clinician already said: restating discharge instructions or a care plan in simpler words, with links back to the original document.
- General education: how a condition or a type of medication works, without applying it to the patient's own results.
- Message routing by topic: sending billing, scheduling and records requests to the right team. Routing by clinical urgency is a different matter, covered below.
- Drafting help for patients: turning a long, anxious message into a clear question for the care team.
Which uses need a clinician in the loop?
Anthropic's usage policy lists healthcare among its high-risk areas. In the medical area it covers diagnosing a condition, recommending, starting, stopping or adjusting a treatment or medication, interpreting symptoms or test results to triage, diagnose or advise on treatment risks, and assessing or treating mental health conditions. For these, the policy requires that a qualified person meaningfully review the AI's recommendation, with authority to change it, before it reaches the individual, and that individuals be clearly told that AI was used. A qualified person needs training or experience in the field and, where the law requires, a license.
In a portal, the pattern that fits these rules is a draft that a clinician reviews. A patient sends a message, the system drafts a reply from the chart, a nurse or physician edits and sends it, and the patient sees a note that AI helped prepare it. Symptom checkers and automated triage that answer the patient directly sit on the other side of the line and need regulatory review before they are built.
Should AI explain lab results to patients?
Not on its own. Interpreting a test result for an individual is in Anthropic's high-risk medical category, and decision support aimed at patients meets FDA's device definition. What a portal can do safely is narrower: show the result with the lab's reference range, link to the clinician's note about it if one exists, explain in general terms what the test measures, and help the patient write a question to their care team. If the clinician has already commented on the result, helping the patient understand that comment falls within the policy's exclusion for advice a qualified person has already given.
What safeguards should every portal AI feature have?
- A clear label: tell patients when they are reading something AI produced or helped draft. Anthropic's policy requires this for high-risk uses, and it costs little to do everywhere.
- Answers tied to the patient's own records: use citations so each statement links to the document it came from. Claude's citations feature returns pointers that are guaranteed to reference the documents you supply, though a valid pointer still needs testing to confirm it supports the claim.
- Permission to say no: the assistant should say when it does not know and offer to send the question to the care team.
- A fast human path: one tap from any AI answer to a message, a call or the practice's urgent care instructions.
- Strict data scope: the assistant sees only the logged-in patient's records, and a proxy sees only what their proxy permissions allow.
- Testing across groups: covered entities must make reasonable efforts under 45 CFR 92.210 to identify decision support tools that use race, color, national origin, sex, age or disability as inputs and to mitigate the risk of discrimination. Test answers across languages and patient groups before launch.
How do you keep PHI safe when the portal calls a model?
The model provider needs a BAA, and you must stay inside what it covers. Anthropic's privacy center says it provides a BAA covering its HIPAA-ready services, including its first-party API, and lists covered and uncovered features one by one; features it does not list are not covered. Amazon Bedrock is covered by the AWS BAA as a HIPAA eligible service. Check the exact features your portal uses against the agreement for the route you choose.
On our projects, we sign a BAA before anyone sees PHI and run a penetration test before launch. Our AI assistants cite their sources and run on Claude via AWS Bedrock or Anthropic directly, with provider switching. We built an assistant that reads a patient's records and answers with linked sources, on Claude under a BAA, inside a live telehealth platform, and on the clinician side each clinician sees only their own patients.
How do you know a portal AI feature is helping?
- Before launch, run a test set of real patient questions, including ones the AI should refuse or hand off, and have clinicians grade the answers.
- Track how often patients move from an AI answer to a human, and read those conversations. They show where the AI is unclear or out of scope.
- For drafted replies, track how much clinicians edit before sending. Heavy edits mean the drafts are not saving time.
- Ask patients whether the answer was clear, and review negative ratings every week in the first months.
- Re-run the test set whenever the prompt, the model or the data the assistant can see changes.
Start with the lower-risk features. Navigation, administrative help and clinician-reviewed drafts can reduce routine work without putting the patient in front of an unsupervised clinical opinion, and they build the data and trust you will need for anything more ambitious.
Onix Systems has been building software since 2000 and has 370+ engineers. If you are planning AI features for a patient portal and want to know which ones you can ship safely first, talk to one of our engineers. Our free five-day technical audit ends with a written report.
Sources
- FDA: Clinical Decision Support Software guidance (January 2026, PDF)
- Anthropic: Usage Policy
- NIST AI 600-1: Generative Artificial Intelligence Profile (July 2024)
- Claude Docs: Citations
- Anthropic Privacy Center: Business Associate Agreements (BAA) for commercial customers
- 45 CFR 92.210: Nondiscrimination in the use of patient care decision support tools (Cornell LII)