Security Checklist for AI Radiology Reporting Tools: 12 Questions Before You Send a Single Study

The real security risk in AI radiology reporting is not the vendor you evaluate carefully. It is the radiologist who pastes a report into a consumer chatbot at 2 a.m. because no approved tool existed. A proper review of an AI report-drafting tool is therefore worth doing quickly and well: the goal is a sanctioned path that is safer than the unsanctioned one everyone is already using.

This is the checklist I would hand a compliance officer or IT reviewer at a radiology group or health system. Twelve questions, what a good answer looks like, and the red flags. It is written by a radiologist who uses such a tool on every shift, and it is deliberately vendor-neutral; where MyRadAgent is mentioned, the statement is one you can check on its security page.

1. Where does protected health information go?

The first question decides whether the rest matter. A good answer names the step at which PHI is removed, the identifier categories covered, and what the model vendor actually receives.

• Good: "Text is de-identified on our server using the HIPAA Safe Harbor identifier categories before any model call; the vendor receives redacted text only; identifiers are re-inserted on the user's side; only redacted text is stored."
• Red flag: "We rely on the vendor's BAA" with no redaction, or "the model is private so it does not matter". A BAA governs liability; it does not remove the data from the request.

Ask separately about images. Burned-in identifiers on radiographs and screenshots are not removed by text redaction. A good vendor says so plainly and tells you what the DICOM import strips and what the user must crop.

2. Is there a BAA with every party that touches PHI?

Not just the vendor you contract with. Every AI model provider, the hosting company and any logging or email service that could see request content. Ask for the list with dates, and compare it with the subprocessor table in the privacy policy. If the two disagree, the privacy policy is the one that binds.

• Good: named providers, "executed" with a date, and a BAA offered to your practice on paid plans.
• Red flag: "in progress", "enterprise tier only" without a named tier, or a payment processor listed as a PHI subprocessor (it should not be; billing data is not PHI).

3. Is your data used to train models?

Get it in writing for both the vendor and the upstream model providers. The acceptable answer is no, for inputs, drafts and signed reports alike. Note the distinction between training a foundation model (unacceptable) and learning a single radiologist's style within their own account (the point of the product). Ask that the second never crosses accounts.

4. Where is it hosted and how is it encrypted?

TLS 1.2 or higher in transit, between the user and the vendor and between the vendor and every subprocessor; encryption at rest at the storage layer; a named hosting provider under BAA. Ask about region if your policy requires it. Ask what the vendor's own staff can see and under what controls.

5. What is retained, for how long, and how is it deleted?

Three separate schedules: unconfirmed drafts, confirmed (signed) reports, and logs. A good answer gives a number for each, states that deletion happens on request within a fixed period, and names what survives deletion (audit logs for a statutory period). Confirm the user can export everything in an open format at any time without asking the vendor.

6. How are accounts isolated?

The tool will hold each radiologist's style rules, corrections, retrieval memory and, often, a bank of de-identified teaching cases. Ask:

• Can any other user, group or administrator view list another account's material?
• Are group workspaces isolated from every other customer?
• Does the vendor's own support staff see report text, and is that access logged?

The right answer is that everything is keyed to the account, with practice rules as the only shared layer within a group.

7. How is access controlled?

Password hashing with a modern algorithm (bcrypt or equivalent), signed sessions that never appear in a URL, rate limits on login and signup, account lockout after repeated failures, and role separation enforced on the server rather than hidden in the interface. For a health system, single sign-on: OpenID Connect with Google or Microsoft today, and a clear statement about SAML federation. "On the roadmap" is an acceptable answer; "yes" without a demonstration is not.

8. Is there an audit log, and can you have it?

Logins, administrative actions, exports, deletions and other high-impact events, with the acting user and timestamp. Ask how long it is kept and whether your compliance officer can get an export for your users without a support ticket per request.

9. Where does the output stop?

This is the clinical boundary and it is also a security question. A drafting tool should produce a draft for a licensed radiologist to review and sign. It should not sign, should not send a report to any system on its own, and should not render a diagnosis. Check that the product says so in its terms and in the interface, not only in a disclaimer page nobody reads.

10. Which models, and what happens when one fails?

Ask which model providers are used for which tasks, whether routing can change without notice, and what happens on a provider outage. A router across several providers is a resilience feature as long as every provider is under BAA and the vendor can tell you which ones handled your studies. Ask whether the vendor publishes how its accuracy figures are measured; a vendor that will not show the method on a page like a validation page should not be trusted on the number.

11. What is the incident process?

A named contact for security reports, a stated acknowledgement practice, and breach notification as required by the BAA and applicable law. Ask whether they have ever had an incident; "never" from a young company is plausible, "we cannot say" is not.

12. What do they not claim?

The most useful question on the list. HIPAA has no certification; a vendor that says "HIPAA certified" has told you something about their carefulness. Ask directly: do you hold a third-party security attestation, have you had an independent penetration test, do you offer SAML, do you offer on-premises deployment? A vendor who answers "not yet" to some of these, in writing, on their own site, is usually the one whose yes answers are true.

How to run the review in an afternoon

1. Send the twelve questions and ask for written answers.
2. Read the vendor's security page and privacy policy yourself and check that the subprocessor tables and retention schedules match the answers.
3. Ask for the BAA template.
4. Have one radiologist run ten de-identified cases on the trial and confirm the output boundary: a draft, in a review window, nothing sent anywhere.
5. Write down the "not yet" list and decide whether any item blocks a pilot. Usually none does; usually one blocks a group-wide rollout, and that becomes a contract milestone.

Where MyRadAgent stands

Every answer above is published on /security: the data flow, the identifier categories, the subprocessor and BAA table with dates, retention, isolation, authentication, logging, incident contact and the clinical boundary. So is the "not yet" list: no third-party attestation or independent penetration test yet, no SAML, no on-premises option. The privacy policy carries the contractual version, the terms the output boundary, and the clinical disclaimer the one-paragraph summary your radiologists will actually read.

If the list raises a question the page does not answer, email support@myradagent.ai with "Security review" in the subject; the founder reads every message. If you would rather see the product first, the free trial takes a minute and needs no card.

MyRadAgent is clinical decision support. Every output is a draft for review and signature by a licensed radiologist, who remains responsible for the final report.