AI Governance in Healthcare

Executive Summary

Artificial intelligence is already embedded in healthcare operations, in clinical workflows, revenue cycle processes, marketing platforms, and the daily tools employees use to draft, summarize, and communicate. In most organizations, this adoption has outpaced the governance infrastructure needed to manage it responsibly.

The stakes of this gap are unusually high. Healthcare operates at the intersection of patient safety, dense regulatory obligations, deep public trust, and significant financial exposure. An AI tool that generates a biased clinical recommendation, inadvertently exposes protected health information, or makes a financial decision based on flawed data does not just create a business problem, it creates a patient safety, compliance, and reputational crisis.

Most organizations do not fail at AI governance because they lack policies. They fail because they lack visibility into what is actually happening across the enterprise.

This white paper provides a practical framework for closing the governance gap. It is built on a central thesis: governance is not a brake on innovation. It is the prerequisite for sustainable, responsible AI adoption. Organizations with mature AI governance move faster, not slower, because they spend less time debating each decision from first principles and less time managing preventable incidents.

If your governance program starts with policy before visibility, it will fail.

The framework covers why healthcare is different from other industries, how to think about the full spectrum of data AI touches, the seven pillars that define what governance must address, and a concrete action plan for the first ninety days.

Why Healthcare Is Different

Every industry faces AI governance challenges. Healthcare faces them within a regulatory environment and risk profile that no other sector matches.

The Regulatory Density

Healthcare AI governance begins with HIPAA, the Privacy Rule, Security Rule, and Breach Notification Rule apply to AI systems the moment they touch protected health information, whether for training, inference, or any intermediate processing. The HITECH Act extends direct liability to business associates, making AI vendor relationships regulatory relationships, not just procurement decisions. The FDA regulates AI tools that meet the definition of a medical device, including many clinical decision-support tools, and finalized guidance on Predetermined Change Control Plans for AI-enabled devices in December 2024.

At the state level, the landscape has expanded rapidly. In 2025, forty-seven states introduced more than 250 bills including health AI regulation, and thirty-three became law across twenty-one states. California, Texas, Colorado, and Illinois have enacted significant AI requirements effective in 2025 and 2026. At the federal level, a December 2025 executive order established a framework to preempt state AI laws the administration deems burdensome, but state laws remain in effect unless and until Congress acts or courts rule otherwise.

The practical result for healthcare organizations is a dense, layered, and still-evolving compliance landscape that AI governance programs must navigate across HIPAA, FDA, state privacy law, state AI law, and federal policy simultaneously.

Most governance programs are designed to satisfy regulatory review. Very few are designed to manage real-time operational AI usage. That gap is where risk accumulates.

The Stakes Beyond Compliance

Regulations define the floor of AI governance, not the ceiling. Three dimensions of stakes extend beyond any statute:

1.   Clinical influence. AI outputs shape clinician attention, anchor judgment, and change the conditions under which clinical decisions are made, whether the AI is technically making the decision or not. For example, an AI triage tool that subtly prioritizes certain patients can shift clinician attention patterns, without any explicit clinical directive.

2.   Patient trust. Patients extend trust to healthcare institutions that is qualitatively different from the trust they extend to other industries. AI that operates on patient data without transparency erodes that trust in ways that are difficult to rebuild.

3.   Health equity. Models trained on historical data can encode and amplify existing disparities in care, at scale, across populations, and with an appearance of objectivity that makes bias harder to detect.

The Dual-Data Challenge

Most healthcare organizations have built mature governance around one category of data, protected health information, and something closer to ad hoc oversight around everything else. AI collapses this distinction. AI systems do not respect the organizational boundaries that governance programs were built around.

Two Buckets, Five Data Categories

Bucket 1: Regulated health data. Includes PHI (EHR data, claims, imaging, clinician notes, patient communications) and clinical research data (study protocols, consented datasets, adverse event reports). Governed by HIPAA, FDA, IRB oversight, state health privacy laws, and institutional policies. Most organizations govern this bucket well. However, existing programs were not designed with AI in mind.

Bucket 2: Business and consumer data. Includes operational and financial data (revenue cycle, supply chain, workforce, financial planning), marketing and consumer data (CRM, website analytics, patient acquisition), and administrative data (contracts, HR records, internal communications). Governed by a more diffuse combination of SOX, FTC Act, state consumer privacy laws, and internal controls. This is where AI adoption has moved fastest, and governance attention has been weakest.

Why It Matters

A single AI platform may be subject to HIPAA, SOX-adjacent controls, and FTC scrutiny simultaneously depending on which data is being processed. AI tools routinely cross bucket boundaries within weeks of deployment. For example, a marketing AI tool initially trained on CRM data may quickly incorporate scheduling data or clinical signals to improve targeting, crossing into regulated territory without explicit oversight. Governance must be organized around data flows, not around tools, and data lineage must be traceable when a regulator or board member asks.

The Seven Pillars of AI Governance

The framework is organized around seven pillars, each addressing a distinct question that governance must answer. All seven must be present, even in basic form, before AI systems move into production at scale.

#	Pillar	Core question
1	Accountability & ownership	Who owns this decision, and who is answerable when something goes wrong?
2	Transparency & explainability	Can we explain how the AI reached its output, appropriate to the context?
3	Data integrity & lineage	Where did the data come from, how was it transformed, and is it fit for purpose?
4	Privacy & security	How is data protected throughout the AI lifecycle, from training through retirement?
5	Fairness & equity	Does the AI perform equitably across patient populations and demographics?
6	Compliance & auditability	Can we demonstrate governance to regulators, auditors, and our own board?
7	Continuous monitoring	Is this system still safe and effective after go-live, and how will we know?

Each pillar requires both a policy dimension (what the organization has committed to) and an operational dimension (what the organization does). A pillar that exists in policy but not in practice is a pillar that will fail under pressure, during an incident, an audit, or a regulatory inquiry.

Where You Probably Are Today

Five patterns characterize the starting conditions at most healthcare organizations:

•     The HIPAA assumption. Organizations with mature privacy programs believe those programs extend naturally to AI. They cover one critical risk and leave the rest unaddressed.

•     Department-led adoption. AI tools enter through individual departments without enterprise visibility. No single person can enumerate what is in use.

•     Bottlenecks driving shadow use. Formal approval processes designed for a different cadence push employees to adopt tools without asking.

•     Inventory blindness. The organization cannot produce a current list of AI tools in production across the enterprise.

•     Unclear ownership. Ask three leaders who owns AI governance, and you get three different answers.

Shadow AI is the unauthorized or unvetted use of AI tools by employees, departments, or embedded within purchased software, and is the most common and most underestimated risk. It is not primarily a disciplinary problem; it is a visibility problem. The governance response should begin with discovery and end with integration, not enforcement. Example: a revenue cycle team using a generative AI tool to draft payer appeals may unknowingly input PHI into an unapproved system, creating risk without any malicious intent.

Governing by Risk: A Three-Tier Model

Not all AI use cases carry the same risk. Mature governance programs differentiate intensity by tier so that low-risk use cases move quickly and high-risk use cases get the scrutiny they require.

Tier	Characteristics	Governance pathway
Low	No PHI, no clinical impact, enterprise-licensed tool, limited consequence of failure	Governance lead approves; 5-day turnaround
Medium	Business-sensitive data, operational decisions with financial or quality implications, vendor review needed	Committee review; 15-day turnaround
High	PHI, clinical decisions, patient-facing, regulatory exposure, potential patient harm	Full committee + exec sponsor; bias assessment; 30-day turnaround

When a use case does not fit cleanly into one tier, the default is to escalate to the next higher tier until the committee reviews it. Edge case decisions should be documented, as they build the institutional knowledge that refines the criteria over time.

A common failure mode is over-classifying use cases as high-risk, which slows adoption and drives shadow AI. Effective governance balances speed with control.

Where to Start: A 30/60/90-Day Action Plan

The full governance program takes years to mature. The first ninety days are about building the foundation that everything else depends on.

Days 1–30: Visibility and Ownership

•     Name an owner. Designate an executive sponsor and a working governance lead. Formalize what is probably already happening informally.

•     Conduct a rapid inventory. Survey department leaders, review procurement records, consult IT security. Aim for 70% completeness in two weeks. Frame it as discovery, not enforcement.

•     Issue interim guidance. A one-page guideline: do not enter PHI or confidential data into unapproved AI tools, here are the tools that are approved, here is who to contact with questions.

Mistake to avoid: Trying to achieve 100% inventory completeness.

Reality: Directional visibility (60–70%) is enough to begin governing effectively.

Days 31–60: Decision-Making Infrastructure

•     Convene a governance committee. IT, legal, compliance, clinical leadership, and at least one business-side representative. Adopt an interim charter. Review the inventory.

•     Define risk tiers. A three-tier model with clear criteria. It does not need to be perfect; it needs to be clear enough to apply consistently.

•     Build a basic intake process. A short form, a routing path by risk tier, and a published turnaround commitment. Speed matters more than sophistication.

Mistake to avoid: Over-engineering governance workflows.

Reality: Adoption depends on speed and clarity, not perfection.

Days 61–90: First Formal Artifacts

•     Publish a foundational policy. An AI Acceptable Use Policy covering clinical and business use cases, communicated enterprise-wide with executive endorsement.

•     Begin shadow AI monitoring. Engage IT to identify AI traffic and AI-enabled SaaS. Lead with discovery, not enforcement.

•     Schedule a board briefing. Cover current state, key risks, the program being built, and the roadmap. Establish the reporting cadence.

Mistake to avoid: Publishing policy without integration into workflows.

Reality: Policy that is not embedded into how teams work will be ignored.

Moving Forward

The question for healthcare leaders is no longer whether AI governance is necessary. Most organizations reading this are operating with less than 50% visibility into AI usage across their enterprise. The organizations that get governance right don't start by writing policies; they start by finding out what's already happening. The tools are already in use. The data is already flowing. The regulatory expectations are already forming. The question is whether your organization will govern AI deliberately or whether it will govern by default, discovering its gaps only when an incident, an audit, or a patient's experience reveals them.

Organizations with mature AI governance do not move slower than their ungoverned peers. They move faster, because they spend less time debating each individual decision from first principles, less time managing incidents that could have been prevented, and less time explaining to regulators and boards why they do not have answers to straightforward questions.

Organizations that begin with visibility, align governance to workflows, and prioritize speed alongside control will not just reduce risk, they will outpace competitors who treat governance as a compliance exercise. For those ready to start the conversation: the first ninety days begin with one question.

If a regulator, a board member, or a journalist asked us tomorrow what AI tools we have in production, who approved each one, what data each one touches, and how we are monitoring them, could we answer?