โ†
AI for Pharma & Life Sciences
Visionary ยท M5 ยท lesson 5 of 19 ยท queued
Preview โ€” browse every lesson free. Enroll to mark lessons complete, open partner links and save your progress. Login & enroll โ†’
Cross-Functional AI Governance: Ethics, AI Council, Quality, IT, Privacy
๐Ÿ“–
now learning

Cross-Functional AI Governance: Ethics, AI Council, Quality, IT, Privacy

15 min

Six weeks after the enterprise AI policy was approved, a high-risk classification landed on a desk that did not exist. A clinical-development team had built an AI tool that proposed adaptive-trial dose modifications, the policy correctly flagged it as high-risk, and the policy correctly said high-risk uses require approval by "the cross-functional governance body," and at that moment everyone discovered there was no such body, only five councils, each of which believed the decision belonged to one of the others. The Ethics Committee thought it was a Quality question. Quality thought it was an IT validation question. IT Governance thought it was a clinical-science question. The Privacy Council had not been told the tool existed. The AI Council, newly stood up, had no charter that said it could decide anything. The policy had created a decision right and assigned it to a vacuum, and the dose-modification tool sat unapproved for eleven weeks while five bodies negotiated who owned it. A policy without a governance structure is a constitution without a government, and this lesson is about building the government: the Ethics Committee, the AI Council, the Quality Council, IT Governance, and the Privacy Council, with decision rights that do not overlap, escalation paths that resolve rather than circulate, and integration into the existing GxP quality system so that AI governance is not a parallel bureaucracy but a coherent extension of the controls the company already runs.

Why Five Councils and Not One Committee

The instinct after the eleven-week deadlock is to collapse everything into a single all-powerful AI committee, and that instinct is wrong, because AI governance in a biopharma spans genuinely distinct domains of expertise that no single body can hold, and a single committee either becomes a bottleneck that everything queues behind or a rubber stamp that approves things it does not understand. The decision about whether an adaptive-trial dose-modification tool is scientifically and ethically sound is a different competence from the decision about whether its validation meets GxP, which is different again from whether its data handling meets privacy law, which is different from whether its technical architecture meets the enterprise's IT and security standards. Five councils exist because there are five distinct kinds of judgment, and the design goal is not to eliminate the distinctions but to make the boundaries between them so clear that a high-risk classification never again lands in a vacuum. The art is allocation of decision rights, not consolidation of power, and the leader who understands this builds a federation with clear jurisdiction rather than an empire with a single throne.

The five-council structure also reflects a reality the program has built toward since Level 4: AI risk in drug development is irreducibly multi-dimensional, and collapsing the dimensions loses information that the organization needs to make defensible decisions. The Ethics Committee owns the question of whether the AI use is right, including patient impact, fairness, and the appropriateness of AI involvement in a given decision. The AI Council owns the question of whether the use is technically sound and aligned to the enterprise AI policy, serving as the integrating body. The Quality Council owns the question of whether the use is GxP-compliant and validated. IT Governance owns the infrastructure, security, and architecture. The Privacy Council owns the data: consent, PHI, trade secret, and cross-border transfer. Each council answers a question the others are not equipped to answer, and the dose-modification tool needed all five answers, which is exactly why it deadlocked when no structure said how the five answers combine into one decision. The next sections build that structure.

Decision Rights: The Allocation That Prevents the Vacuum

The single artifact that would have prevented the eleven-week deadlock is a decision-rights matrix that names, for every category of AI decision, which council is accountable, which are consulted, and which are merely informed, and the discipline of the matrix is that exactly one council is accountable for any given decision. Accountability that is shared is accountability that is absent, which is the lesson the dose-modification tool taught at the cost of eleven weeks, so the matrix assigns each decision a single accountable owner even when many councils must be consulted. For the dose-modification tool, the accountable council is the Ethics Committee, because the irreducible question is whether AI should be proposing dose modifications that affect patient safety at all, and the Quality Council, IT Governance, and Privacy Council are consulted because their domains are implicated, but they do not each hold a veto that can be exercised in isolation, which is what produced the circular deadlock. The AI Council is accountable for confirming policy alignment and for convening the consulted councils, which is the integrating role that ensures the consultations actually combine into a decision rather than circulating indefinitely.

Designing the matrix well requires resisting two failure modes that organizations fall into predictably. The first is the everyone-is-accountable design, where the matrix lists all five councils as accountable for high-risk decisions out of a desire to be inclusive, which reproduces the vacuum because shared accountability means no one can actually decide. The second is the everyone-must-consent design, where every council holds a veto, which produces governance by the most risk-averse council and grinds high-value AI to a halt as each body protects itself by refusing. The defensible design names one accountable council per decision type, makes consultation mandatory but advisory, and reserves the veto for a narrow, well-defined set of conditions, a genuine privacy-law violation, a genuine GxP non-compliance, a genuine ethical red line, where a single council's domain expertise is dispositive. The leader who allocates decision rights this way builds a structure that can actually say yes, which is the property that distinguishes governance that enables work from governance that only obstructs it.

The Escalation Path That Resolves Rather Than Circulates

Decision rights handle the normal case; the escalation path handles the case the deadlock actually exposed, which is what happens when the councils disagree or when a decision falls genuinely between two domains. The failure of the dose-modification tool was not only that no council was accountable; it was that there was no path upward when the councils could not agree, so the disagreement circulated horizontally among five bodies forever instead of escalating vertically to a point of resolution. A working escalation path names, for each kind of disagreement, where it goes when the accountable council cannot resolve it, and it terminates in a single authority with the standing to decide, typically the AI Council chair for policy-alignment disputes and an executive AI steering body, a Chief AI Officer or an equivalent C-suite owner, for disputes that cross council domains or carry enterprise risk. The escalation path is what converts a horizontal federation of councils into a system with a top, and a system without a top cannot resolve a genuine disagreement, it can only circulate it.

The escalation path must also be time-bound, because an escalation that has no clock reproduces the deadlock at a higher altitude, and the eleven weeks were spent partly because nothing forced a resolution by a date. A defensible design attaches a decision deadline to each council and to each escalation tier, so that a high-risk classification that has not been resolved by the accountable council within a defined window escalates automatically to the next tier, and a classification that reaches the executive tier carries a hard decision date. This time-binding does something culturally important beyond preventing deadlock: it signals that the governance system exists to enable decisions, not to defer them, which is the cultural difference between a governance structure that the organization respects and one it routes around. A governance system that the organization routes around is worse than none, because the routing-around happens invisibly and produces exactly the ungoverned high-risk AI use the whole structure was built to prevent, so the leader's design imperative is a governance system fast and decisive enough that no one is tempted to bypass it.

Integrating With the Existing GxP Quality System

The most consequential design decision in the entire governance structure is whether the councils are a parallel bureaucracy or an extension of the existing GxP quality system, and the defensible answer is unambiguously the latter, because a parallel AI-governance structure that does not connect to the company's quality system produces decisions that the quality system cannot see, document, or defend in an inspection. The Quality Council in particular is not a new invention; it is the existing quality organization's authority extended to cover AI, and the AI-governance decisions it makes must flow into the same quality management system, the same document control, the same CAPA process, the same change control, and the same audit trail that govern every other GxP decision in the company. When the dose-modification tool is finally approved, that approval must be a controlled document in the quality system with a defined owner, a validation record, and a change-control linkage, not a slide in a council's meeting minutes, because an inspector reads the quality system, and an approval that lives only in a council's notes is an approval the company cannot prove it made under control.

This integration is what makes the AI-governance structure inspection-ready rather than inspection-exposed, and it is the point at which the Level 5 leader's understanding of GxP becomes load-bearing. The FDA-EMA Guiding Principles' "governance and documentation" principle and the program's recurring insistence on the audit trail both demand that AI governance decisions be documented in a system an inspector can read, and the existing GxP quality management system is that system. The councils' decisions become quality records; the AI inventory from the policy lesson becomes a quality-controlled register; the model cards, data cards, and validation packages become controlled documents; and the escalation decisions become documented rationales with named decision-makers. The leader who builds the councils as an extension of the quality system rather than alongside it ensures that when a Pre-Approval Inspection asks how the company governs its high-risk AI, the answer is a coherent set of quality records, not a folder of meeting minutes, and that difference is the difference between a governance structure that survives inspection and one that becomes the finding.

What Each Council Actually Owns, Concretely

Abstraction is the enemy of a working governance structure, so it is worth stating concretely what each council owns and decides, because the eleven-week deadlock happened precisely because no one could say concretely whose decision it was. The Ethics Committee owns the appropriateness question: should AI be involved in this decision at all, is the use fair across patient populations, does it respect the patient, and does it stay on the right side of ethical red lines, and it is accountable for high-risk uses where an irreducible human or patient-impact judgment is at stake, such as the dose-modification tool. The AI Council owns policy alignment and technical soundness: does the use conform to the enterprise AI policy, is the model fit for purpose, is the reference-pattern-and-variant discipline followed, and it serves as the integrating body that convenes the others and ensures consultations resolve into decisions. These two councils carry the decisions that are most distinctively about AI, and clarity about their boundary, the Ethics Committee owns whether the AI should, the AI Council owns whether the AI may under policy, is what keeps high-risk decisions from falling between them.

The Quality Council owns GxP compliance and validation: is the use validated to the appropriate standard, does its audit trail meet 21 CFR Part 11, is it integrated into change control, and it is accountable for the question of whether a regulated or high-risk AI use is defensible under inspection. IT Governance owns infrastructure, security, and architecture: is the deployment secure, does it meet the enterprise's technical and access-control standards, is the vendor's integration sound, and it is accountable for the technical-platform decisions. The Privacy Council owns data: is consent adequate, is PHI handled lawfully, are trade secrets and commercial confidential information protected, does any cross-border transfer comply, and it is accountable for the data-handling decisions and holds a genuine veto where a privacy-law violation is at stake. Each council's ownership is stated as the specific question it answers and the specific decisions it is accountable for, and the dose-modification tool, mapped against this structure, resolves cleanly: the Ethics Committee is accountable, the other four are consulted, the AI Council convenes them, the decision becomes a quality record, and the eleven weeks become eleven days.

The Council Cadence and the Anti-Bottleneck Design

A governance structure that is correct in its decision rights but slow in its operation will still be routed around, so the final design dimension is cadence and throughput, the practical engineering that keeps the councils from becoming the bottleneck that the single-committee design was rightly feared to be. The anti-bottleneck design rests on the risk-proportionality the policy already established: the vast majority of AI uses are internal-use and never reach a council at all, routing through self-attestation; most regulated uses are handled by a defined approval that the Quality Council and AI Council can process on a regular cadence without convening all five bodies; and only the genuinely high-risk uses, the small minority, require the full cross-functional convening. This funnel means the councils spend their scarce cross-functional attention only on the decisions that genuinely need it, which is what keeps the cadence fast enough that no one is tempted to bypass the structure for a high-value, time-sensitive use.

The cadence design also requires pre-delegated authority for the routine cases, so that the councils are not convened for decisions that fit an established pattern, which is where the reference-pattern discipline from the scaling lesson pays a second dividend. Once a high-risk reference pattern has been approved by the full council structure, its registered controlled variants can be approved by a delegated authority against the established pattern, without re-convening all five councils for each variant, because the hard cross-functional judgment was made once when the reference pattern was approved. This is the governance expression of the scale-criteria discipline: the councils make the expensive decision once, at the reference-pattern level, and delegate the cheap delta-decisions to a defined authority operating within the approved envelope, so the structure scales with the AI estate rather than becoming a bottleneck that grows with it. The leader who builds this delegation correctly produces a governance system that is rigorous on the decisions that matter and fast on the decisions that are already settled, which is the only kind of governance an organization both respects and follows.

What This Means for the Leader on Monday

The leader standing up the governance councils should begin not with the council charters but with the decision-rights matrix, because the matrix is what prevents the vacuum, and a council without a clear decision to own is a meeting without a purpose. The first Monday action is to enumerate the categories of AI decision the enterprise policy creates, internal-use approval, regulated approval, high-risk approval, reclassification, vendor qualification, escalation, and to assign each a single accountable council with named consulted councils, so that no decision the policy creates lands on a desk that does not exist. The second action is to define the escalation path with a named terminal authority and a clock at every tier, so that disagreement resolves vertically instead of circulating horizontally. The third action is to wire every council decision into the existing GxP quality management system as a controlled record, so that the governance structure is inspection-ready from its first decision rather than retrofitted into the quality system after an inspection exposes the gap.

The deeper lesson is that governance structure is where an AI policy either becomes real or becomes theater, because a policy that creates decision rights without bodies to exercise them produces the eleven-week deadlock, and a policy with bodies but no clear decision rights produces the same deadlock by a different route. The five councils, with non-overlapping decision rights, a time-bound escalation path, integration into the GxP quality system, and a risk-proportionate cadence with pre-delegated authority for settled patterns, are the government that makes the policy's constitution operative. Built this way, the structure resolves the dose-modification tool in eleven days instead of eleven weeks, survives a Pre-Approval Inspection as a coherent set of quality records, and is fast enough that no one routes around it, which is the test every governance structure ultimately faces. The councils are how the enterprise governs its AI, and the next chapter turns outward, to how the enterprise represents that governance to the regulators themselves, in the FDA Emerging Drug Safety Technology Program, the EMA AI Workplan, the PMDA AI Working Group, and the MHRA AI Airlock.

Key Takeaways

  • A policy creates decision rights; without a governance structure to exercise them, a high-risk classification lands in a vacuum and deadlocks. Five councils exist because AI risk in drug development is irreducibly multi-dimensional, and the Ethics Committee, AI Council, Quality Council, IT Governance, and Privacy Council each answer a question the others cannot, so the design goal is clear allocation of jurisdiction, not consolidation of power.
  • The decision-rights matrix names exactly one accountable council per decision, with others consulted but not each holding a veto. Shared accountability is absent accountability, and the two predictable failure modes, everyone-accountable and everyone-must-consent, both reproduce the vacuum, so the defensible design names one accountable owner, makes consultation mandatory but advisory, and reserves the veto for narrow dispositive conditions.
  • The escalation path must resolve vertically and be time-bound, terminating in a single executive authority. A horizontal federation of councils with no top can only circulate a disagreement, and an escalation with no clock reproduces the deadlock at a higher altitude, so each tier carries a decision deadline that signals the system exists to enable decisions, not defer them, which is what keeps the organization from routing around it.
  • The councils must be an extension of the existing GxP quality system, not a parallel bureaucracy. Council decisions must flow into the same quality management system, document control, CAPA, change control, and audit trail, so an approval is a controlled quality record with a validation linkage rather than a slide in meeting minutes, which is the difference between a structure that survives inspection and one that becomes the finding.
  • A risk-proportionate cadence with pre-delegated authority for settled patterns keeps the councils from becoming the bottleneck. Most uses are internal-use and never reach a council, and once a high-risk reference pattern is approved by the full structure its registered controlled variants are approved by delegated authority within the envelope, so the councils make the expensive cross-functional judgment once and the structure scales with the AI estate rather than growing into the bottleneck the single-committee design was feared to be.