6d (edited) • General discussion
📕📃The AI Governance Problem
The central problem with AI governance is not that organizations have failed to implement enough policies, controls, committees, or risk assessments. The deeper problem is that absolute AI governance is structurally impossible. Governance requires sufficient visibility into what is being governed, sufficient authority to control it, and sufficient evidence to establish accountability. Modern AI systems break all three conditions. They are distributed across model providers, cloud platforms, third-party applications, enterprise data, departments, APIs, agents, human users, and downstream systems. No single organization can fully see, control, or reconstruct every material element of that environment. What the industry commonly calls “AI governance” is therefore not governance in the absolute sense. It is bounded governance over the portions of an AI ecosystem that an actor can actually observe and influence.
NIST itself exposes this contradiction. The AI Risk Management Framework makes GOVERN a cross-cutting function intended to operate throughout the AI lifecycle and organizational hierarchy, including third-party software, hardware, and data. MAP goes further by calling for risks and benefits to be mapped across all components of an AI system, including third-party components. Yet NIST also explicitly recognizes that an AI lifecycle contains many interdependent actors who often do not have full visibility or control over other parts of that lifecycle, making impacts difficult to anticipate reliably. Those two realities cannot be reconciled into absolute governance. An organization can be responsible for managing risks that originate beyond the limits of its own knowledge and authority. That is risk management. It is not governance closure.
The problem becomes obvious inside a real enterprise. A company may use GPT and Claude in the same workflow, Microsoft infrastructure underneath them, Salesforce or Workday downstream, proprietary data in retrieval, and an agent deciding which service to call. OpenAI can govern the portions of GPT that OpenAI controls. Anthropic can govern Claude. The enterprise can govern how its employees, applications, data, permissions, and workflows use those models. None of them governs the entire resulting system. The problem becomes even more fragmented across departments. HR, finance, legal, engineering, security, and marketing may use the same underlying model under completely different permissions, data classifications, laws, consequences, and risk tolerances. Saying that the enterprise has “governed GPT” therefore says very little. A governed component does not produce a governed system, and even several individually governed components do not automatically produce a governed composition.
This creates a recursive third-party problem—a governance spider web. Every enterprise depends on providers, and those providers depend on other providers, infrastructure, data sources, software, models, and subcontractors. NIST specifically recognizes that third-party systems increase complexity and opacity and that provider and deployer risk methodologies may not align or even be fully disclosed. Governance therefore cannot simply be inherited through trust. If an enterprise uses OpenAI, OpenAI's governance does not become the enterprise's governance. If the enterprise builds an agent on top of OpenAI and Claude, neither provider's governance covers the resulting composition. Each actor can supply evidence about its own boundary, but no actor possesses complete evidence about the entire chain. As dependencies multiply, the gap between responsibility and visibility grows with them.
Agentic AI makes the problem harder because the governed system may not even have a fixed architecture. An agent can select a model, retrieve information, invoke a tool, call another model, interact with an API, and initiate an action based on conditions that arise at runtime. The actual system that produces an output can therefore be different from the system diagram that was approved before deployment. This is why lineage matters. A meaningful governance system should be able to show which model, version, data, tools, policies, permissions, instructions, approvals, and third parties contributed to a consequential output. But lineage does not make absolute governance possible. Lineage proves what can be known; it also exposes what cannot be known. A model-agnostic framework can preserve this evidence across GPT, Claude, Gemini, internal models, and future systems, but it cannot manufacture information that an upstream provider does not possess or disclose.
The EU AI Act reaches the same practical boundary through law. For high-risk systems it requires continuous lifecycle risk management, including known and reasonably foreseeable risks, post-market monitoring, and, where relevant, analysis of interactions with other AI systems. Yet the Act explicitly recognizes residual risk, requires risks to be reduced only as far as technically feasible, and provides for mitigation of risks that cannot be eliminated. That is an important admission. Even a legally compliant governance regime does not produce complete control. It produces an organized process for identifying, reducing, documenting, monitoring, and accepting uncertainty. Compliance can be demonstrated. Absolute governance cannot.
The mistake, then, is treating AI governance as a binary state: governed or ungoverned. For modern AI, that claim is too broad to be meaningful. Governance has boundaries. It belongs to particular actors, systems, uses, departments, dependencies, and moments in time. Models change. Vendors change. Data changes. permissions change. Agents construct new execution paths. Unknown risks become known only after deployment. Even if complete governance could theoretically be established at one instant, the governed object would begin changing immediately afterward.
The stronger and more defensible objective is therefore not complete AI governance, but provable governance within declared boundaries. An organization should be able to state what it governs, what it does not govern, where authority changes hands, what evidence it possesses, which dependencies it relies upon, what uncertainty remains, and how a particular consequential output was produced. Under that standard, the most important governance question is no longer, “Is this AI system governed?” It is: “Which parts of this system are governed, by whom, under what authority, with what evidence, and where does that governance stop?”
That question exposes the central contradiction the field has largely avoided: AI governance expands across the entire system, while visibility and control contract at every organizational and technical boundary. The farther governance attempts to reach toward the whole, the less any single actor can actually know or command. Absolute AI governance is therefore not an unfinished destination waiting for better tools. It is an impossible condition. The legitimate task of AI governance is to make its boundaries, dependencies, evidence, lineage, authority, and remaining uncertainty explicit rather than pretending those limits do not exist.
0
0 comments
📕📃The AI Governance Problem
powered by
skool.com/trans-sentient-intelligence-8186
Turn ChatGPT into a structured work environment with chat-native workflows—no agents, coding, APIs, or custom models required.
Suggested communities
Powered by