AI Governance and Security Policy: Enterprise Framework 2026
AI governance security is the discipline of deciding which AI systems your organization may build, buy and run, and then proving, with technical controls rather than slideware, that they behave within those limits. Most enterprises have one half of this. Compliance teams write principles that engineers never see, while security teams harden individual models without any shared definition of what is allowed. This framework joins the two: an operating model, an inventory-driven risk process, a policy you can adopt, a control baseline mapped to real attack classes, and a 90-day plan to put it in place.
- Why AI Governance and Security Must Be One Program
- The 2026 Regulatory & Framework Landscape
- The Enterprise Operating Model
- AI Inventory & Risk Tiering
- The AI Security Policy: Core Clauses
- Technical Control Baseline
- Shadow AI & Agentic AI Controls
- Third-Party & Vendor AI Risk
- Metrics & Audit Evidence
- 90-Day Roadmap & Checklist
- Frequently Asked Questions
Governance answers what is permitted and who is accountable. Security answers whether the permitted behavior is actually enforced and can be evidenced. When they are run separately, two failure modes appear. A governance-only program produces approved-use-case lists that nothing enforces, so employees paste customer data into whichever chatbot is fastest. A security-only program hardens the systems it happens to know about and misses the unsanctioned ones, and it has no authority to say a risky use case should not exist.
The practical test is simple: for every governance question a regulator, customer or board member can ask, there should be a named control that produces the answer on demand.
| Governance Question | Security Control That Answers It | Evidence Produced |
|---|---|---|
| What AI systems do we run, and who owns each? | Discovery and a living AI inventory | Inventory export with owner, purpose, data classes, tier |
| What data can each system see? | Data classification, DLP, retrieval-layer authorization | Data-flow map; DLP and access-policy logs |
| Who approved this model, and where did it come from? | Approved model registry, signed and hash-pinned artifacts | Registry record with provenance and scan results |
| Can it take actions on its own? | Agent permission scoping and human-approval gates | Tool allowlists; approval logs |
| Can we prove how it behaved last month? | Prompt, output and tool-call logging with retention | Searchable audit trail tied to a system ID |
| What happens when it fails or is attacked? | AI-specific incident response and rollback | Playbooks, drill records, rollback tests |
Enterprises do not need to implement every framework separately. They need one control backbone that can be mapped to each. The instruments below are the ones that most commonly drive enterprise AI programs in 2026.
| Instrument | Type | What It Asks For | Status (October 2026) |
|---|---|---|---|
| EU AI Act (Reg. 2024/1689, amended by Omnibus Reg. 2026/1744) | Binding law | Risk-tiered duties; bans on certain practices; obligations for high-risk systems, general-purpose AI models and transparency | Prohibited practices apply since 2 Feb 2025. Annex III high-risk moved to 2 Dec 2027; Annex I product-embedded systems to 2 Aug 2028. New bans on non-consensual intimate imagery and CSAM generation, and Article 50(2) duties for generators already on the market, apply from 2 Dec 2026. |
| NIST AI RMF 1.0 and the Generative AI Profile (NIST AI 600-1) | Voluntary framework | Four functions: Govern, Map, Measure, Manage; GenAI-specific risks and suggested actions | Widely used as the control backbone in US enterprises and referenced by many procurement questionnaires |
| ISO/IEC 42001:2023 | Certifiable standard | An AI management system: policy, roles, risk assessment, impact assessment, controls, monitoring, continual improvement | Third-party certification available; increasingly requested by enterprise customers as assurance |
| Colorado ADMT law (SB 26-189, replacing SB 24-205) | US state law | Per secondary sources: consumer notice before use, explanation after adverse outcomes, human review where feasible, three-year records | Reported effective 1 Jan 2027, with enforcement paused during litigation. Rules and exemptions are still unsettled; verify with counsel. |
| OWASP Top 10 for LLM Applications (2025) and MITRE ATLAS | Threat taxonomies | Named attack classes and adversary techniques against AI systems | Use them to test whether your controls cover real attacks, not just policy language |
- NIST AI RMF as the control backbone. Its Govern, Map, Measure and Manage functions map cleanly to the operating model, inventory, testing and incident processes in this article.
- ISO/IEC 42001 as the auditable wrapper. If customers or regulators ask for certification, the management-system structure is already what you are building.
- EU AI Act as the risk-tier driver. Even for non-EU organizations, its categories are a useful starting taxonomy for your internal tiers (Section 4), and EU-facing products need them anyway.
- OWASP and ATLAS as the test plan. Controls are credible only if you can show which named attacks they stop.
Governance fails most often because nobody can say who decides. The operating model below uses the familiar three-lines structure so it fits existing risk and audit processes instead of creating a parallel bureaucracy.
Owns policy, risk appetite, tier-1 approvals and exceptions
| Decision | Accountable | Responsible | Consulted |
|---|---|---|---|
| Risk appetite and policy changes | AI Governance Board | CISO office, Legal | Business units, Audit |
| Approve a new tier-1 (high-risk) use case | AI Governance Board | System owner | Security, Privacy, Legal |
| Approve a tier-2 or tier-3 use case | Security + Privacy reviewers | System owner | Legal as needed |
| Add a model or dataset to the approved registry | Security Architecture | ML Platform team | Procurement, Legal (licenses) |
| Declare and run an AI security incident | CISO | Incident response, system owner | Legal, Communications |
| Grant a policy exception | AI Governance Board | Requesting owner | Security, Risk (time-boxed, logged) |
You cannot govern what you cannot list. The inventory is the single most valuable artifact in the program: it drives risk tiering, review depth, vendor scrutiny, incident scoping and regulatory reporting. Treat it as a system of record, not a spreadsheet someone updates once a year.
# ai_inventory/payments-support-assistant.yaml system_id: AI-0142 name: Payments Support Assistant owner: jane.doe@example.com # a named, accountable person business_purpose: Draft replies to customer payment queries for agent review lifecycle: production # proposed | pilot | production | retired sourcing: third_party_api # built | fine_tuned | open_weights | third_party_api | embedded_saas model: vendor-model-x (version pinned: 2026-08-01) data_classes: [customer_pii, payment_metadata] data_leaves_org: true # sent to an external provider? retrieval_sources: [kb_payments_v3] # RAG corpora, with their own owners tools_and_actions: [read_ticket, draft_reply] # no send, no refund human_in_loop: required_before_send affects_individuals: true # influences decisions about people? risk_tier: 2 # computed, then confirmed by a reviewer last_review: 2026-09-12 next_review: 2027-03-12
Tiering should be a repeatable function of facts in the record, then confirmed by a human reviewer. This version is intentionally conservative: any one high-impact trait pushes a system up a tier.
# risk_tier.py: compute a starting tier from inventory facts (1 = highest scrutiny) PROHIBITED_PURPOSES = {"social_scoring", "emotion_inference_workplace", "untargeted_face_scraping"} CONSEQUENTIAL_DOMAINS = {"employment", "credit", "insurance", "education", "housing", "healthcare", "essential_services", "law_enforcement"} SENSITIVE = {"special_category", "payment_card", "credentials", "health"} def starting_tier(s: dict) -> int: if s.get("purpose_category") in PROHIBITED_PURPOSES: return 0 # 0 = blocked; no exception path without board approval score = 0 if s.get("domain") in CONSEQUENTIAL_DOMAINS and s.get("affects_individuals"): score += 3 # consequential decisions about people if set(s.get("data_classes", [])) & SENSITIVE: score += 2 if s.get("data_leaves_org"): score += 1 if s.get("external_facing"): score += 1 if s.get("autonomous_actions") and not s.get("human_in_loop"): score += 3 # can act with side effects and no approval if s.get("sourcing") == "open_weights" and not s.get("model_scanned"): score += 1 if score >= 5: return 1 if score >= 3: return 2 if score >= 1: return 3 return 4
| Tier | Typical System | Review Depth | Re-review |
|---|---|---|---|
| 0 Blocked | Purposes matching prohibited practices | Not permitted without board decision | n/a |
| 1 High | Consequential decisions about people; sensitive data; autonomous actions | Board approval; impact assessment; AI red team; legal review | 6 months and on material change |
| 2 Elevated | Customer-facing assistants; internal assistants with sensitive data | Security and privacy review; injection and leakage testing | 12 months |
| 3 Moderate | Internal productivity tools on non-sensitive data | Lightweight intake; vendor check; acceptable-use training | 12 to 18 months |
| 4 Low | Sandboxed experiments with synthetic or public data | Register only | On change |
A usable policy is short, testable and tied to controls. The clauses below form a core you can adapt. Each uses "must" language that an auditor can check, and each names the control from Section 6 that enforces it.
This is the minimum control set behind the policy. Each control addresses a named attack class, and most map to deeper guidance elsewhere in this series.
| # | Control | Threat Addressed | Implementation Notes |
|---|---|---|---|
| 1 | Discovery and inventory | Shadow AI; unknown exposure | Combine egress and DNS logs, SaaS and OAuth-grant review, code and secret scanning for AI API keys, and procurement data |
| 2 | Data controls | Sensitive data leakage to models and providers | Classification, DLP on AI-bound traffic, redaction before external calls, enterprise agreements with no-training terms |
| 3 | Model and package supply chain | Malicious models and packages; typosquatting | Approved registry, SafeTensors, pinned revisions, hash-verified installs, scanning. See AI Model Supply Chain Risks. |
| 4 | Pre-release adversarial testing | Prompt injection, jailbreaks, leakage, misuse | Automated probes plus manual red team for tier 1; fail the release on regressions. See LLM Prompt Injection. |
| 5 | Retrieval and context security | Indirect injection; authorization bypass; vector store poisoning | Per-user filtering at retrieval time, ingestion validation, context isolation. See RAG Security. |
| 6 | Agent and tool authorization | Excessive agency; confused-deputy actions | Per-task tool allowlists, scoped credentials, human approval for high-impact actions (code in Section 7) |
| 7 | Logging and monitoring | Undetected misuse; no audit trail | Structured logs keyed to system ID; alerts on anomalous tool use, policy violations and data-exfiltration patterns |
| 8 | Training-data integrity | Data and model poisoning | Dataset manifests, screening, trusted holdout and canary gates. See Model Poisoning Attacks. |
| 9 | MLOps platform hardening | Unauthenticated registries and tracking servers | Authentication, network segmentation, patching, immutable audit logs on registry writes |
| 10 | Incident response and rollback | Slow containment of a compromised or misbehaving model | AI-specific playbook, retained prior versions, rollback drills, kill switch per system |
Two areas outrun most governance programs: AI that nobody told you about, and AI that can act. Both need controls that work without waiting for a human to notice.
Employees adopt AI tools because they are useful, so a ban without an approved alternative mostly drives usage out of sight. Start with discovery, then offer a sanctioned option for the common cases. The sources that find the most are:
- Egress, proxy and DNS logs for traffic to AI provider domains
- OAuth and SaaS grants where an AI app has been given access to mail, files or chat
- Browser extensions that read page content and send it to AI services
- Source repositories and CI secrets containing AI provider API keys
- Expense and procurement data showing AI subscriptions
# shadow_ai_scan.py: summarize proxy-log traffic to known AI provider domains import csv, collections AI_DOMAINS = { "api.openai.com", "chatgpt.com", "api.anthropic.com", "claude.ai", "generativelanguage.googleapis.com", "gemini.google.com", "api.mistral.ai", "api.cohere.com", "huggingface.co", } SANCTIONED = {"api.anthropic.com"} # replace with your approved, contracted services def scan(log_csv: str): # expected columns: user, dest_host, bytes_out by_host = collections.defaultdict(lambda: {"users": set(), "bytes": 0}) with open(log_csv, newline="") as f: for row in csv.DictReader(f): host = row["dest_host"].lower() if host in AI_DOMAINS: by_host[host]["users"].add(row["user"]) by_host[host]["bytes"] += int(row["bytes_out"]) for host, d in sorted(by_host.items(), key=lambda kv: -kv[1]["bytes"]): status = "sanctioned" if host in SANCTIONED else "REVIEW" print(f"{status:10} {host:40} users={len(d['users']):4} MB_out={d['bytes']/1e6:8.1f}") # Feed REVIEW rows into the inventory intake: register, replace with a sanctioned tool, or block.
An agent that can call tools is a new kind of principal. Authorization should be decided outside the model, in code the model cannot rewrite, and every decision should be logged.
# agent_policy.py: deny-by-default authorization for agent tool calls from dataclasses import dataclass import json, time READ, WRITE, IRREVERSIBLE = "read", "write", "irreversible" @dataclass class ToolPolicy: impact: str # read | write | irreversible needs_approval: bool max_calls_per_task: int # Per-agent allowlist. Anything not listed is denied. POLICIES = { "support-assistant": { "read_ticket": ToolPolicy(READ, False, 20), "draft_reply": ToolPolicy(WRITE, False, 5), "send_reply": ToolPolicy(IRREVERSIBLE, True, 1), } } def authorize(agent: str, tool: str, call_count: int, approved_by: str | None) -> bool: policy = POLICIES.get(agent, {}).get(tool) allowed, reason = False, "not_in_allowlist" if policy: if call_count >= policy.max_calls_per_task: reason = "rate_limit" elif policy.needs_approval and not approved_by: reason = "approval_required" else: allowed, reason = True, "ok" # Audit every decision, allowed or denied print(json.dumps({"ts": int(time.time()), "agent": agent, "tool": tool, "allowed": allowed, "reason": reason, "approved_by": approved_by})) return allowed # Note: approved_by must come from your approval system, never from model output.
Most enterprise AI exposure arrives through vendors: AI features added to existing SaaS, API model providers, and agencies using AI on your behalf. Standard security questionnaires miss the AI-specific questions below.
| Area | Ask the Vendor | Red Flag |
|---|---|---|
| Data use | Is our data used to train or improve models? Can that be contractually excluded? | Opt-out only, or terms that can change without notice |
| Retention | How long are prompts, outputs and uploaded files stored, and where? | No stated retention; no deletion mechanism |
| Subprocessors | Which model providers and subprocessors see our data? | Undisclosed or frequently changing chain |
| Model provenance | What models power the feature, and how are versions controlled and notified? | Silent model swaps that change behavior |
| Security testing | Do you test for prompt injection, leakage and misuse? Can we see a summary? | "The model provider handles that" |
| Agent features | What actions can the AI take in our tenant, and can we restrict them? | Broad default permissions with no admin controls |
| Assurance | ISO/IEC 42001 certification, SOC 2 scope covering AI features? | Certifications that exclude the AI components |
| Incidents | How and how fast will you notify us of an AI-related incident? | No AI-specific notification commitment |
A governance program is credible when it can show numbers that move. Choose a small set that reflect control operation, not activity, and make each one reproducible from system data.
| Metric | Definition | Direction |
|---|---|---|
| Inventory coverage | Registered systems divided by systems found through discovery | Toward 100% |
| Tiered and reviewed | Share of systems with a current tier and a review inside its cadence | Toward 100% |
| Unapproved model usage | Models or datasets in use that are not in the approved registry | Toward zero |
| Release-gate pass rate | Tier 1 and 2 releases passing adversarial tests on first attempt, and number blocked | Track trend; blocks are a healthy sign |
| Agent actions with approval | High-impact agent actions executed with a recorded human approval | Toward 100% |
| Time to contain | Median time from AI incident detection to containment or rollback | Downward |
| Exceptions open | Policy exceptions active, and how many are past expiry | Downward; expired should be zero |
| Training and awareness | Staff completing acceptable-use training for AI tools | Toward 100% |
- The current inventory export and the discovery report it was reconciled against
- Risk assessments and approvals for a sample of tier 1 and tier 2 systems
- Registry records with hashes, scan results and approvers for models in production
- Red-team or adversarial test reports and the remediation tickets they created
- Agent approval logs and denied-call logs for a chosen period
- Incident records, post-incident reviews and the latest rollback drill
- The exception register with owners and expiry dates
Do not try to launch everything at once. The sequence below delivers visibility first, then enforcement, then assurance.
- ☐An executive sponsor and AI governance board exist, with a written charter and decision rights
- ☐A living AI inventory exists with a named owner, data classes and risk tier for every system
- ☐Discovery runs on a schedule and its findings are reconciled against the inventory
- ☐The AI security policy is approved, published, and each clause maps to a control and a metric
- ☐Models and datasets come from an approved, hash-verified registry; remote-code execution is prohibited by default
- ☐Restricted data is blocked from unapproved AI services by DLP and egress rules
- ☐Agents use deny-by-default tool allowlists with human approval for high-impact actions
- ☐Tier 1 and 2 releases are gated on adversarial testing and logged against a system ID
- ☐Vendor onboarding includes AI-specific questions and contract terms on training use and model changes
- ☐An AI incident playbook exists and a rollback drill has been completed in the last 12 months