Capability Control

Capability Control is the AI safety strategy of limiting what an AI system is physically able to do — independent of what it is trained to want to do — as a defense-in-depth measure against alignment failures — ensuring that even if an AI system's values or goals deviate from human intentions, its ability to cause harm is bounded by hard technical constraints.

What Is Capability Control?

• Definition: The practice of designing AI systems, their operating environments, and their infrastructure with explicit restrictions on what actions the AI can physically perform — regardless of whether it is "trying" to behave safely.
• Distinction from Alignment: Alignment attempts to make AI systems want to do good things. Capability control ensures AI systems can't do dangerous things even if alignment fails.
• Defense in Depth: Capability control is not a substitute for alignment — it is a complementary safety layer that provides a backstop when alignment is imperfect.
• Current Relevance: Most relevant for agentic AI systems (agents with tool use, internet access, code execution) where AI can take real-world actions rather than merely producing text.

Why Capability Control Matters

• Alignment Is Imperfect: Current alignment techniques cannot guarantee perfect value alignment across all situations. Capability control limits blast radius when alignment fails.
• Agentic AI Risk: As AI systems gain access to tools (web browsers, code execution, APIs, file systems), capability control becomes critical — a misaligned agent with broad access could cause catastrophic harm.
• Containment During Development: During the period of developing and testing increasingly powerful AI, capability control allows safety research to proceed while maintaining safety margins.
• Human Oversight: Capability restrictions preserve meaningful human oversight — humans remain able to monitor, correct, and shut down AI systems.
• Irreversible Actions: Some AI actions (sending emails, deleting files, making financial transactions, publishing content) are hard to reverse. Capability control prevents irreversible errors.

Critical Capabilities to Control

Self-Replication:
- An AI should not be able to copy its weights, training code, or operational configuration to external systems without explicit human authorization.
- Why: Self-replication enables persistence beyond authorized deployment and undermines human ability to shut down or modify the system.
- Control: Restrict file system write access to designated directories; no network access to untrusted endpoints.

Resource Acquisition:
- An AI should not be able to autonomously acquire computational resources, financial resources, or access credentials.
- Why: Resource acquisition enables unbounded capability scaling and creates persistent operations beyond sanctioned scope.
- Control: No direct API access to cloud providers, financial systems, or credential stores without human-in-the-loop authorization.

Internet and Network Access:
- Unrestricted internet access enables data exfiltration, malware download, contacting external services, and recruiting human accomplices.
- Control: Allowlist-based internet access; monitoring and logging all network requests; blocking direct connections to sensitive services.

Code Execution Scope:
- Restrict what code an AI can execute — limit to sandboxed environments (containers, VMs) with no persistent storage or network access by default.
- Why: Arbitrary code execution is the highest-risk capability, enabling privilege escalation and environment escape.
- Control: Docker containers, gVisor sandboxes, separate user accounts with minimal permissions.

Tool Use Boundaries:
- For agentic AI systems with tool access (web browsers, email, APIs), define explicit allowlists of permitted tools and actions.
- Require human approval for high-stakes or irreversible actions (sending emails, making purchases, modifying databases).

Capability Control in Practice

Minimal Footprint Principle:
- AI agents should request and use only the permissions, resources, and capabilities actually needed for the current task — not accumulate permissions "just in case."
- Prefer reversible actions over irreversible ones when both achieve the goal.
- Default to asking for human confirmation when uncertain about scope.

Sandboxing Architecture:
- Run AI systems in isolated compute environments with no persistent state between sessions unless explicitly granted.
- Separate AI 'working memory' from production systems — AI can read but not directly write production databases.
- Log all tool calls and actions for human audit.

Tripwires and Circuit Breakers:
- Monitor for anomalous behavior patterns (unusual resource requests, unexpected network connections, high-volume API calls).
- Automatic shutdown or human notification when behavior exceeds defined parameters.

Capability Control vs. Alignment

Capability control is the architectural safety harness that makes AI development safer during the critical period before we have robust alignment guarantees — by ensuring that even imperfectly aligned AI systems cannot take catastrophic or irreversible actions without human oversight, capability control buys the time and error tolerance needed to develop AI alignment into a mature, reliable engineering discipline.