The comprehensive failure intelligence platform. Predict, detect, and recover from AI agent failures before they reach production.
pip install failsafe
Your AI agents are running in production right now. When they fail, most teams don't find out until the damage is done.
Your customer-facing AI confidently cites a nonexistent return policy. By the time support flags it, 2,000 customers have received false information and your brand trust is eroding.
An API timeout in one agent triggers a retry storm. The retry storm overwhelms your rate limits. Downstream agents get stale data. One failure becomes twenty in under sixty seconds.
A code-generation agent with access to your production database starts executing write operations it was never authorized to perform. No guardrails. No alerts. No audit trail.
FailSafe gives you full-spectrum visibility into how your AI agents fail, why they fail, and how to recover automatically.
Any AI operation
Real-time analysis
Classify failure
Auto-remediate
Over 150 documented failure modes across 8 categories. Each failure mode includes detection heuristics, prevention strategies, recovery procedures, and real-world examples. Built from production incident data, not theory.
150+ failure modesLightweight monitors that run alongside your agents. Detect hallucinations, safety violations, cascade failures, and behavioral drift in milliseconds. Sub-1ms overhead per operation. Zero impact on agent performance.
<1ms latencyDefine recovery strategies for every failure mode. Retry with backoff, fallback to safe defaults, circuit-break cascading failures, or escalate to human operators. Recovery executes in under 300 milliseconds.
Auto-remediationEvery failure event logged with full context: what failed, when, why, and what recovery action was taken. Exportable audit trails for SOC 2, HIPAA, and ISO 27001 compliance requirements.
SOC 2 readyA structured, machine-readable classification of every way AI agents can fail. Click any category to explore specific failure modes.
False assertions, fabricated data
Timeouts, loops, resource exhaustion
APIs, auth, network partitions
Boundary breaches, unauthorized access
Gradual degradation, reward hacking
Outdated info, wrong tool selection
Corruption, race conditions, state
Output decline, latency, error growth
Go from zero failure coverage to full observability in under ten minutes. No infrastructure changes required.
# Install the SDK $ pip install failsafe # Or use npm $ npm install @failsafe/core
# failsafe.yaml monitors: - hallucination - safety_violation - cascade_failure - behavioral_drift recovery: auto
from failsafe import Monitor monitor = Monitor( config="failsafe.yaml" ) monitor.start()
@monitor.on_failure def handle(event): if event.recoverable: event.auto_recover() else: event.escalate()
Native libraries for Python and TypeScript. REST API for everything else. Production-ready in minutes.
from failsafe import FailSafe, Monitor, RecoveryStrategy # Initialize FailSafe with your configuration fs = FailSafe( monitors=[ Monitor.HALLUCINATION, Monitor.SAFETY_VIOLATION, Monitor.CASCADE_FAILURE, Monitor.BEHAVIORAL_DRIFT, ], recovery=RecoveryStrategy.AUTO, audit_log="./logs/failsafe.jsonl", ) # Wrap your agent with FailSafe monitoring @fs.monitor async def run_agent(query: str) -> str: response = await llm.generate(query) return response # Register custom recovery handlers @fs.on_failure("hallucination") async def handle_hallucination(event): # Re-run with stricter temperature and grounding return await llm.generate( event.original_query, temperature=0.1, grounding=True, ) # Access failure analytics report = fs.get_report(last_hours=24) print(f"Failures detected: {report.total_failures}") print(f"Auto-recovered: {report.auto_recovered}") print(f"Escalated: {report.escalated}")
import { FailSafe, Monitor, RecoveryStrategy } from '@failsafe/core'; // Initialize FailSafe with typed configuration const fs = new FailSafe({ monitors: [ Monitor.HALLUCINATION, Monitor.SAFETY_VIOLATION, Monitor.CASCADE_FAILURE, Monitor.EXECUTION_TIMEOUT, ], recovery: RecoveryStrategy.AUTO, auditLog: './logs/failsafe.jsonl', }); // Monitor any async function const safeAgent = fs.wrap(async (query: string): Promise<string> => { const response = await llm.generate(query); return response.text; }); // Custom recovery for specific failure types fs.onFailure('hallucination', async (event) => { return await llm.generate(event.originalQuery, { temperature: 0.1, grounding: true, }); }); // Run with full protection const result = await safeAgent('Analyze Q4 revenue trends'); console.log(result);
# Submit an agent action for analysis $ curl -X POST https://api.failsafe.dev/v1/analyze \ -H "Authorization: Bearer fs_live_..." \ -H "Content-Type: application/json" \ -d '{ "agent_id": "support-agent-v2", "action": "generate_response", "input": "What is your return policy?", "output": "Our return policy allows...", "monitors": ["hallucination", "safety_violation"] }' # Response { "status": "pass", "failures_detected": 0, "monitors": { "hallucination": { "score": 0.02, "pass": true }, "safety_violation": { "score": 0.00, "pass": true } }, "latency_ms": 12 } # Get failure analytics $ curl https://api.failsafe.dev/v1/report?hours=24 \ -H "Authorization: Bearer fs_live_..."
Build failure reports, calculate composite risk scores, and integrate FailSafe into any Python workflow with a single pip install.
pip install failsafe-ai
npm install @anthropic/failsafe-sdk
The Python package provides a fluent builder API for constructing structured failure reports, a composite scoring engine for risk assessment, and seamless integration with popular AI frameworks.
from failsafe import FailureReportBuilder, calculate_composite_score # Build a structured failure report report = FailureReportBuilder()\ .set_title("Hallucination in customer-facing bot")\ .set_severity("high")\ .set_category("output-quality")\ .set_failure_type("hallucination")\ .build() # Calculate composite risk score score = calculate_composite_score(report) print(f"Risk score: {score.overall}") print(f"Severity: {score.severity_weight}") print(f"Impact: {score.impact_estimate}")
Drop-in integration handlers for CrewAI, LangChain, and OpenAI. Add failure intelligence to your existing agents without rewriting a single line of business logic.
Monitor multi-agent crews with automatic failure detection across agent handoffs, task delegation, and collaborative reasoning chains.
from failsafe.integrations.crewai import \ FailSafeCrewAIHandler # Attach to your crew handler = FailSafeCrewAIHandler() crew = Crew( agents=[researcher, writer], callbacks=[handler] )
Integrate with LangChain chains and agents. Automatically monitor LLM calls, tool usage, retrieval quality, and chain-of-thought reasoning.
from failsafe.integrations.langchain import \ FailSafeLangChainHandler # Add as a callback handler handler = FailSafeLangChainHandler() chain = LLMChain( llm=llm, callbacks=[handler] )
Wrap the OpenAI client with FailSafe monitoring. Catch hallucinations, token limit issues, and safety violations on every API call.
from failsafe.integrations.openai import \ FailSafeOpenAIWrapper # Wrap your OpenAI client wrapper = FailSafeOpenAIWrapper( api_key="sk-..." ) response = wrapper.chat(prompt)
Use FailSafe as a ChatGPT Custom GPT Action via OpenAPI, or integrate natively with Claude through the Model Context Protocol (MCP).
Turn FailSafe into a ChatGPT Skill using the OpenAPI specification. Your Custom GPT can submit failure reports, query the taxonomy, and calculate risk scores through natural conversation.
Copy the OpenAPI spec from spec/openapi.yaml in the GitHub repo
In ChatGPT, go to Explore GPTs → Create → Configure → Actions
Paste the OpenAPI schema and set authentication. Your GPT can now report and analyze AI failures.
Integrate FailSafe directly with Claude via the Model Context Protocol. The MCP server exposes failure reporting tools that Claude can invoke natively during conversations.
Install the MCP server: npm install @anthropic/failsafe-mcp
Add to your Claude Desktop config or MCP client configuration
Claude can now submit failure reports, browse the taxonomy, and score risks using FailSafe tools
Domain-specific failure detection tuned to the unique risks and compliance requirements of your industry.
Catch diagnostic errors, drug interaction hallucinations, compliance violations, and treatment recommendation failures before they impact patient outcomes. HIPAA-compliant audit trails included.
Prevent fraudulent transaction approvals, detect data integrity violations in real-time trading systems, and enforce regulatory compliance across automated financial workflows.
Detect hallucinated case citations, fabricated statutes, and incorrect legal precedent references. Prevent confidentiality breaches and ensure accuracy in contract analysis and legal research.
Catch agent hallucinations before they reach customers. Monitor response quality in real time, enforce policy compliance, and maintain consistent service across thousands of concurrent conversations.
Safety-critical failure prevention for robotics, self-driving, and industrial automation. Real-time boundary monitoring, emergency shutdown protocols, and redundant safety verification layers.
Prevent insecure code output, catch syntax errors and logic bugs, validate generated solutions against test suites, and block code that introduces known vulnerabilities or anti-patterns.
A complete toolkit for AI failure intelligence. From SDKs to storage, monitoring to compliance.
Language-agnostic HTTP endpoints
Native Python with async support
Fully typed with generics
Model Context Protocol native
Terminal-first workflows
Zero-config local persistence
Full compliance audit trail
Guides, API ref, examples
Open source core. Enterprise plans for teams that need SLA guarantees, dedicated support, and custom failure mode development.
npm install @failsafe/core