Getting Started

Architecture Overview

Architecture Overview

GovernanceAI is built as a distributed, multi-tenant platform designed for enterprise scale. This document explains the major components and how they interact.

High-Level Architecture

┌─────────────────────────────────────────────────────────────────┐
│                         Client Applications                      │
│                 (Your LLMs, Agents, Services)                   │
└────────────────────┬────────────────────────────────────────────┘
                     │
         ┌───────────┴──────────────┐
         │                          │
    ┌────▼──────┐          ┌────────▼─────┐
    │ Dashboard │          │  APIs/SDKs   │
    │    UI     │          │  (REST, gRPC)│
    └────┬──────┘          └────────┬─────┘
         │                          │
         └──────────┬───────────────┘
                    │
        ┌───────────▼───────────────┐
        │   Control Plane (CP)      │
        │  ┌─────────────────────┐  │
        │  │ Auth & Session Mgmt │  │
        │  │ Policy Management   │  │
        │  │ Configuration       │  │
        │  └─────────────────────┘  │
        └───────────┬───────────────┘
                    │
    ┌───────────────┼───────────────┐
    │               │               │
┌───▼───┐    ┌──────▼──────┐  ┌─────▼─────┐
│Runtime│    │  Scanner &  │  │  Reporter │
│Engine │    │  Inventory  │  │ & Auditor │
└───┬───┘    └──────┬──────┘  └─────┬─────┘
    │               │               │
    └───────────────┴───────────────┘
            │
    ┌───────▼──────────┐
    │  Observability   │
    │ (Logs, Metrics)  │
    └──────────────────┘

Core Components

1. Control Plane (CP)

The Control Plane is the central management hub for the entire system.

Responsibilities:

  • Authentication & Authorization - Handles login, OAuth, SSO, and token management
  • Session Management - Maintains user sessions with secure cookies
  • Organization & Workspace Management - Multi-tenant hierarchy, user roles, permissions
  • Policy & Guardrail Configuration - Define and manage governance policies
  • Settings & Integration Management - Configure third-party integrations (GitHub, Jira, etc.)

Access Patterns:

  • Web Dashboard (UI) - For interactive configuration
  • Control Plane APIs - Programmatic configuration (requires OAuth session)

2. Runtime Engine

The Runtime Engine enforces policies on live AI requests and responses.

Responsibilities:

  • Request Validation - Check incoming LLM requests against policies
  • Response Evaluation - Analyze LLM outputs for compliance violations
  • Policy Enforcement - Block, log, or transform requests/responses based on rules
  • Agent Guardrails - Control tool execution and actions in multi-step agents
  • Real-time Decision Making - Sub-millisecond latency policy enforcement

Access Patterns:

  • Runtime API - Bearer token authentication (app API keys)
  • Alternative Headers - X-API-Key, X-App-API-Key, X-Pipeline-Token
  • LiteLLM Proxy Integration - Drop-in replacement for LLM providers

Typical Flow:

Your Application
    │
    ├─ Request: POST /v1/guardrails/evaluate
    │           Headers: Authorization: Bearer <app_api_key>
    │           Body: { "messages": [...], "context": {...} }
    │
    └─ GovernanceAI Runtime
        ├─ Load relevant policies
        ├─ Evaluate against guardrails
        ├─ Return decision (allow/block/transform)
        │
        └─ Response: { "decision": "allow", "transformed_response": {...} }

3. Scanner & Inventory

The Scanner component discovers and catalogs AI usage across your infrastructure.

Responsibilities:

  • Code Scanning - Analyze repositories for AI model usage, LLM calls, and dependencies
  • Model Inventory - Build AI Bill of Materials (AI BOM) with complete dependency trees
  • Dependency Analysis - Track model versions, licenses, and security patches
  • Integration Scanning - Discover connected integrations and their permissions
  • Compliance Gap Analysis - Identify deviations from compliance frameworks

Access Patterns:

  • Scheduled Scans - Automatic discovery on interval
  • On-Demand Scans - Triggered via API or Dashboard
  • Integration Webhooks - Triggered by repository events (push, PR)

4. Reporter & Auditor

The Reporter component generates insights and audit trails.

Responsibilities:

  • Activity Logging - Complete audit trail of all system actions
  • Report Generation - PDF, JSON, CSV exports for compliance
  • Compliance Reporting - Framework-specific reports (SOC2, HIPAA, etc.)
  • Dashboard Rendering - Executive and operational dashboards
  • Data Retention - Long-term storage and archival of audit logs

Access Patterns:

  • Dashboard UI - View reports and metrics
  • Report APIs - Programmatic access to reports
  • Export APIs - Download reports in various formats

Data Flow Examples

Example 1: Runtime Policy Enforcement

- Your Application
   ├─ Calls LLM with user query
   └─ Receives response from model
- Your Application → GovernanceAI Runtime
   POST /v1/guardrails/evaluate
   {
     "messages": [{"role": "user", "content": "..."}],
     "response": "Generated answer...",
     "context": {"user_id": "123", "org_id": "456"}
   }
- GovernanceAI Runtime
   ├─ Fetches policies for org_id
   ├─ Runs policy evaluation engine
   ├─ Logs activity for audit trail
   └─ Returns decision
- Response → Your Application
   {
     "decision": "allow",
     "policy_violations": [],
     "transformed_response": "Safe version of response"
   }
- Your Application
   └─ Returns response to user

Example 2: Compliance Report Generation

- User in Dashboard
   └─ Clicks "Generate SOC2 Report"
- Control Plane
   ├─ Fetches policy state
   ├─ Queries audit logs
   ├─ Calls Reporter component
   └─ Returns report metadata
- Reporter
   ├─ Aggregates all audit events
   ├─ Maps to SOC2 controls
   ├─ Generates PDF
   └─ Stores for audit trail
- Dashboard
   └─ Displays download link to user

Deployment Models

SaaS (Multi-tenant)

  • Hosted by GovernanceAI - No infrastructure management
  • Shared Control Plane - Segregated by organization
  • Isolated Data - Your data never leaves your organization’s tenant
  • Automatic Updates - Latest features and security patches
  • Global Availability - Multiple regions, high availability

On-Premise

  • Your Infrastructure - Deploy in your own data center or cloud
  • Private Control Plane - Dedicated instance for your organization
  • Air-gapped Option - No external connectivity required
  • Custom Integrations - Extend with internal systems
  • Long-term Support - Stable versions with extended support

Security Architecture

  • TLS Encryption - All traffic encrypted in transit
  • RBAC - Role-based access control throughout
  • Token-based Auth - Stateless API authentication with tokens
  • Session Security - Secure cookies with SameSite, HttpOnly flags
  • Audit Logging - Complete audit trail of all actions
  • Data Isolation - Multi-tenant isolation at application and database levels

Scalability

  • Horizontal Scaling - Runtime engine scales with request volume
  • Caching - Policy caches reduce computation
  • Async Processing - Scans and reports run asynchronously
  • CDN Integration - Cache frequently accessed data globally
  • Database Optimization - Indexed queries for fast retrieval

Next Steps