Skip to content

Overview

Architecture Overview

Real-Time Voice AI Accelerator

Azure Communication Services voice agent accelerator with modular AI agents, real-time audio processing, and enterprise deployment patterns.

Core Capabilities

Feature What's Included Purpose
Real-time Audio ACS + Speech Services integration Voice conversation processing
AI Agent Framework Modular, swappable agent system Industry-specific implementations
Intelligent Barge-in Voice activity detection patterns Natural conversation flow
Serverless Scaling Container Apps with auto-scaling Cost-effective, elastic hosting
Development Ready Public endpoints with managed identity Quick deployment and testing

Deployment Architecture Options

Current Terraform: Container Apps with public endpoints for rapid development

Available Bicep: Enterprise production architecture with API Management, and private networking. (Advanced, WIP)

Deployment Architecture

Streamlined deployment with Container Apps and public endpoints

Simplified Production Architecture

Current Terraform deployment with Container Apps, AI Foundry, and public endpoints. App Gateway, APIM, and private networking are intentionally excluded to maintain simplicity and flexibility for rapid development.

Agent framework and processing pipeline architecture

Agent Architecture

Detailed view of the agent orchestration, processing components, and data flow patterns within the simplified production architecture.

Real-time voice processing with live orchestration

Voice Live API

Voice live orchestration architecture showing real-time audio processing, conversation management, and agent coordination patterns.

Infrastructure Deployment Approach

The Terraform deployment intentionally excludes App Gateway, API Management, and private networking to provide a malleable foundation that consumers can extend based on their specific requirements. Production enterprise features are available through separate Bicep templates.

Azure infrastructure with Container Apps, AI Foundry, and public endpoints

Microsoft Learn Resources

Current Terraform Deployment

Simplified Public Infrastructure - The Terraform deployment creates a streamlined development-focused architecture with public endpoints and Container Apps hosting. Advanced features like API Management, AI Gateway, private networking, and Application Gateway are available in the Bicep templates for production scenarios.

Key Infrastructure Components

Container Apps Environment:

  • Auto-scaling - KEDA-based scaling for frontend and backend containers
  • Public Ingress - External endpoints for development and testing
  • Managed Identity - Azure AD authentication across all services
  • Application Insights - Centralized logging and monitoring

AI Services:

  • Azure AI Foundry - LLM Model hosting, unified resource for Speech/Cognitive Services

Data Layer:

  • Cosmos DB (MongoDB API) - Session and conversation storage
  • Redis Enterprise - High-performance caching with RBAC
  • Storage Account - Audio files and prompt storage
  • Key Vault - Secure secret management

Advanced Networking:

  • Hub-spoke VNet topology with private endpoints
  • Application Gateway with WAF protection
  • NSG rules and traffic control

API Management & AI Gateway:

  • Token management and PTU optimization
  • Load balancing and cost analytics
  • Content safety and multi-region routing

Deployment Comparison

Terraform: Streamlined development infrastructure with public endpoints and Container Apps

Bicep: Enterprise-grade production architecture with private networking, API Gateway, and Application Gateway

Microsoft Learn References:

Architecture Deep Dives

Document Focus What You'll Learn
LLM Orchestration AI routing and conversation management Multi-agent coordination, dependency injection patterns, orchestrator design
Speech Recognition Real-time STT processing Azure Speech integration, WebSocket handling, and transcription accuracy
Speech Synthesis Dynamic TTS generation Low-latency audio synthesis, voice font customization, and output streaming
ACS Call Flows Three-thread voice processing Real-time audio handling, WebSocket patterns, media lifecycle
Data Flows Storage and caching patterns State management, Redis coordination, Cosmos DB persistence
Integrations Cross-cloud connectivity External service patterns, authentication flows

Quick Start Paths

  1. Getting Started - Environment setup and prerequisites
  2. Local Development - Run the accelerator locally
  3. API Reference - Endpoints and WebSocket protocols
  1. Data Flow Patterns - Storage strategies and state management
  2. Production Deployment - Infrastructure and scaling
  3. Integrations Overview - External service connectivity
  1. Monitoring Guide - Application insights and observability
  2. Load Testing - Performance validation and capacity planning
  3. Troubleshooting - Issue resolution and debugging

Additional Resources

For more comprehensive guidance on development and operations: