Skip to content

Overview

Architecture Overview

Real-Time Voice AI Accelerator

Azure Communication Services voice agent accelerator with modular AI agents, real-time audio processing, and enterprise deployment patterns.

Core Capabilities

Feature What's Included Purpose
Real-time Audio ACS + Speech Services integration Voice conversation processing
AI Agent Framework Modular, swappable agent system Industry-specific implementations
Intelligent Barge-in Voice activity detection patterns Natural conversation flow
Serverless Scaling Container Apps with auto-scaling Cost-effective, elastic hosting
Development Ready Public endpoints with managed identity Quick deployment and testing

Deployment Architecture Options

Current Terraform: Container Apps with public endpoints for rapid development

Available Bicep: Enterprise production architecture with API Management, and private networking. (Advanced, WIP)

Deployment Architecture

Streamlined deployment with Container Apps and public endpoints

Simplified Production Architecture

Current Terraform deployment with Container Apps, AI Foundry, and public endpoints. App Gateway, APIM, and private networking are intentionally excluded to maintain simplicity and flexibility for rapid development.

Agent framework and processing pipeline architecture

Agent Architecture

Detailed view of the agent orchestration, processing components, and data flow patterns within the simplified production architecture.

Real-time voice processing with live orchestration

Voice Live API

Voice live orchestration architecture showing real-time audio processing, conversation management, and agent coordination patterns.

Low-code intent mapping with Dynamics and ACS orchestration

Dynamics IVR Bridge

Dynamics 365 handles IVR intent mapping and automated workflows, then performs a SIP transfer bridge into Azure Communication Services for custom real-time orchestration with the RT Agent.

Infrastructure Deployment Approach

The Terraform deployment intentionally excludes App Gateway, API Management, and private networking to provide a malleable foundation that consumers can extend based on their specific requirements. Production enterprise features are available through separate Bicep templates.

Azure infrastructure with Container Apps, AI Foundry, and public endpoints

Microsoft Learn Resources

Current Terraform Deployment

Simplified Public Infrastructure - The Terraform deployment creates a streamlined development-focused architecture with public endpoints and Container Apps hosting. Advanced features like API Management, AI Gateway, private networking, and Application Gateway are available in the Bicep templates for production scenarios.

Key Infrastructure Components

Container Apps Environment:

  • Auto-scaling - KEDA-based scaling for frontend and backend containers
  • Public Ingress - External endpoints for development and testing
  • Managed Identity - Azure AD authentication across all services
  • Application Insights - Centralized logging and monitoring

AI Services:

  • Azure AI Foundry - LLM Model hosting, unified resource for Speech/Cognitive Services

Data Layer:

  • Cosmos DB (MongoDB API) - Session and conversation storage
  • Redis Enterprise - High-performance caching with RBAC
  • Storage Account - Audio files and prompt storage
  • Key Vault - Secure secret management

Advanced Networking:

  • Hub-spoke VNet topology with private endpoints
  • Application Gateway with WAF protection
  • NSG rules and traffic control

API Management & AI Gateway:

  • Token management and PTU optimization
  • Load balancing and cost analytics
  • Content safety and multi-region routing

Deployment Comparison

Terraform: Streamlined development infrastructure with public endpoints and Container Apps

Bicep: Enterprise-grade production architecture with private networking, API Gateway, and Application Gateway

Microsoft Learn References:

Architecture Deep Dives

Section Documents What You'll Learn
Agents Framework, Handoffs YAML-driven agent system, multi-agent coordination, handoff patterns
Orchestration Cascade, VoiceLive Dual orchestrator modes, event handling, turn processing
Speech Modes, Recognition, Synthesis Audio streaming, STT/TTS integration, handler selection
Data Session, Flows MemoManager, Redis patterns, state persistence
ACS Call Flows, Integrations ACS media handling, telephony integration, IVR bridging
Telemetry Observability OpenTelemetry, App Insights, SLOs and alerting

Quick Start Paths

  1. Getting Started - Environment setup and prerequisites
  2. Local Development - Run the accelerator locally
  3. API Reference - Endpoints and WebSocket protocols
  1. Data Flow Patterns - Storage strategies and state management
  2. Production Deployment - Infrastructure and scaling
  3. ACS Integrations - External service connectivity
  1. Monitoring Guide - Application insights and observability
  2. Load Testing - Performance validation and capacity planning
  3. Troubleshooting - Issue resolution and debugging

Additional Resources

For more comprehensive guidance on development and operations: