Skip to content

Local Development

⚡ Local Development

Run the ARTVoice Accelerator locally with raw commands. No Makefile usage. Keep secrets out of git and rotate any previously exposed keys.

1. Scope

What this covers:

  • Local backend (FastAPI + Uvicorn) and frontend (Vite/React)
  • Dev tunnel for inbound Azure Communication Services callbacks
  • Environment setup via venv OR Conda
  • Minimal .env files (root + frontend)

What this does NOT cover: - Full infra provisioning - CI/CD - Persistence hardening

2. Prerequisites

Tool Notes
Python 3.11 Required runtime
Node.js ≥ 22 Frontend
Azure CLI az login first
Dev Tunnels Getting Started Guide
(Optional) Conda If using environment.yaml
Provisioned Azure resources For real STT/TTS/LLM/ACS

If you only want a browser demo (no phone), ACS variables are optional.

3. Clone Repository

git clone https://github.com/Azure-Samples/art-voice-agent-accelerator.git
cd art-voice-agent-accelerator

4. Python Environment (Choose One)

Option A: venv

python -m venv .venv
source .venv/bin/activate
pip install --upgrade pip
pip install -r requirements.txt

Option B: Conda

conda env create -f environment.yaml
conda activate audioagent
pip install -r requirements.txt   # sync with lock

5. Root .env (Create in repo root)

Sample Configuration

Use .env.sample as a starting template and customize with your Azure resource values.

Using Azure Developer CLI (azd)

If you provisioned infrastructure using azd provision, an environment file will be automatically generated for you in the format .env.<azd-env-name>.

To use the azd-generated configuration: 

# Copy the azd-generated environment file
cp .env.<your-azd-env-name> .env

# Example: if your azd environment is named "dev"
cp .env.dev .env

The azd-generated file contains all the Azure resource endpoints and configuration needed for local development.

Manual Configuration Template (edit placeholders; DO NOT commit real values):

# ===== Azure OpenAI =====
AZURE_OPENAI_ENDPOINT=https://<your-aoai>.openai.azure.com
AZURE_OPENAI_KEY=<aoai-key>
AZURE_OPENAI_DEPLOYMENT=gpt-4-1-mini
AZURE_OPENAI_API_VERSION=2024-12-01-preview
AZURE_OPENAI_CHAT_DEPLOYMENT_ID=gpt-4-1-mini
AZURE_OPENAI_CHAT_DEPLOYMENT_VERSION=2024-11-20

# ===== Speech =====
AZURE_SPEECH_REGION=<speech-region>
AZURE_SPEECH_KEY=<speech-key>

# ===== ACS (optional unless using phone/PSTN) =====
ACS_CONNECTION_STRING=endpoint=https://<your-acs>.communication.azure.com/;accesskey=<acs-key>
ACS_SOURCE_PHONE_NUMBER=+1XXXXXXXXXX
ACS_ENDPOINT=https://<your-acs>.communication.azure.com

# ===== Optional Data Stores =====
REDIS_HOST=<redis-host>
REDIS_PORT=6380
REDIS_PASSWORD=<redis-password>
AZURE_COSMOS_CONNECTION_STRING=<cosmos-conn-string>
AZURE_COSMOS_DATABASE_NAME=audioagentdb
AZURE_COSMOS_COLLECTION_NAME=audioagentcollection

# ===== Runtime =====
ENVIRONMENT=dev
ACS_STREAMING_MODE=media

# ===== Filled after dev tunnel starts =====
BASE_URL=https://<tunnel-url>

Ensure .env is in .gitignore.

6. Start Dev Tunnel

Required if you want ACS callbacks (phone flow) or remote test:

devtunnel host -p 8010 --allow-anonymous

Copy the printed HTTPS URL and set BASE_URL in root .env. Update it again if the tunnel restarts (URL changes).

The Dev Tunnel URL will look similar to: 

https://abc123xy-8010.usw3.devtunnels.ms

Security Considerations for Operations Teams

Dev Tunnels create public endpoints that expose your local development environment to the internet. Review the following security guidelines:

  • Azure Dev Tunnels Security - Comprehensive security guidance
  • Access Control: Use --allow-anonymous only for development; consider authentication for sensitive environments
  • Network Policies: Ensure dev tunnels comply with organizational network security policies
  • Monitoring: Dev tunnels should be monitored and logged like any public endpoint
  • Temporary Usage: Tunnels are for development only; use proper Azure services for production
  • Credential Protection: Never expose production credentials through dev tunnels

InfoSec Recommendation: Review tunnel usage with your security team before use in corporate environments.

7. Run Backend

cd apps/rtagent/backend
uvicorn apps.rtagent.backend.main:app --host 0.0.0.0 --port 8010 --reload

8. Frontend Environment

Create or edit apps/rtagent/frontend/.env:

Sample Configuration

Use apps/rtagent/frontend/.env.sample as a starting template.

Use the dev tunnel URL by default so the frontend (and any external device or ACS-related flows) reaches your backend consistently—even if you open the UI on another machine or need secure HTTPS.

# Recommended (works across devices / matches ACS callbacks)
VITE_BACKEND_BASE_URL=https://<tunnel-url>

If the tunnel restarts (URL changes), update both BASE_URL in the root .env and this value.

9. Run Frontend

cd apps/rtagent/frontend
npm install
npm run dev

Open: http://localhost:5173

WebSocket URL is auto-derived by replacing http/https with ws/wss.

10. Alternative: VS Code Debugging

Built-in debugger configurations are available in .vscode/launch.json:

Backend Debugging

  1. Set breakpoints in Python code
  2. Press F5 or go to Run & Debug view
  3. Select "[RT Agent] Python Debugger: FastAPI"
  4. Debug session starts with hot reload enabled

Frontend Debugging

  1. Start the React dev server (npm run dev)
  2. Press F5 or go to Run & Debug view
  3. Select "[RT Agent] React App: Browser Debug"
  4. Browser opens with debugger attached

Benefits: - Set breakpoints in both Python and TypeScript/React code - Step through code execution - Inspect variables and call stacks - Hot reload for both frontend and backend

11. Alternative: Docker Compose

For containerized local development, use the provided docker-compose.yml:

# Ensure .env files are configured (see sections 5 & 8 above)

# Build and run both frontend and backend containers
docker-compose up --build

# Or run in detached mode
docker-compose up --build -d

# View logs
docker-compose logs -f

# Stop containers
docker-compose down

Container Ports:

  • Frontend: http://localhost:8080 (containerized)
  • Backend: http://localhost:8010 (same as manual setup)

When to use Docker Compose:

  • Consistent environment across team members
  • Testing containerized deployment locally
  • Isolating dependencies from host system
  • Matching production container behavior

Dev Tunnel with Docker

You still need to run devtunnel host -p 8010 --allow-anonymous for ACS callbacks, as the containers need external access for webhook endpoints.

12. Optional: Phone (PSTN) Flow

  1. Purchase ACS phone number (Portal or CLI).

  2. Ensure these vars are set in your root .env (with real values):

ACS_CONNECTION_STRING=endpoint=...
ACS_SOURCE_PHONE_NUMBER=+1XXXXXXXXXX
ACS_ENDPOINT=https://<your-acs>.communication.azure.com
BASE_URL=https://<tunnel-hash>-8010.usw3.devtunnels.ms
  1. Create a single Event Grid subscription for the Incoming Call event pointing to your answer handler:
  2. Inbound endpoint:
    https://<tunnel-hash>-8010.usw3.devtunnels.ms/api/v1/calls/answer
  3. Event type: Microsoft.Communication.IncomingCall
  4. (Callbacks endpoint /api/v1/calls/callbacks is optional unless you need detailed lifecycle events.)

If tunnel URL changes, update the subscription (delete & recreate or update endpoint).

Reference: Subscribing to events

  1. Dial the number; observe:
  2. Call connection established
  3. Media session events
  4. STT transcripts
  5. TTS audio frames

13. Quick Browser Test

  1. Backend + frontend running.
  2. Open app, allow microphone.
  3. Speak → expect:
  4. Interim/final transcripts
  5. Model response
  6. Audio playback

14. Troubleshooting

Symptom Likely Cause Fix
404 on callbacks Stale BASE_URL Restart tunnel, update .env
No audio Speech key/region invalid Verify Azure Speech resource
WS closes fast Wrong VITE_BACKEND_BASE_URL Use exact backend/tunnel URL
Slow first reply Cold pool warm-up Keep process running
Phone call no events ACS callback not updated to tunnel Reconfigure Event Grid subscription
Import errors Missing dependencies Re-run pip install -r requirements.txt

15. Testing Your Setup

Quick Unit Tests

Validate your local setup with the comprehensive test suite:

# Run core component tests
python -m pytest tests/test_acs_media_lifecycle.py -v

# Test event handling and WebSocket integration
python -m pytest tests/test_acs_events_handlers.py -v

# Validate DTMF processing (if using phone features)
python -m pytest tests/test_dtmf_validation.py -v

Load Testing (Advanced)

Test WebSocket performance with realistic conversation scenarios:

# Generate realistic audio for testing
make generate_audio

# Run WebSocket load test locally
locust -f tests/load/locustfile.py --web-host 127.0.0.1 --web-port 8089

What the load tests validate: - ✅ Real-time audio streaming - 20ms PCM chunks via WebSocket - ✅ Multi-turn conversations - Insurance inquiries and quick questions - ✅ Response timing - TTFB (Time-to-First-Byte) measurement - ✅ Barge-in handling - Response interruption simulation - ✅ Connection stability - Automatic WebSocket reconnection

Additional Resources

For more comprehensive guidance on development and operations:

Keep secrets out of commits. Rotate anything that has leaked.