Blog Details

Building an AI voice agent on top of physical telephony hardware (as opposed to a cloud telephony API) requires a clear separation of concerns between four layers: the phone's call-state service, the ADB control plane, the audio bridge, and the workflow engine that actually runs your business logic. This article lays out a reference architecture for that stack, based on patterns that hold up in production rather than in a demo.

Why a Layered Architecture Matters Here

The naive version of this system is a single script that polls call state, answers calls, and pipes audio — and it works right up until you need to handle a dropped Bluetooth connection mid-call, a phone reboot, or adding a second phone line. Separating concerns early avoids a rewrite later:

┌─────────────────────────────────────────────────────┐
│                  Workflow Engine                     │
│   (business logic, call routing, session state,      │
│    integration with CRM/calendar/ticketing systems)  │
└───────────────────────┬───────────────────────────────┘
                         │ events + commands
┌───────────────────────▼───────────────────────────────┐
│                  Voice Pipeline Layer                  │
│        (STT → LLM → TTS, turn-taking logic)            │
└───────────────────────┬───────────────────────────────┘
                         │ audio streams
┌───────────────────────▼───────────────────────────────┐
│                   Audio Bridge Layer                   │
│         (Bluetooth HFP source/sink management)          │
└───────────────────────┬───────────────────────────────┘
                         │ call state events
┌───────────────────────▼───────────────────────────────┐
│               Phone Control Plane (ADB)                │
│    (call state polling/streaming, answer/hangup cmds)   │
└───────────────────────┬───────────────────────────────┘
                         │
                  [Android phone + SIM]

Layer 1: Phone Control Plane

This layer's only job is to expose call state as events and accept commands (answer, hang up, mute). Two implementation choices:

Polling via adb shell dumpsys — simplest to build, adequate for prototypes, but adds 200-500ms of detection latency depending on poll interval and creates unnecessary ADB traffic.

Companion Android app with PhoneStateListener — a small installed APK that listens for telephony state changes natively and pushes them to a local TCP/Unix socket or broadcasts them via ADB-triggered intents. This is the right choice for production: near-instant event detection, lower resource usage, and a natural place to add call metadata (caller ID, call direction) that dumpsys output doesn't cleanly expose.

// Minimal companion service sketch
public class CallStateService extends Service {
    private final PhoneStateListener listener = new PhoneStateListener() {
        @Override
        public void onCallStateChanged(int state, String phoneNumber) {
            switch (state) {
                case TelephonyManager.CALL_STATE_RINGING:
                    emitEvent("RINGING", phoneNumber);
                    break;
                case TelephonyManager.CALL_STATE_OFFHOOK:
                    emitEvent("OFFHOOK", phoneNumber);
                    break;
                case TelephonyManager.CALL_STATE_IDLE:
                    emitEvent("IDLE", phoneNumber);
                    break;
            }
        }
    };
}

Design the event contract as a small, versioned JSON schema ({"event": "RINGING", "caller": "+1...", "ts": ...}) from day one — it's the seam between this layer and everything above it, and changing it later means touching every downstream consumer.

Layer 2: Audio Bridge

This layer owns the Bluetooth HFP connection lifecycle: pairing state, profile negotiation (HFP vs. A2DP), and exposing stable audio source/sink handles to the pipeline layer above. Key responsibilities:

• Detect and force the HFP profile (many OSes default to A2DP-only, which has no microphone path)
• Monitor connection health and reconnect automatically on drop
• Expose a stable device name/handle so the pipeline layer doesn't need to know Bluetooth internals

# Health check example (systemd timer or supervisor loop)
if ! pactl list cards short | grep -q "hfp_head_unit"; then
    log "HFP profile not active, attempting recovery"
    pactl set-card-profile "$PHONE_CARD" handsfree_head_unit
fi

Treat this as a supervised process with its own restart/backoff policy, independent of the call-state layer — a Bluetooth drop is a different failure class than a dropped cellular call, and conflating their recovery logic makes both harder to debug.

Layer 3: Voice Pipeline

This is the STT → LLM → TTS chain covered in depth in our companion articles on latency measurement and optimization. Architecturally, it should:

• Consume audio from the bridge layer's exposed handle, agnostic to whether that's HFP, a SIP trunk, or a WAV file (useful for offline testing)
• Emit turn-level events (user_started_speaking, agent_response_ready) that the workflow engine can subscribe to
• Never directly touch ADB or Bluetooth APIs — if this layer needs to know it's running over a phone call versus a test harness, that's a sign the abstraction boundary has leaked

Layer 4: Workflow Engine

This is where actual business logic lives: routing calls based on caller ID or IVR-style menu selection, calling out to a CRM or ticketing API, deciding when to hand off to a human, and logging call outcomes. Keep this layer stateless with respect to audio/telephony details — it should operate purely on the event stream (transcript, intent, tool results) coming from the pipeline layer.

# Workflow engine pseudocode
def handle_turn(event):
    if event.type == "call_started":
        session = create_session(event.caller_id)
        return greet(session)
    if event.type == "user_utterance":
        intent = classify(event.transcript)
        if intent == "check_order_status":
            result = crm_api.lookup_order(session.caller_id)
            return respond_with_order_status(result)
        if intent == "request_human":
            return transfer_to_human(session)

Failure Isolation Across Layers

The reason this layering pays off is failure isolation:

Failure Contained to Recovery ADB connection drops Control plane Reconnect ADB, resume polling/socket Bluetooth HFP disconnects mid-call Audio bridge Reconnect, force HFP profile, alert if call still active STT provider outage Pipeline layer Fall back to secondary STT provider or degrade to DTMF menu Workflow logic bug/exception Workflow engine Catch, apologize gracefully via TTS, log for review, don't crash the call

Without these boundaries, a bug in your CRM integration can crash the process handling ADB polling, dropping every active call — a failure mode that's obvious in hindsight but easy to introduce when everything lives in one script.

Deployment Notes

• Run each layer as a separately supervised process (systemd units or a process manager like supervisord), communicating over local sockets or a lightweight message bus.
• Keep a health-check endpoint per layer so monitoring can distinguish "phone unreachable" from "LLM provider slow" from "workflow engine crashed."
• For multi-line deployments, run one instance of layers 1-2 per physical phone, but layers 3-4 can be shared services handling multiple concurrent call sessions.