Scalable voice agent design with Amazon Nova Sonic: multi-agent, instruments, and session segmentation

design patterns for scalable voice brokers matter for organizations that must ship quick, pure, and dependable voice experiences. Many groups face challenges like excessive latency, managing real-time audio, and coordinating a number of brokers in complicated workflows.

On this put up, you’ll discover ways to use Amazon Nova Sonic, Amazon Bedrock AgentCore, and Strands BidiAgent to construct scalable, maintainable voice brokers that deal with these challenges effectively, leading to extra responsive and clever buyer interactions.

We’ll discover three in style architectural patterns for voice brokers, highlighting their trade-offs and finest practices for minimizing latency.

The constructing blocks

Earlier than diving deeper into the structure patterns, right here’s a fast overview of the three key parts used because the pattern answer on this put up.

Amazon Nova Sonic is a basis mannequin that creates pure, human-like speech-to-speech conversations for generative AI functions. Customers can work together with AI via voice in actual time, with capabilities for understanding tone, pure conversational movement, and performing actions.

Amazon Bedrock AgentCore Runtime is a serverless internet hosting atmosphere for AI brokers. You bundle your agent as a container, deploy to AgentCore Runtime, and it handles scaling, session isolation, and billing. For voice brokers, it supplies bidirectional WebSocket streaming with SigV4 auth, microVM-level session isolation to keep away from noisy-neighbor latency spikes, AgentCore Gateway for shared instrument internet hosting utilizing the Mannequin Context Protocol (MCP) open supply protocol, persistent reminiscence throughout periods, and telemetry for voice-specific metrics like time-to-first-audio.

Strands Agents is an open supply framework for constructing AI brokers. Its BidiAgent class is one integration possibility between Nova Sonic and your utility. It manages the bidirectional stream lifecycle, routes instrument calls, and handles session administration, simplifying the voice agent utility via the mannequin SDK interface.

Three integration patterns: instrument, agent-as-tool (sub-agent), and session segmentation

As an alternative of constructing one omnipotent agent, fashionable voice programs are more and more composed of tool-driven brokers, sub-agents appearing as instruments and session segmentation methods that isolate prompts, reminiscence, and permissions. These patterns enable groups to decompose giant assistants into smaller, specialised, and reusable parts whereas sustaining clear safety boundaries.

Earlier than operating the samples within the following sections, set up Python and the required dependencies, together with strands-agents and boto3, and ensure your IAM setup has the mandatory permissions for the required providers. For the total instance, seek advice from the GitHub repository.

Sample 1: AgentCore Gateway – instrument choice for low latency

A instrument name is when a voice agent sends enter to an exterior operate or service, which processes it and returns output. It lets the agent carry out duties like querying a database or triggering a service shortly and securely, with out additional reasoning steps.

With AgentCore Gateway, you expose your current enterprise logic as instruments, discrete capabilities that Nova Sonic can name straight throughout a dialog. The voice mannequin selects which instrument to invoke, passes parameters, will get a outcome, and speaks it again. There’s no intermediate reasoning layer between the mannequin and the instrument.

AgentCore Gateway hosts MCP servers as managed endpoints. MCP is the protocol, AgentCore Gateway is the AWS function that runs them. The voice agent connects by way of Gateway ARNs.

# Nova Sonic calls instruments straight by way of AgentCore Gateway
mannequin = BidiNovaSonicModel(
    model_id="amazon.nova-2-sonic-v1:0",
    mcp_gateway_arn=[
        "arn:aws:bedrock-agentcore:us-east-1:123456789012:gateway/auth-tools",
        "arn:aws:bedrock-agentcore:us-east-1:123456789012:gateway/banking-tools",
        "arn:aws:bedrock-agentcore:us-east-1:123456789012:gateway/mortgage-tools",
    ],
)

When a consumer says “What’s my account steadiness?”, Nova Sonic:

1. Understands the intent from speech.
2. Selects get_account_balance from the obtainable MCP instruments.
3. Calls the instrument with the fitting parameters.
4. Speaks the outcome again.

Commerce-off: Nova Sonic makes all the choices. If a instrument name requires multi-step validation, conditional logic, or chaining a number of operations collectively, that reasoning burden falls fully on the voice mannequin’s system immediate. For easy instruments that is wonderful. For complicated workflows, it will get brittle.

Sample 2: Sub-agent – extra reasoning with decoupled brokers

With the sub-agent or agent-as-tool sample, your current enterprise logic runs in autonomous brokers, every with its personal mannequin, system immediate, instruments, and reasoning capabilities. The voice orchestrator delegates complete duties to those sub-agents as an alternative of calling particular person instruments.

There are various methods to connect with a sub-agent out of your voice agent. Agent-to-Agent (A2A) and Strands Agent-as-Instrument are two widespread approaches:

• Native agent-as-tool: The sub-agent runs in-process, wrapped as a @instrument operate utilizing the Agents as Tools pattern in Strands. That is probably the most easy strategy with no community hop and no separate deployment. The trade-off is that the sub-agent shares the identical course of and scales with the orchestrator.
• Distant agent by way of A2A protocol: The sub-agent is deployed as an unbiased A2A server on AgentCore Runtime (or a distant server) and invoked over the community. A2A is an open protocol for agent-to-agent communication. As MCP connects brokers to instruments, A2A connects brokers to different brokers. Because the AWS weblog on A2A protocol support in AgentCore Runtime explains, brokers constructed with totally different frameworks (Strands, OpenAI, LangGraph, Google ADK) can share context and reasoning in a typical format. This supplies full deployment independence and cross-framework interoperability.

Strands Brokers has built-in help for each protocols, MCP for instrument entry and A2A for agent-to-agent communication. For a hands-on walkthrough, see the group information on Agent Collaboration: Strands Agents, MCP, and the Agent2Agent Protocol.

Right here’s the native agent-as-tool strategy, every sub-agent is a @instrument wrapping a full Strands Agent:

# sub_agents.py — Outline sub-agents as Strands instruments utilizing the Brokers-as-Instruments sample
from strands import Agent, instrument
from strands.fashions import BedrockModel

# Every sub-agent is a full Strands Agent wrapped as a @instrument
# The BidiAgent orchestrator calls these by way of Nova Sonic's instrument use

@instrument
def authenticate_customer(account_id: str, date_of_birth: str) -> str:
    """Authenticate a buyer utilizing their account ID and date of start.
    Handles the total verification movement together with identification checks and retry logic.
    Returns authentication standing and token."""
    auth_agent = Agent(
        mannequin=BedrockModel(model_id="amazon.nova-lite-v1:0"),
        system_prompt="""You might be an authentication agent. Confirm the client's identification
        utilizing the offered account ID and date of start. Name verify_identity to test
        credentials. Return a transparent auth standing in 1-2 sentences.""",
        instruments=[verify_identity, check_account_exists],  # Sub-agent's personal instruments
    )
    outcome = auth_agent(f"Authenticate account {account_id}, DOB: {date_of_birth}")
    return str(outcome)


@instrument
def handle_banking_inquiry(question: str, auth_token: str) -> str:
    """Deal with banking questions — balances, transactions, transfers.
    Validates permissions and returns a conversational abstract."""
    banking_agent = Agent(
        mannequin=BedrockModel(model_id="amazon.nova-lite-v1:0"),
        system_prompt="""You're a banking agent. Use the offered instruments to reply
        the client's question. Summarize leads to 2-3 pure sentences.
        Don't return uncooked JSON.""",
        instruments=[get_account_balance, get_recent_transactions, transfer_funds],
    )
    outcome = banking_agent(question)
    return str(outcome)


@instrument
def handle_mortgage_inquiry(question: str) -> str:
    """Deal with mortgage questions — charges, calculations, eligibility, utility standing.
    Performs its personal calculations and reasoning."""
    mortgage_agent = Agent(
        mannequin=BedrockModel(model_id="amazon.nova-lite-v1:0"),
        system_prompt="""You're a mortgage specialist. Assist with charge inquiries,
        fee calculations, and eligibility assessments. Hold responses concise
        and conversational — this might be spoken aloud.""",
        instruments=[get_mortgage_rates, calculate_payment, check_eligibility],
    )
    outcome = mortgage_agent(question)
    return str(outcome)

The voice orchestrator then makes use of BidiAgent with these sub-agent instruments:

# voice_orchestrator.py — BidiAgent with sub-agents as instruments
from strands.experimental.bidi.agent import BidiAgent
from strands.experimental.bidi.fashions.nova_sonic import BidiNovaSonicModel
from sub_agents import authenticate_customer, handle_banking_inquiry, handle_mortgage_inquiry

mannequin = BidiNovaSonicModel(
    area="us-east-1",
    model_id="amazon.nova-2-sonic-v1:0",
    provider_config={"audio": {"voice": "tiffany", "input_sample_rate": 16000, "output_sample_rate": 16000}},
)

agent = BidiAgent(
    mannequin=mannequin,
    instruments=[authenticate_customer, handle_banking_inquiry, handle_mortgage_inquiry],
    system_prompt="""You're a banking voice assistant. Route buyer requests to the
    acceptable specialist. All the time authenticate earlier than accessing account information.
    Hold your individual responses temporary — the sub-agents deal with the main points.""",
)

await agent.run(inputs=[ws_input], outputs=[ws_output])

The sub-agent does its personal considering. Nova Sonic doesn’t must orchestrate the person steps. It delegates and speaks the outcome.

Commerce-off: Every sub-agent name provides latency: the sub-agent’s personal mannequin inference plus its instrument calls. In a voice dialog, this implies longer silence whereas the sub-agent causes. The AWS weblog on multi-agent voice assistants recommends beginning with smaller, environment friendly fashions like Amazon Nova 2 Lite for sub-agents to cut back latency whereas nonetheless dealing with specialised duties successfully.

Amazon Nova 2 Sonic helps asynchronous tool calling, so the dialog continues naturally whereas instruments run within the background. It retains accepting enter, can run a number of instruments in parallel, and gracefully adapts if the consumer adjustments their request mid-process, delivering all outcomes whereas specializing in what’s nonetheless related.

Sample 3: Session segmentation for ultra-low latency

There’s a 3rd strategy value contemplating. It doesn’t map neatly to the MCP or sub-agent patterns, however is purpose-built for voice eventualities the place latency is the overriding concern.

As an alternative of delegating exterior instruments or sub-agents, you phase the dialog into logical phases, every with its personal Nova Sonic session, system immediate, and power set. When the dialog transitions from one section to the following (for instance, from authentication to account inquiry), you shut the present session and open a brand new one with a unique immediate and instruments, throughout the similar WebSocket connection. Every sub-voice-agent can use its personal MCP gateways, instruments, and even sub-agents — the variations that it operates with a targeted immediate and minimal instrument floor, decreasing reasoning overhead and latency.

Consider a banking voice assistant with three dialog phases: authentication, account administration, and mortgage inquiry. Relatively than loading one large system immediate with each instrument, you run every section as a targeted Nova Sonic session:

# Section 1: Authentication
auth_session = BidiNovaSonicModel(
    model_id="amazon.nova-2-sonic-v1:0",
    mcp_gateway_arn=["arn:...gateway/auth-tools"],  # Solely auth instruments
)
auth_agent = BidiAgent(
    mannequin=auth_session,
    instruments=[],
    system_prompt="""You might be an authentication assistant. 
    Accumulate the consumer's account ID and date of start. 
    Name verify_identity to authenticate. 
    As soon as verified, say 'You are all set' and cease.""",
)
# Run till authentication completes
await auth_agent.run(inputs=[ws_input], outputs=[ws_output])

# Section 2: Account administration (new session, new immediate, new instruments)
banking_session = BidiNovaSonicModel(
    model_id="amazon.nova-2-sonic-v1:0",
    mcp_gateway_arn=["arn:...gateway/banking-tools"],  # Solely banking instruments
)
banking_agent = BidiAgent(
    mannequin=banking_session,
    instruments=[],
    system_prompt="""You're a banking assistant. The consumer is already authenticated.
    Assist with steadiness inquiries, transactions, and transfers.
    Hold responses to at least one or two sentences.""",
)
await banking_agent.run(inputs=[ws_input], outputs=[ws_output])

Every section will get a clear Nova Sonic session with:

• A targeted system immediate: Shorter, extra particular, much less room for the mannequin to get confused.
• Solely the related instruments: by way of MCP gateways, native instruments, or each. The mannequin doesn’t waste reasoning cycles selecting between 15 instruments when it solely wants 3.
• Optionally its personal sub-agents: a section that requires deeper reasoning can use Sample 2 internally, whereas easier phases keep tool-only.
• The earlier session context might be handed into the brand new session as chat historical past, so the general dialog retains continuity.

In comparison with instrument, sub-agent, and session segmentation patterns

Latency finest practices for voice brokers

Latency is a key consideration when constructing voice versus textual content brokers. Listed here are sensible strategies to maintain response occasions quick and responsive:

Begin with small fashions for sub-agents. Your voice orchestrator makes use of Nova Sonic for the dialog, however sub-agents don’t want a big mannequin. Begin with Amazon Nova 2 Lite or Nova 2 Micro. They’re quick, value optimized, and deal with most specialised duties effectively. You may all the time improve a selected sub-agent to a bigger mannequin if high quality requires it, however default to small.

design stateful sub-agents with caching. A stateless sub-agent that hits a database or API on each name provides latency each time. As an alternative, design sub-agents to cache outcomes from information sources (APIs, AWS Lambda capabilities, databases) inside a session. If the banking sub-agent fetches account particulars as soon as, it ought to maintain that information in reminiscence and serve subsequent questions (steadiness, transactions, abstract) from cache moderately than making repeated backend calls.

Prefetch information after authentication. That is particularly priceless for contact middle eventualities. After a buyer authenticates, you already know who they’re. Don’t anticipate them to ask earlier than pulling their information. Instantly fetch account balances, current transactions, pending alerts, and mortgage standing within the background. When the client asks “What’s my steadiness?”, the reply is already in reminiscence.

Parallelize unbiased instrument calls. If the consumer asks “Give me an summary of my accounts”, don’t name get_checking_balance, then get_savings_balance, then get_credit_card_balance sequentially. Use concurrent execution so three calls occur without delay. Strands helps this natively. The agent’s instrument executor runs unbiased calls in parallel by default.

Use filler phrases to masks instrument latency. When a instrument name or sub-agent delegation is unavoidable, instruct the voice mannequin to talk a short filler whereas ready: “Let me test that for you…” or “One second whereas I look that up…” This retains the dialog feeling alive as an alternative of dropping into silence.

Reduce instrument rely per session. Instrument choice will get slower because the variety of obtainable instruments grows. In case your agent has 15 instruments however a typical dialog solely makes use of 3 to 4, take into account the session segmentation sample to load solely the related instruments per section.

Clear up

After you end testing the pattern, bear in mind to wash up the assets you created to keep away from pointless prices. Comply with the repository directions to cease providers and delete any deployed infrastructure.

Conclusion

Migrating a textual content chatbot to a voice assistant isn’t a simple wrapper job. The interplay mannequin is essentially totally different, from response design to latency budgets to turn-taking conduct. However with a well-structured multi-agent structure and Amazon Bedrock AgentCore, the enterprise logic layer stays intact.

The sub-agents you’ve already constructed are your largest asset. Reuse them.

For a working instance of a Strands BidiAgent voice assistant deployed on AgentCore Runtime with WebSocket streaming, see the AgentCore bidirectional streaming sample.

Subsequent steps

Subsequent, you’ll be able to prolong the pattern to suit your personal use case, combine your small business instruments, refine prompts for voice interactions, and take a look at the agent in real-world eventualities to organize for manufacturing deployment. To study extra about voice brokers on AWS, go to:

In regards to the authors