Single agent (inbound + outbound in one prompt) vs. two specialized agents — what's the right call?

Building a B2B outbound voice agent on Retell (n8n picks who gets called and when, hands the context to the agent, Retell executes the call). Works well for outbound — the agent always knows exactly who it's calling and why, so the prompt is fully tailored to that one scenario.

Now I want to support inbound too — someone sees a missed call and calls back, or hears about us and calls in cold. Problem: my current prompt assumes outbound context everywhere. It opens with "Is this [name]?" and assumes it already knows her city — none of that holds for an inbound call from an unknown number.

I see two ways to solve this:

One agent, bigger prompt — add a full inbound branch: "if this is inbound with no known context, do X instead." One agent, one number to manage, but the prompt grows a lot, and now the model has to figure out which branch applies on every single call, on top of holding both sets of instructions.
Two specialized agents — a lean outbound-only agent and a lean inbound-only agent, each with a fully dedicated single-purpose prompt, routed by call direction at the phone number level (inbound_agent vs outbound_agent on Retell).

My honest concern with option 1: does a longer, branchy prompt with in-call conditional logic measurably hurt response quality or latency, or meaningfully raise cost, compared to two lean single-purpose prompts? Or am I overthinking it and modern models handle this fine?

Curious how people running production voice agents (Retell or similar) handle this — one flexible agent, or multiple specialized ones per direction/use case? Real experience > theory here, especially anything on cost or quality tradeoffs you've actually measured.

2 comments

Single agent (inbound + outbound in one prompt) vs. two specialized agents — what's the right call?