While building my AI Support Agent, I reached another important milestone: I can now track the token usage and estimated cost of individual LLM interactions. Previously, my focus was mainly: Question → RAG → LLM → Answer Now I am looking at the other side of the equation: Question → Tokens → Latency → Cost → Business Decision For each relevant LLM request, I’m capturing: 🔹 Input tokens 🔹 Output tokens 🔹 Total tokens 🔹 Request ID 🔹 Latency 🔹 Timestamp 🔹 Estimated input cost 🔹 Estimated output cost 🔹 Total estimated cost 🎯 The interesting part: cost-aware routing Not every customer interaction should necessarily result in an LLM call. For example: Customer Query → Intent Router ➡️ If the query can be handled through the appropriate AI/RAG path → process it and track the LLM usage/cost. ➡️ If the intent requires human support / handoff → avoid unnecessary LLM processing and therefore avoid unnecessary model cost. This made me think about AI from a different perspective. AI engineering is not only about generating a good answer. It is also about understanding: When should I call the model? Why am I calling the model? How many tokens am I consuming? What does that request cost? Can I avoid an unnecessary call? 💡 FDE takeaway This is where technical implementation connects with business impact. If an AI application handles 10 conversations, cost may not be a major concern. But what happens with: 10 → 1,000 → 10,000 → 100,000 conversations? Small inefficiencies in token usage and unnecessary LLM calls can become significant operational costs. So I'm gradually moving my AI Support Agent from: “It works” to: “It works, I can evaluate it, I can observe it, and I can understand what it costs.” That's another step toward thinking about AI solutions from an FDE + production engineering perspective. 🚀 #FDE#AIEngineering#AIAgents#LLM#GenerativeAI#RAG#TokenUsage#LLMCost#AIObservability#CostOptimization#AIArchitecture#ProductionAI#CustomerExperience#BusinessImpact#ContinuousLearning

Most engineers protect their indispensability. Be the only one who understands the system. Be the one call that has to happen before anything ships. Feels safe. Feels like job security. For an FDE, it's actually a ceiling. 𝐇𝐞𝐫𝐞'𝐬 𝐭𝐡𝐞 𝐭𝐫𝐚𝐩: You build something great. The client loves it. You become the single point of contact for everything — every question, every tweak, every "quick thing." Feels like leverage. It's actually a bottleneck. And bottlenecks don't scale. 𝐖𝐡𝐚𝐭 𝐡𝐚𝐩𝐩𝐞𝐧𝐬 𝐢𝐟 𝐲𝐨𝐮 𝐬𝐭𝐚𝐲 𝐭𝐡𝐞 𝐛𝐨𝐭𝐭𝐥𝐞𝐧𝐞𝐜𝐤 → You can't take a second account. You're the only one who can touch the first one. → The client quietly resents the dependency. Nobody wants to need one person forever. → You become a support ticket, not a strategic partner. Every conversation is "can you fix this," not "what should we build next." → Your growth caps at one account. Forever. 𝐓𝐡𝐞 𝐩𝐚𝐫𝐚𝐝𝐨𝐱 The FDEs who get to Lead / Staff, who run multiple accounts, who get the comp bump — they're not the ones holding on tightest. They're the ones who design their own exit into every engagement from day one. Not because they're leaving. Because they're building something that outlives their daily presence — which is the only way to prove you're ready for more scope, not less. 𝐖𝐡𝐚𝐭 𝐭𝐡𝐢𝐬 𝐚𝐜𝐭𝐮𝐚𝐥𝐥𝐲 𝐥𝐨𝐨𝐤𝐬 𝐥𝐢𝐤𝐞 - Documentation isn't an afterthought — it's shipped alongside the feature, not requested after you're already three accounts deep into burnout. - You train an internal owner — someone on the client side who can run the system without you in the room. Early. Not at offboarding. - You shift from builder to escalation point — day-to-day questions go to the internal owner. You show up for the hard stuff. That's a promotion in disguise, even if nobody calls it one. - You measure success by what runs without you — not by how many Slack messages have your name on them. 𝐓𝐡𝐞 𝐦𝐢𝐧𝐝𝐬𝐞𝐭 𝐬𝐡𝐢𝐟𝐭 Junior FDEs think: "If they don't need me, I lose value." Senior FDEs think: "If they can't run this without me, I can't take on the next account — and neither can the company scale past me."

URL: https://www.linkedin.com/posts/pkl87_fde-scale-automated-share-7495076299967774720-cu_Q/?utm_source=share&utm_medium=member_desktop&rcm=ACoAACr01jQByvOc24VPQ3bbS2VmVTh5YSyUcmo While building my AI Agent, I initially tested it the traditional way: Ask a question → Check the response → Repeat → Decide whether it worked. That approach is fine for a quick demo. But once an AI agent starts evolving, manual testing doesn't hashtag#scale. So I implemented an hashtag#automated testing/hashtag#evaluation approach. Instead of manually asking questions every time, I created a set of test questions with their expected outcomes and allowed the application to execute and evaluate them automatically. What does the flow look like? Test Dataset → Agent → Response → Evaluation → PASS / FAIL The test cases cover different scenarios, including: ✅ Questions the agent should answer using RAG ✅ Questions that require the correct intent routing ✅ Questions where the expected result is Handoff ✅ Questions that should not be answered from unsupported knowledge ✅ Retrieval/evaluation scenarios This changed my perspective. The hashtag#FDE angle 🎯 For an hashtag#FDE, testing isn't just about: “Does the demo work?” It's about: “Can I repeatedly prove that the solution is working as expected?” When we change the prompt, retrieval logic, chunking strategy, model, or guardrails, automated evaluation gives us a repeatable way to understand whether the change improved or degraded the solution.

In my previous post, I shared a real lesson from building my AI Agent: Functional correctness ≠ Production readiness. The agent was working correctly — RAG, intent routing, evaluations, guardrails and UI were all functioning. Then I hit: 429 — RESOURCE_EXHAUSTED The API quota had been exceeded. That created a new engineering question: What should an AI application do when the model/API temporarily refuses a request? The first solution I implemented For my current evaluation/testing scenario, I introduced a controlled delay using Python's: time.sleep() Instead of continuously firing requests, the test execution pauses between calls. Conceptually: Request → Wait → Request → Wait → Request This helped me avoid sending requests too aggressively during automated evaluation. And it solved the immediate problem in my development/testing environment. #But the bigger #FDE lesson was not time.sleep(). It was understanding rate limiting and resilience. An AI application needs to consider: 🔹 API quotas 🔹 Request frequency 🔹 Retry behavior 🔹 Exponential backoff 🔹 Concurrency 🔹 Caching 🔹 Token consumption 🔹 Monitoring & observability 🔹 Graceful failure / fallback Because at scale, this becomes both a technical problem and a business problem. 💰 Think about it from a customer perspective Imagine an application receiving: 10 requests → Fine 100 requests → More API calls 1,000 requests → Higher token/API consumption 10,000 requests → Quota, latency, concurrency and cost become serious considerations So the FDE question isn't simply: “Can the AI answer the question?” It's: “Can the AI solution continue to provide a reliable customer experience when usage increases or external services become constrained?” That's a completely different level of thinking. My current learning

Your agent can be: ✅ Correct ✅ RAG working ✅ Intent routing working ✅ UI working and still have: ❌ Rate-limit problem ❌ No retry strategy ❌ No caching ❌ No monitoring ❌ No concurrency strategy