Voice Agents with Large Prompts

Bora Bekar

Aug '25 • General discussion

Complex agents can increase costs due to long prompts.

So here I will briefly explain how you can do fine-tuning in the simplest way possible.

This will allow you to reduce costs and increase the intelligence of your voice agent threefold.

(If you need any help, just mention me in the comments and I'll try to help you)

1- Prepare your detailed prompt.

2- Send it to Gemini and ask it to optimise this document for fine-tuning. Request that it optimise for maximum agent intelligence performance using metadata, detailed triggers, if conditions, and structural adjustments.

3- Export the document in any format. txt, rtf, etc.

4- Go to Cursor (you need to download it to your computer) and upload the document to the left panel. In the chat section, mention the document using ‘@’ and ask it to generate the document in jsonl format. Specify that no changes should be made to the main content.

5- Save the JSONL output to your desktop and go to this link: https://platform.openai.com/finetune/ftjob-sdyqx7k3DmRpf0pQa85PYOKg

6- Upload the jsonl document to the field that appears on the screen. Set a fixed seed, such as ‘53’. Enter your name in the suffix field for the model. It can be anything. Select GPT 4.1 as the base model.

7- In the Configure hyperparameters section, open ‘Number of epochs’ and set it to 3. Then press the Create button.

8- Once the process is complete, go to Postman from the web.

9- Enter the following formats and click the Send button.

POST: https://api.vapi.ai/assistant

Headers:

Key: Authorization

Value: Bearer [YOUR_VAPI_PRIVATE_KEY]

Body: {

"name": "[YOUR_NEW_VAPI_AGENT_NAME]",

"model": {

"provider": "custom-llm",

"url": "https://api.openai.com/v1/chat/completions",