How do you choose your AI Stack? (LLM, DB, and Memory)

With so many options available in the market, curious to know how you all decide on the best "trio" for your AI projects. When building a chat-based application or an automation (like in n8n), how do you evaluate these three critical components?

1. Choosing the LLM Model & Provider

Beyond just the cost, what factors make you choose one provider (Open AI, Anthropic, Google, or open-source) over another?

Do you prioritize latency and speed for real-time chat?
How much does the context window size influence your decision?
Do you look for specific fine-tuning or domain-specific capabilities?

2. Deciding on the Database (DB)

In tools like n8n, there are many DB options (PostgreSQL, Supabase, Pinecone, Redis, etc.). Besides cost, what do you look for before choosing?

Vector Search vs. Relational: When do you opt for a dedicated vector store versus a traditional SQL DB with vector support?
Scalability: How do you assess if a DB can handle your project's future growth?
Ease of Integration: Does the available n8n node support everything you need ?

3. Selecting Memory Options

Managing conversation history is tricky. How do you decide which memory strategy to use?

Buffer Window vs. Summary: Do you prefer keeping the last 'X' messages or using an LLM to summarize the history?
Persistent vs. Ephemeral: When is it worth using a database for long-term "brain" memory versus just keeping it in the session?

About the "rules of thumb" or frameworks you use to make these choices. Looking forward to your insights!

1 comment