I don't have any question here but want to share, because it feels like a huge win for me but nobody in my life understands - the other day, I made two huge milestones.
- After researching and learning about RAG methodologies and strategies for weeks, I finally got a RAG ingestion workflow working that seems like it's going to work for my use-case and yield really good, accurate results. I have a lot to add to it yet to further automate, but determining how to chunk it out and what tools to use, and actually understanding a bit of the "why" behind all of these choices, was a milestone for me. I haven't officially starting my testing yet but already it's accuracy is looking SO much better. (For the curious - I'm manually breaking up the chunks semantically, I'm using an LLM to enrich each chunk with metadata/tags/chunk summaries/example questions, and I switched to openai embeddings which apparently are better with semantic search). I also setup the metadata in a way that will make it easier to translate into a postgresql table if I decide I need hybrid search in the future to further increase accuracy, which learning about postgres in and of itself was a large endeavor.
2. I got one of my agent workflows setup with hybrid text/image RAG! I had to tag my RAG docs to look for the images and make the metadata flag it, then I had to set up a firebase storage, and I'm just so damn pleased that it all actually worked. When my chat test came back with the images along with the answer, I straight up giggled.
For some people here, these might seem like pedestrian or super easy, afternoon type projects. For other people here, this might seem almost like another language still. I hope by sharing that I inspire someone else to keep going! I love reading posts like these myself (thanks for always posting !) so I thought I'd try and share too.