New challenge; how to cluster 1000+ records

Following my challenge of semantically grouping 1000+ records I now have a follow-up challenge where I need to cluster a sub-set of the semantic relevance output. As before my current scenario appears limited to processing a few hundred records. (In the video, I show how it processed 238 just 7 days ago and made me think it's done and dusted).

But I just tried using the same scenario to process 1300 records and the agent didn't even try; it processed 0 records (effectively giving up before it even began).

So this time I want to be proactive and not get myself in the mess that caused me to consume 17000 credits in less than an hour!

Thanks to

@Aashiya Bi

for helping me come up with a 2-part scenario I think will work for large-scale semantic relevance processing.

But how do I do something similar for this follow-up scenario. What do I need to change to my current scenario or LLM so that the scenario can execute. Any ideas up front will be most welcome.

cc.

@Mohamed Jahar

5 comments

New challenge; how to cluster 1000+ records