Test Driving AIs...Still Learning! · AI Automation Society

Test Driving AIs...Still Learning!

I have found myself the past few weeks after learning the essentials trying to find the AI that is going to fit well for my startup. The dove deep into Claude the last month and I've just now put that subscription on pause while I try out ChatGPT. These are just the $20 subscription models. I'm not using the latest and greatest models yet until I find one that I actually like.

I have been impressed with ChatGPT Work so far. It is not as verbose as Claude, and has done requests in similar time and speed. I am mainly using the Sol model, but I just made a skill this morning to have it determine which model I should use based on the request.

I still want to try Grokbot at some point. I've been watching/listening to their Galaxy event this week. Right now I am focused on getting my business officially opened the first week of October, so I am sticking with ChatGPT until then. I'll reevaluate at the middle of next month to see if I need to go back to Claude Code.

BTW.. I tried Vellum as well the past three weeks as I was looking for an alternative to Grok. It has some pros, but also has some cons. It handled personal assistant type stuff very well, but it's limited scope on plugins files created a mess with GitHub and my website recently. I have since moved everything to ChatGPT. Also, with the release of Muse, that might be the personal assistant to go with now.

I hope everyone is doing well and are having some prosperous opportunities in your life.

BTW - This is the skill I created in ChatGPT

Created and validated the Efficient Street Model Router skill.

Going forward, as chief of staff I will:

Assess substantive Efficient Street requests for complexity, risk, ambiguity, and context requirements.
Use Luna for mechanical work, Terra for routine execution, and Sol for consequential strategy or difficult synthesis.
Delegate only when the likely savings or quality improvement exceeds the coordination overhead.
Tell you when I actually use another agent or a meaningfully more expensive model.
Monitor conversation length and recommend consolidation, compression, hindsight, or a fresh chat at a natural stopping point.
Preserve decisions in authoritative project files before recommending a new chat.

One limitation remains: I cannot silently replace the model powering the current conversation. I can delegate bounded work to an appropriately selected model and synthesize its results here, or recommend that you manually change the main model when that would be more efficient.

The skill passed validation and was committed to version control.

4 comments

Test Driving AIs...Still Learning!