Finding relevant question on social media around the topic of evaluations so that I can answer them.

@Ashish Ranjan Have you tried Claude's new co-work feature?

I have translated the first exercise into typescript. Here is the blog post for the first exercise. https://learninglog.svbtle.com/eval @Vivian Aranha More than happy to publish these as posts here when I get to Level 3, just don't have the ability to post yet.

Got exercise 2 out. This one covers aligning your Ilm judges in Typescript. https://learninglog.svbtle.com/trust

I always think “aliens”. It’s basically like having a new form of intelligence to interact with.

When you're putting together an evaluation suite for an AI product, what pass rate do you look for? What is an acceptable threshold to establish?

@Juwaria Qadri I guess it depends what languages you know and what you're trying to build. Assuming you we're considerng most popular modern programming language and a general purpose hackathon, then I would go with Typescript so that you don't have to write different languages on the front and backend (which will save you a bunch of time with tooling at a hackathon). If you're writing your code in JS/TS then Vercel's AI-SDK is your one-stop shop for building with AI.

Complete liability in my opinion. "Getting lucky" isn't something we want to be shipping to production on.