Personal Assistant for Teens: Two Weeks In, the Results!

Two weeks ago I shipped a personal assistant for my daughter Luna, 17, in a single day, and promised I'd come back here with results. (read original post). Here they are, with the lessons that came with them.

Last time I said: measure the before, so you can show the after. So here it is. Before, planning her day took about half an hour, and the worst part was carrying it all in her head, always half-worried she'd forgotten something. Two weeks in, the half hour is basically gone, and what she told me herself is that the worry went with it. She got through a trip without it hanging over her. The minutes were never the point. The relief was.

But asking the user about the system is not enough. So I ran an audit. I found out there was a learning loop that was writting to a file nobody was re-reading when needed. The assistant was built to turn every correction Luna makes into a written rule, in the file that owns it. It did exactly that. But those corrections landed in a file the morning routine never re-opened, so the same correction came back more than once in a single day. The fix wasn't a smarter model or a better prompt. It was one line: put the corrections file inside the load path of the routine it's meant to fix. Fix the context, not the agent.

Audit the system you shipped, don't admire it. Two weeks of real use is a first, good and honest test, so I ran an audit: her before and after side by side, in her own words, and I walked the delivered files cold, with her permission, since the workspace is hers. Watching it work told me it was great. Walking the files told me where it was still broken. Both were true, and only the audit showed me the second one.

Ship it, let someone live in it, then go back and measure what actually changed!

BTW: Luna's assistant is called Janet.

0 comments

Personal Assistant for Teens: Two Weeks In, the Results!