Lots of orgs are getting excited about LLM visibility and getting tools to help them monitor how they appear for various prompts.
And, because everyone and their brother is interested in LLMs, i see way more people, from far wider across the organization, wanting access to these tools than anything we've brought online before. And for each person that wants access, 5 people are asking questions they have decided can be answered by the new tool.
And some orgs are doing a really bad job of controlling who gets access at what permission level. Not to mention a bad job of prioritizing ad hoc asks.
For most orgs the two following steps are working at least for day to day:
- I've suggested people attend 20 minutes of training or watch a 20 minute video before they get access.
- Ive drafted a tiny policy about what the tool is for (monitoring progress of llm visibility initiatives, not idle curiosity) and how to tag prompts you add/check before removing prompts.
But ive got an org where everyone is too busy for the training and not sure if they really need a policy.
Anyone have tips or recommended reading on how to sell people in this kind of situation on the need for governance?