Andrej Karpathy is one of the original co-founders of OpenAI.
And he has been one of my trusted sources for years now.
Andrej just published a 3h video explaining how ChatGPT is build and how it works.
Video is made for general public, it starts with explaining how data was collected, how raw data gets turned into tokens and what tokens are..
Wonโt spoil it to you, but if you will watch this do share what has been a new information to you.
P.S. in the beginning Andrej shows turning tokens into emojis which has been a big prompt hacking technique for a while to use emojis as secret tokens to get LLMs say what they should not.