Seeing like an algorithm
Apologies to James C. Scott
This isn’t about generative AI, but I wrote about loops yesterday and wanted to continue the theme today.
ML modes get better with training. This happens four ways:
A model can get better at understanding a particular user
A model can get better at understanding a particular piece of content
A particular model can get better based on general user inputs
A particular model can get better based on general content
GenAI is primarily trained on general content, with the size of corpus, number of features, and training time driving costs. But many of the most popular ranking models rely on real-time user inputs to deliver better user experiences. TikTok is the canonical example, where the user interface is designed to capture feedback which is then fed into the ranking algorithm in real time.
Likes, follows, messages, shares dominate the screen, all of which tell the algorithm how much the user likes this particular video, and also how promising this video is, which then gets fed into two different recommendation decisions: 1) what video to show this user next, and 2) whether to show this video to more people.
The most critical bit of user input is negative signal — does the user just skip to the next video quickly or not. Negative signal can often be more useful than positive.
The YouTube interface is similar, but handles negative signal in a slightly different way:
You can see the main video, with the basic input signals (thumbs up/down, subscribe share etc.) at the bottom. On the left is recommended content, which is all related. However, if the user skips to the end of the video, or scrolls far down the recommended list, the algorithm will interpret this as negative signal and suggestions will move to other topics. If the user watches until the end, it will stick with the topic, and assume the user wants to dive deeper.
GenAI workflows, such as vibecoding, have lots of negative signal, but it’s captured in clarification prompts, trouble shooting questions, switching to different models, etc. I don’t think this information can be fed into the model in realtime, but it’s possibly a source of valuable signal to see how well the model is doing, and where it can be improved.