TLDR Tech / AI
Why do OpenAI's GPT-2 weights beat mine?
This researcher trained their own LLM from scratch but discovered that their models were worse at instruction-following than the original OpenAI GPT-2 small weights, even when their model got better results in a more technical evaluation. This post is the first in a series where the researcher looks at the nature of the problem and explores options for what might be the cause.
Read the full story