TLDR Tech / Dev
I wrote a 70x faster SQL parser while barely looking at the code
A new SQL parser for PostHog was developed using AI, with a huge speed improvement compared to the previous C++ based ANTLR parser. The process involved using property-based testing for generating and validating test cases, alongside prompt engineering to guide the AI in implementing the parser. The resulting parser is a recursive-descent structure designed to handle SQL queries while maintaining compatibility with existing query structures.
Read the full story