Recent independent tests of OpenAI’s GPT-4o (Nov 20) show a noticeable decrease in performance compared to the August version. Key results include:
- The Artificial Analysis Quality Index dropped from 77 to 71, aligning with GPT-4o Mini.
- GPQA Diamond scores fell from 51% to 39%, and MATH scores dropped from 78% to 69%.
- Output speed increased from ~80 tokens per second (Aug) to ~180 tokens per second (Nov), signaling a notable speed improvement.
Although the model is faster, it appears to be smaller and less capable, with no price change. Developers are encouraged to carefully evaluate the new version before switching from the August release.
Google’s Gemini AI Set to Get Voice Chat Support in Latest Update