

I wouldn’t argue with someone who said reasoning models are a substantial advance
Oh, I would.
I’ve seen people say stuff like “you can’t disagree the models have rapidly advanced” and I’m just like yes I can, here: no they didn’t. If you’re claiming they advanced in any way please show me a metric by which you’re judging it. Are they cheaper? Are they more efficient? Are they able to actually do anything? I want data, I want a chart, I want a proper experiment where the model didn’t have access to the test data when it was being trained and I want that published in a reputable venue. If the advances are so substantial you should be able to give me like five papers that contain this stuff. Absent that I cannot help but think that the claim here is “it vibes better”.
If they’re an AGI believer then the bar is even higher, since in their dictionary an advancement would mean the models getting closer to AGI, at which point I’d be fucked to see the metric by which they describe the distance of their current favourite model to AGI. They can’t even properly define the latter in computer-scientific terms, only vibes.
I advocate for a strict approach, like physicist dismissing any claim containing “quantum” but no maths, I will immediately dismiss any AI claims if you can’t describe the metric you used to evaluate the model and isolate the changes between the old and new version to evaluate their efficacy. You know, the bog-standard shit you always put in any CS systems Experimental section.
Hey mate what do you think learning is. Like genuinely, if you were to describe the process of learning a subject to me.