• 2 Posts
  • 174 Comments
Joined 9 months ago
cake
Cake day: February 9th, 2026

help-circle



  • The scaling laws maxed out around mid 2023 with the release of GPT-4 - that was around the time that everyone got excited about “test time inference” which is where LLMs spit out reasoning streams to convince you they are “thinking”. If you look at where LLMs have improved since then it’s mainly in tasks where they can be strapped into a harness and given concrete feedback like “your program didn’t compile” which are also coincidentally use cases where data from gpt and Claude users can easily be harvested and new training data can be squeezed out of underpaid third world mturkers.

    In other words they’ve improved their training data in the specific domains they are operating in like we’ve been doing in machine learning since the mid 90s…

    The big labs have just been doing an incredible job of marketing (and particularly in Anthropic’s case, brain damaging themselves into believing their own marketing)