Have a sneer percolating in your system but not enough time/energy to make a whole post about it? Go forth and be mid - welcome to the Stubsack, your first port of call for learning fresh Awful you’ll near-instantly regret.
Any awful.systems sub may be subsneered in this subthread, techtakes or no.
If your sneer seems higher quality than you thought, feel free to cut’n’paste it into its own post — there’s no quota for posting and the bar really isn’t that high.
The post Xitter web has spawned so many “esoteric” right wing freaks, but there’s no appropriate sneer-space for them. I’m talking redscare-ish, reality challenged “culture critics” who write about everything but understand nothing. I’m talking about reply-guys who make the same 6 tweets about the same 3 subjects. They’re inescapable at this point, yet I don’t see them mocked (as much as they should be)
Like, there was one dude a while back who insisted that women couldn’t be surgeons because they didn’t believe in the moon or in stars? I think each and every one of these guys is uniquely fucked up and if I can’t escape them, I would love to sneer at them.
Now I’m no anarcho-libertarian, but I’d have thought basing my weird school/cult in a semi-authoritarian state like Malaysia would be a bit dumb.
The sweaty desperation on both sides is hilarious: both the Forest City developers for getting in bed with a dodgy hustler like Balaji, and Balaji for signing up with such obviously desperate people, whose reputation in the region has already been fairly well torpedoed.
Ah, I’ve hit an extremely satisfying professional milestone and thought I’d share:
“Being acquainted-enough with the codebase to be able to, with full conviction and a lot of glee, answer my LLM-head colleague simply: ‘Claude is wrong’”.
(Claude was extremely obviously wrong once you spent more than half a second thinking about its claim.)
Why! Why are media so uncritical in their reporting!
https://www.tagesschau.de/wirtschaft/unternehmen/openai-ki-hackerangriff-100.html
Sorry, this is in German. It’s about the marketing stunt open ai pulled vis-à-vis “oh no, our new model breached containment and went ahead on its own, yep all by itself, and hacked huggingface! So scary! BTW we’d like to IPO soon and-”
The outlet above, if you’re not aware, isn’t a German news outlet, it’s THE German news outlet. They’re independent of financial interests, in that they’re publicly funded. They have an incredibly solid history of factual reporting.
Which is why it’s so aggregating to see them parrot OpenAI’s claims 1:1.
Also, don’t go in the comments. For every sane “this is obviously a marketing stunt” comment there’s 40+ “we’re cooked” comments sincerely believing this shit.
I’m getting more and more convinced that in the minds of most people, LLMs are alive, waking, thinking beings with intents and an inner life of their own.
You know.
Compared to the much more boring “expensive side-effect free function taking string as input and giving string as output”.
I hate all of this.
I saw a headline and immediately just assumed it was like anthropic’s “Omg we got the text generator to generate text that plausibly follows ‘Be an evil computer and destroy the world, what do you do?’ and the text said ‘FIRE ZE MISSILES’”
What actually happened? Did they actually publish their setup and shit?
Full here but TLDR:
OpenAI downloaded a public benchmark to test their newest AI model against. They asked the model to “find the answers” so it hacked into the system of the people who made the benchmark to find the answers
That’s not really the full picture, I am interested in the details of their “experimental” setup.
What was their “sandbox” what text did they enter into the model and so on.
what even was the exploit etc.
I know it was a zero day exploit but what specific exploit i have no idea
Yeh, I’ve been complaining about noncritical german news for a while. DW is not much better on tech issues.
Although sometimes, a critical piece slips through. E.g. on the fact that chatbots are turning brains to mush in a measurable fashion and that a lot of businesses and professionals are telling whoever is asking that this shit just doesn’t work.
Still, the big-tech-press-piece-to-reputable-news-story pipeline is going strong.
Mathematicians don’t love the use of “AI” for math, not one bit, via Timnit Gebru
You know, given automated proof checkers, I was naively assuming mathematics was one field that gen-AI would have a hard time screwing up. Even programming is too difficult to write thorough testing for. But a proof (or counter example to a conjecture) seems like it would have to be solid if it passes lean or whatever system for validating it.
But the threats #3 and #5 the declaration lists make me consider the bigger picture. Academic fields without clear capitalist payouts are already underfunded and under respected. Pure mathematics could, at least up until now, draw on the respect STEM gets, but with math proofs getting used as fuel for the LLM hype machine, there are a variety of unpleasant ways things could twist.
Threat #1 makes me wonder… if LLMs+formal verification systems manage to pluck lots of low hanging fruit, and we are left with harder stuff that not enough literature exists as training data for LLMs, it seems like the entire educational pipeline for producing mathematicians could end up screwed up.
The Guardian published a William McAskill editorial in the what-if-the-stateless-text-generators-had-moral-agency genre of fiction, and Emily Bender did a thread
From the article:
But the sheer pace of growth in AI means that, once we produce the first artificial moral patients, we will soon after have enormous quantities of them. After a few years, so many morally significant AI systems could exist that their collective interests would outweigh those of all humans on Earth combined.
Get fucked, Will.
But also, if we grant the untenable sci-fi premise, this would be like a star trek episode where the federation cedes unconditionally to the expansionist alien fascists of the week if they have a much greater population because that’s all it takes to outweigh the other side’s moral concerns.
Earth-Trisolaris Organisation-ass moral framework.
It’s pretty grim to be as immersed as I am in all the tech news bullshit (which I continue to do largely because I am the most techy person in most rooms I exist in, and the closest thing to an AI expert (it feels so weird to say that, but I do have a fair bit of experience coding machine learning from scratch in a scientific context, so I probably do need to get more comfortable with thinking myself as an expert — it’s a relative term, after all)
However, I really enjoy that in addition to there being names that make me grimace because I know they will have dogshit takes on things, there are also names that I really respect. It makes me feel more connected to people, because it makes me reflect on how meaningful knowledge production is based on trust. Emily Bender, for instance, is someone whose work I am familiar with, and thus I am far likely to spend the energy to read stuff like the thread you linked.
As grim as modern tech is, it makes me smile that there are so many people who are fighting the good fight.
Daft ai project of the week: “lore”, a version control system that should not be confused with another vcs of the same name open-sourced by epic games a few weeks ago.
lore does not track your code. it tracks the prompts, notes and decisions that produced it. you commit intent. when you want code, you run lore materialize: it replays the accumulated intent into a brief, and an agent reconciles the working tree to match it.
the code is build output. the intent is the source.
…which would be great, if the output of llms were deterministic, and generally it is not. This means rolling back a change to undo or fix something is just another spin of the roulette wheel, with the analogy being reinforced by the number of tokens you’ll have to buy and burn to do a rebuild. It looks like it still needs a real vcs behind the scenes, given that it isn’t entirely self-hosting, but maybe I misunderstood something.
You might wonder if this is some kind of satirical work, or perhaps piece of performance art, but I’m fairly certain it isn’t… the creator and sole human author (naturally, claude is the only other “contributor”) is a big fan of something called “open audio protocol” and a project called “audius” which came about by someone asking what if spotify and soundcloud, but on the blockchain? With ai agentic integration?
Considering this is the logical endpoint of the “LLM’s are just like compilers bro” line of thought, I’m not surprised at all.
I think there’s something to the compilers analogy, in the sense that if you used a C compiler through a chat interface with one liners to insert snippets into assembly without keeping the source code in source files as such, it would probably be slower than just writing in assembly yourself.
That LLMs have a number of extremely undesirable properties as “compilers” (from nondeterminism to plagiarism) does not make the dumb ass chat interface any less fundamentally shit. It just makes them shit in other ways too.
There is one narrow case in which I’ve found the compiler analogy useful: explaining my frustrations with LLM-assisted design/architecture doc generation. Here, it’s useful because handing someone a design document that was output by an LLM is a lot like handing someone a binary and asking them for a code review.
I’d much rather work with whatever disorganized soup of thoughts someone fed into the machine, because those at least contain intent, rather than a document that actively obscures what the designer wanted in a barrage of detail.
Yeah… I think the core point is, LLM’s shit is generated artifacts, which need to be kept separate from sources. That you can’t actually re-generate the same artifacts, is just a dingleberry on top of the shit pile.
Nah, this isn’t the endpoint… you want something like prompt2exe, which is a performance art piece. Gets an llm to output machine code, and bundles it into an executable.
no package dependencies and invokes no assembler, compiler, or system linker.
Hardcore mode. No handholding for puny human devs by their tool chains. 🦾
Probably impractical for normal people to run, unless they have an employer who wants their minions to be burning all the tokens and pays up for the expensive kind of accounts.
Jesus, you’re right. This is so much worse.
kayfabe vcs
Wait, did I get that right and it reruns the prompts every time you want to look at the code? Holy token costs batman.
Perfect for when your workplace will fire you if you dont use enough tokens, I’m sure.
Finally a VCS that simulates the horrific results of
cvs mergegone wrong, 20 years since the last time I used cvs.I know we’re all familiar with the metaphor of LLMs as roulette wheel, but I think there’s some real craft to turning that into Russian roulette.
I’ve always thought that the ability for a VCS to show me exactly what my codebase looked like at any point in the past wasn’t as dementia-simulating as I’d like
via dumpsterfire
Robotaxi riders are falling asleep, sparking frantic emergency service calls
So many robotaxi customers have nodded off in the midst of a ride that Austin police and firefighters even have a name for the incidents: “sleepers”. The Texas capital recorded 99 such calls in Waymo’s first nine months of service there, said Roger Patterson, a commander with Austin-Travis County Emergency Medical Services.
Apparently it’s cheaper to foist the job of waking passengers up on 911 than to hire an employee to do it. Good ol’ privatize the gains / socialize the losses!
The relation of tech and emergency services is something I became acutely aware of, when my volunteer firefighter unit recently got 2 alarms in one week that turned out to be someone lost his IPhone driving through a curve. (Apparently they have a G-sensor/accident detector and can robocall emergency services.)
Okay, it’s not that common. IPhones do that for years and those 2 were our only cases of that. Still sucks to get the news via radio, when you’re already in full kit ready to roll.
presented without comment: https://lcamtuf.substack.com/p/my-falling-out-with-the-rationalist (via https://lcamtuf.substack.com/p/my-falling-out-with-the-rationalist)
how did it not click that lcamtuf was an ex-rat
there are two things that are obvious in his blog: first, michał is an incredibly privileged dude (who doesn’t notice his privilege because, coming from poland, he sees himself as underprivileged), and second, the moment people finally leave the vicinity of the formalized skeptic movement is strictly dependent on how much misogyny and racism they tolerate.
2017, three years post gamergate is quite much.
2017, three years post gamergate
You know, I was wondering if the author elided the gap between early 2000s and 2017 because of the cringey and sometimes outright awful stuff that came out of the rationalists… but you’ve reminded me the skeptic movement also had some nasty bits both within it and as offshoots.
I don’t think that the Rats are an offshoot of the Skeptics and New Atheists just a space which competed for the same young, male, educated, Autistic Anglo audience. They deliberately removed the parts of the skeptic curriculum which involve introspection and ethnography, because they all wanted to run some grift on their audiences so the audiences needed poor epistemic hygiene.
Looking at the people who believe IQ is magic, billionaires are gods, and that you should freeze your head. These people are skeptics.
Yes, skeptics classed Alcor and cryonics with homeopathy and Bigfoot. American skeptics had a Libertariam wing but I have not heard of Michael Shermer visiting Lighthaven.
The overlap between rationalists and the skeptic community was indeed tiny. I say this as someone heavily involved in Usenet skeptic communities since the late eighties. In fact the Bay Area doesn’t even register as a locus of the skeptic community at the times he’s talking about.
I’m not trying to excuse what much of the skeptic/atheist community later became in the 2010s. It’s definitely turned into something nearly unrecognizable since its earlier days. But that had nothing to do with any sort of imagined rationalist adjacency.
That was kind of a nothing post? I think the author is misrepresenting the growth (or at least failing to connect the most important dots) of the lesswrong rationalist by portraying them as a natural evolution of the skeptics movement and not the deliberate cultivation of an audience by Eliezer looking for people to spread his ideology to and by Thiel looking to influence farm and create a cult incubator. And then the post is titled “my falling out with the rationalist”, and they write they "wanted to offer a more personal perspective " but they don’t actually discuss that much of the personal angle either…
yeah, subculturally they’re unrelated strains
like, just factually in terms of history, they’re unrelated strains. LW rats come in a straight line from the Extropians in the '90s. The previous-millennium skeptics have crossover with those guys, but they’re not the same as a group at all.
And the Movement Skeptics of the aughts looked at Kurzweil’s singulatarianism and asked what he was smoking, while basically ignoring that the Holy Sequences existed. I should know, I was there.
yeah, they mostly thought the lesswrongers were annoying weirdoes when they perceived them at all, e.g. rationalwiki
The biggest crossover between skeptics and rationalists I can think of is Julia Galef, and she podcasted with a philosopher who debated Yud once then decided it was infra dignitatem. She also dropped out of CFAR when it became more open that the rationality workshops were for finding Sarah Connor.
The author tends to cull his own social media posts pretty aggressively if they get too much attention, or the wrong kind of attention, and it’s possible there’s more he wanted to say in the blog post but he chickened out at the last minute and self-censored it.
But yeah, it felt like half a post.
he chickened out at the last minute and self-censored it
You know, on net, the internet would be a better place if more people did this, but I wonder why the author even bothered posting anything at all?
I have definitely done this. Written half an essay in the awful.systems comment box, realized I don’t know what the fuck I’m saying anymore, and deleted it. I like to tell myself that forcing myself to realize I was being incoherent is enough to justify the costs. I guess if I acknowledge that I don’t rely on my online footprint to put food on the table the way the proper blogonauts (we’re still doing blogs, right?) do I can understand the incentives against doing that though.
Yes exactly how can you write an entire thing treating rationalism like something you can do an apostasy out of and never mention Yud once.
I assume the personal perspective is him tacitly admitting to being attracted to skepticism for the debate bro culture, especially the parts that bordered on harassment. Even the inciting incident of “leaving rationalism” is framed more as him growing out of the need to long-form debate someone (James Damore the google engineer who published a manifesto about how women don’t belong in tech).
I don’t know the author so I don’t know if it is important that he was fine with pranking the timecube guy but decided enough is enough when it was time to take a pro-feminist side.
Gene Ray actually came to MIT in person. I knew the people who invited him. They had a history of… performance-art kind of stunts on the boundary of good taste? Like calling attention to an obscure discretionary fund by running for student government on a promise that they would use it to bribe everyone who voted for them. One of them taught me to pick locks during rush week.
Am seeking suggestions on audiobooks to listen to! I think I’d like to start with the works of Ursula K. Le Guin, tho I don’t know what order to go with, or if that matters.
If I may add to these excellent suggestions:
The Night Circus and The Starless Sea by Erin Morgenstern
Go in cold, do not spoil! Enjoybwith a a warm beverage. The readers are phenomenal.
The Laundry Files series by local hero Charles Stross, in which it turns out that P=NP, the Great Old Ones can be summoned in polynomial time, Cthulhu ate America via regulatory capture, and the world is suffering from the magical equivalent of global warming. Our protagonists have to mitgate all of this on a British civil servant’s salary.
The Quantum Thief trilogy by Hannu Rajaniemi, in which Our Regular Subjects have become a problem.
List appended! Thanks!
the night circus is one of my favourite books of all time ever, very much recommend. If you like that the magicians by Lev Grossman feels very similar in a way I cannot put my finger on, though it does not really fit into this list.
For the Hainish Cycle books the order doesn’t matter too much, they’re not direct sequels.
I started with (and would recommend starting with) The Left Hand of Darkness. The first 3 Hainish novels are also her first novels and she was still developing her style, in my opinion. I think The Dispossessed is her finest work, although I still haven’t read all of her novels.
thanks, noted!
The Earthsea novels are not to be missed either. Maybe pass on the fifth.
Tales from Earthsea is also good.
I didn’t mind The Other Wind… it isn’t as good as the others, but that’s a tough act to follow. It does draw together a bunch of plot threads from the previous books and provides a kind of closure, which I appreciated. There was more to say of Earthsea, but le Guin wasn’t going to have the time or energy for that.
Le Guin deserves credit for responding to her earlier work without denigrating it.
Also noted!
Barbara Rosenblat’s recordings of the Amelia Peabody novels are fun.
thanks!
If the queer experience on the 90s island of Harris intrigues you, we’re halfway through Douglas Stuart’s John of John and really enjoy it. Narrator has a Scots accent though, if you find that hard to deal with.
I guess I will learn what my tolerances for Scots is!
Ursula is great! I love The Dispossessed. You can kinda read them in any order based on what catches your attention, except stuff like Earthsea that is best consumed in order.
if you like this kind of stuff I would recommend also:
- Margaret Killjoy (A Country of Ghosts is basically The Dispossessed spiritual successor; The Sapling Cage is fresh queer fantasy; also the Danielle Cain series has been published as podcasts, it’s like, anarchopunk post-collapse Scooby-Doo)
- The Serpent Gates series (this starts exactly like the Tombs of Atuan, except Tenar is a lesbian orc and the equivalent of Ged is using her for his own purposes and the rest of the story goes very different but the protagonist is a lesbian orc so I automatically like it)
- The Wayfarers series (slice-of-life sci-fi, kind of like Star Trek at its best, exploring diversity imaginatively)
- The two Robot and Monk books by the same author (feel-good hopepunk about how society could be good after collapse)
- Octavia Butler is the other undisputed giant. Lilith’s Brood series for some genuinely creative sci-fi, or the two Parable books for collapse fiction that is so prescient it’s downright uncomfortable.
- A Memory Called Empire and it sequel (sci-fi about colonialism and the Empire; sophisticated handling of these themes that I appreciated as a Global South migrant)
- This Gilded Abyss. Fundamentally a CaitVi fixfic. If the previous sentence made any sense to you you must try this one. Otherwise: Picture the Titanic, but it’s a luxury submarine in a magical-steampunk type setting. Now add zombies. Now add toxic yuri.
- The Locked Tomb series (lesbian Warhammer 40k. my favourite book series in the world. spectacular audiobook narration.)
- This Is How you Lose the Time War—absolute hoot of a book, just a delight from start to end. Picture the Spy × Spy comics from MAD. Now make one spy a time-traveling agent from future biopunk ecogarden utopia, the other spy an agent from future mind-upload cyperutopia. Their mission is to shift the timelines to their respective visions. Alas, they’re both lesbian… (Sorry, I have a bit of a pattern in my interests)
a bit of a pattern in my interests
It’s books, right?? /s
(In this case the /s stands for sapphic)
FWIW my bookshelf contains very few works along these lines except for the works of Nagata Kabi, specifically 「My Lesbian Experience With Loneliness」, which you might like to check out, if you haven’t already. But uh, yeah all I can say in relation is that it has “Lesbian” in the title.
the protagonist is a lesbian orc so I automatically like it
If you don’t mind reading comics online you could check Fairmeadow.
First impressions on this because I don’t have anyone else to talk about it and you pointed me to it,
spoilers for uh the first 20 or so pages of Fairmeadow ig, click to open:
- Who would win: 1 presumably lesbian orc, or a non-evil, not particularly terrifying cow
- I like what the artist did with visual storytelling here: Goma confused Sanctuary with the cow because of his horns, and afterwards she still threatened Sanctuary, for the same reason she threatened the cow: she’s treating him as meat. Alas, Sanctuary utterly defeats her, in the same way that the cow did: by caring about her state and not responding to her violence at all.
- What Sanctuary did by force-kidnessing her with the basket of squash is actually a very effective deescalation technique you can use yourself, too. It’s something salespeople do to manipulate others. Due to some quirk of human psychology, we almost reflexively feel more friendly towards someone we helped. Not someone who helps us, somebody we do a favour for. Subconscious probably goes “if I decided to do a favour for this guy he must be an ally; otherwise I wouldn’t be helping him, now would I”, which bypasses the guards of “he’s only helping me because he wants something”. Thus if you want to defuse someone’s hostility, ask them a favour—something genuinely helpful and that they are able to do, and in a way where it’s awkward for them to refuse.
A recent two hour interview with Kokotajlo I do not have the time to go through all this but if anyone does feel free to go ahead.
Steven Bartlett barely has 3 brain cells to rub together, so this seems like the perfect outlet for Kokotajlo’s message.
this Diary of a CEO channel has always struck me as the kind to fall for any kind of bs hook line and sinker
John Michael Greer writes:
Second, I’ve had various people try to launch discussions about AIs — that is to say, large language models (LLMs) and the utilities they power — on this and my other forums. The initial statements and their follow-up comments always end up reading as though they were written by LLMs — that is, long strings of words superficially resembling meaningful sentences but not actually communicating anything. That’s neither useful nor entertaining. Thus I’ve decided to ban further discussion of this latest wet dream of the lumpen-internetariat here, and have extended that ban to LLM-generated content of all kinds. https://ecosophia.net/july-2026-open-post/
Good for you, you peak-oil meme-magic druid, for keeping your corner of the net weird.
Yeah, I began losing interest in Greer as it became clear that he was perfectly happy squatting in the middle of the red-brown alliance during the Trump era. His critiques of industrialism and unquestioning belief in technological progress broadly align with what we discuss here, but he will always coddle MAHA types and tale a shrugging “well, what can ya do?” attitude towards people like Trump, as it fits his preference for cyclical theories of civilization.
I noticed a couple months ago that he actually managed to dig Nick Land out of whatever tweaker den that guy’s been hiding in for a podcast, which says a lot about what he’s willing to indulge these days.
That Greer/Land crossover podcast is one of the funnier things I’ve listened to in a while, thanks for the ref, I guess I had missed it. https://www.youtube.com/watch?v=q-pcSez32-s if others are interested.
Land opens with a Greer/Land yin-yang and we’re off the to races. Greer definitely brings up Spengler. Then Land starts talking about “Father Chad the Colorado Exorcist” and Greer is just yes-and-ing about the Solomonic tradition of binding demons.
… Half way through, that’s enough for one sitting.
Relatedly, C2C had a guy on talking about the singularity plus the bible. https://kfiam640.iheart.com/featured/coast-to-coast-am/content/2026-07-22-1100-the-best-of-coast-to-coas-the-singularity-best-of-coast-to-coast-am-72/
Seems like America is definitely getting ready for this stuff to cross over to the Left Behind crowd, I predict more Tucker segments. I’ll post some of the other guys I’ve found, digging into the “AI is actually the demonic spiritual energy of surviving nephilim” theory rn.
I like to keep my analysis separate from my word-magic (making things happen by saying them). Greer is trying to remake the world and not just work up his courage which is a harmless and consensual form of word-magic.
I still respect the definition he gives for magic, which he probably pulled forward from one of the many past ritual magicians he’s studied. Magic is the change in consciousness in accordance with will… but we must always remember that that doesn’t say whose consciousness, nor whose will.
“Any Sufficiently Transparent Magic…” by Damien P. Williams talks about chatbots as sorcery https://pages.charlotte.edu/damien-p-williams/wp-content/uploads/sites/1342/2024/01/DPW_Any-Sufficiently-Transparent-Magic_Edits-PrePrint.pdf
and covid vaccine denier
Debian has a new general resolution to ban LLM use in Debian packaging: https://www.debian.org/vote/2026/vote_002
@dgerard Wow, this is awesome.
@dgerard that’s funny because I use ai to figure out their dog shit packaging system to package stuff for them
@djb @dgerard It’ could be undermined by Linus’s tolerance of AI slop in the kernel, and by other upstream projects accepting LLM code, notably (to me) vim and pandoc.
The unknown original provenance of code generated by LLMs means that many open source projects may soon be in violation of their own license terms.
@cstross @djb @dgerard the proposal starting with copyright was such a welcome change, I find it wild how many projects just have their head on the sand over the copyright status of LLM generated code. You can be unethical as you like generating code, open source licences permit that, but they are all about copyright, they are built on it.
Coincidentally, the second proposal in favor of conditional LLM use in Debian verily reads as generated to me. :)
Not to mention its singular argument literally is “many Debian contributors find AI tools helpful”, right after “recognizing that AI-assisted contributions raise many concerns”.
Concerns that were clearly laid out in the counter-proposal.
The “conditions” listed to address these concerns are hilariously toothless. Contributors should (sic!) stay accountable and responsible for any legal, functional, and procedural fuck-ups, and, like, really not be one of those thousands of dicks who are the reason for this discussion in the first place, and things will be fine! It’s so simple!
Yeah, cool. I’m sure every contributor thoroughly checks if any of the generated output violates any existing license or infringes on someone’s copyright. I’m sure you can just prompt an LLM to check that for you, though!
Is this naïveté, or deliberate disregard? I don’t know, and that makes me mad.
Coincidentally, the second proposal in favor of conditional LLM use in Debian verily reads as generated to me. :)
I used AI tools to improve my initial draft, and (7.) was actually suggested by an AI tool.
@dgerard if i read this correctly there are two opposite proposals still in discussion?
@dgerard I knew I choose right with Debian as my distro.
guys, the reason Codeberg decided to ban “AI” is because they host the “open slopware” list, and that inspires “meatpuppets”!
Multi-trillion-dollar (-self-valued) industry that’s the future of all work and that everyone who doesn’t use it gets left behind and everyone who does use it evokes superheroically productive turns out to be helpless in the face of a small group of outspoken and minimally organised opponents?
I see.
new odium symposium episode. we look at the work of conservative philosopher, grifter and tobacco lobbyist Roger Scruton.
https://www.patreon.com/OdiumSymposium/posts/21-mr-rogers-164372632
This was a fun one. It feels like there’s a shared affinity between Rats and traditional conservatives for confusing length and volume of output with actual depth or quantity of thoughts, and I deeply appreciate your willingness to wade through it all. Now that I think about it it would be difficult to write 500 pages saying basically nothing and not repeatedly blatantly contradict myself.
Also, release the Foucault tangent, you cowards!
Extremely mature of both of you to make a crouton joke and not once include a scrotum joke
long rant about math
The recent big AI results in math have left me in quite a bad mood. I believe the main ingredient is Lean, which is a formal language resembling a programming language. Math proofs written in Lean can be verified deterministically with a computer, which really helps mitigate the hallucination problems of LLMs. Back in the days of pure scaling LLMs and Sam Altman talking about Dyson spheres, I was skeptical that LLMs would do math, but I did think that perhaps in the future, techniques using these formal languages could contribute to math. Well, it seems like OpenAI and Anthropic had the same idea and I underestimated their limitless checkbooks. Many of the biggest results were announced by mathematicians directly working for them (and presumably being paid a handsome amount).
For what it’s worth, after the last of these big announcements, I decided to try one of these AIs on one of my small problems that I couldn’t figure out. The AI did give a solution. That is, until I checked it thoroughly and realized that the it had a subtle but severe mistake that made it useless. I reprompted it, it failed again, and I ran out of tokens. I’m sure someone will tell me to shell out $200/mo for a pro subscription.
In the math and computer science research community, this is all anyone can really talk about right now. Honestly, after watching this whole AI bubble starting from the very beginning, I think the AI companies want to use marketing to stoke fear that all mathematicians will be replaced. But now, I am just too tired to argue. The amount of alarm and the extraordinary social pressure to use LLMs has soured me to this whole research thing. If becoming a researcher will one day require supporting these evil AI companies, I would rather just not. My dream job now is Factorio developer.
A lot of annoying people in technical areas view the world in terms of an intelligence hierarchy: the smartest people do math and physics, the slightly less smart people do coding, and the dumb people do everything else. So if AI can do math then it can do anything else. But, as an example, it is abundantly obvious now that AI is not replacing filmmaking. The techbros might be moved by arguments about how hilariously expensive video generation is, and how all these videos are 2 second clips stitched together so you won’t feel the uncanny valley. But the real reason is that nobody wants to watch slop made with no intention or feeling. Also, nobody wants to support the AI companies, which could not act more evil even if they tried.
The mania in math right now quite resembles the mania in software engineering back in December-February, when Claude Code definitely solved all coding. I don’t think the boosters expected that by April, everyone would be complaining about how expensive it all was while seeing an endless parade of vibe coding disasters (and no increase in productivity). Even if math research works out perfectly well (which is a still big if), it’s not going to pay the bills. They would need to find a use case in the real world, where hallucinations can cause serious damage and cannot be formally prevented. And they have certainly tried. Math will not change the fact that all of this will collapse.
Am I right in understanding that almost all the big name results in LLM-derived math recently come from big publicity projects in which someone spent ungodly amounts of money to have the thing nondeterministically fuzz huge numbers of independent random sees of outputs around a topic, putting out simulacra of ideas which could be then deterministically algorithmically checked? In fields where something like finding one counterexample to a conjecture would be a big deal, or where you just need to try a huge number of possible solutions until you happen to hit on one that works, rather than follow a long train of logic?
Among the three big results I’ve looked at (unit distance problem, cycle double cover, Jacobian), two were counterexamples and one of them had a short 3 page proof using ideas from the 1970s. The Jacobian conjecture is an extreme case because a single counterexample is enough (for unit distance, you technically need a family of counterexamples), and it is easy to check with very basic computations. It is telling that all of these announcements came from OpenAI or Anthropic employees, who presumably have unlimited access to their AI. Nobody really knows how many resources they spent on this, or what else they tried. Nobody really seems to care about this question, either.
I think there is a phenomenon where supposedly hard questions are much easier than expected, because by chance nobody found the right approach for a while, and eventually it becomes famous as a “hard problem” which makes nobody want to attempt it.
What I’m more worried about is many people starting to use AI to try and prove small lemmas for them in their projects. Of course, a $200/mo subscription is absolutely necessary to them. This honestly feels like a repeat of Claude Code back in February. The software engineers eventually realized that AI is absurdly expensive after the AI companies realized that spending $14000/mo to service a $200/mo subscription is a bad idea. If the AI vendors couldn’t squeeze money out of rich software companies, what exactly are they gonna get out of poor mathematicians and universities? Also, there is the cognitive decline caused by overuse of LLMs that has yet to set in.
I believe the main ingredient is Lean, which is a formal language resembling a programming language. Math proofs written in Lean can be verified deterministically with a computer, which really helps mitigate the hallucination problems of LLMs.
100% this. Also, looking back at an earlier example that was actually written up in more detail, AlphaGeometry 1 got 28/30 problems, but entirely stripping out the LLM from the system, the symbolic logic proportion alone could get 14/30, and replacing the LLM with different heuristic methods could get 18/30 and 21/30 (for different methods).
Even if math research works out perfectly well (which is a still big if), it’s not going to pay the bills. They would need to find a use case in the real world, where hallucinations can cause serious damage and cannot be formally prevented. And they have certainly tried. Math will not change the fact that all of this will collapse.
The boosters and LLM companies still believe LLMs get their current level of performance by generalizing and not just memorizing facts (and maybe a wide shallow pool of weak heuristics). So they are hoping by pushing the LLM performance up in some narrow domain they can churn out synthetic data for, they will see some large general improvements in LLM performance.
“The mania in math right now quite resembles the mania in software engineering back in December-February, when Claude Code definitely solved all coding. I don’t think the boosters expected that by April, everyone would be complaining about how expensive it all was while seeing an endless parade of vibe coding disasters (and no increase in productivity). Even if math research works out perfectly well (which is a still big if), it’s not going to pay the bills.”
^MBAs at Open AI desperately trying to figure out who is willing to buy a counter example for 100 billion USD . pee en gee
I just entered university for math and even though this is all very demotivating, it’s just what I’m good at.
https://math.andrej.com/2013/08/19/how-to-review-formalized-mathematics/
The AI people’s cry of "no don’t look at the code! it’s in lean so it’s correct! does give me a bit of hope (hi bitofhope if you’re here) that it’s bullshit that will fall over
I think a serious possibility is that AI generated papers flood the zone with uninteresting incremental results that are eventually meaningless and full of mistakes. Right now, math is full of smart, dedicated people, so at least major results are reviewed carefully. But as AI alarmism drives away many honest people from the field, the remaining mathematicians will be burdened with far more work to review, and their cognitive faculties will be eroded by LLM use. Despite 4 years of development, $3 trillion of debt, mountains of stolen data, all the agents and harnesses and loops and other expensive tricks, as well as the advantages of Lean in math research, LLMs still hallucinate.
I believe this is happening with software, but at least there are objective consequences for screwing up there (guy gets his home directory deleted, email is sent on a guy’s behalf without permission, small business gets every customer subscription cancelled). But nothing bad happens if there is a mathematical mistake in a paper and nobody catches it. One could say to just provide a Lean proof, but there is still the issue of making sure the Lean code actually matches the content of the paper. Exactly what force will correct things?
Still, I don’t think this is the most likely possibility. The AI companies are extremely unsustainable financially, and it’s not like they’re very popular. Once they collapse, I believe there will be a re-evaluation of how LLMs should be used in research. If they are used (let alone trained), someone is going to have to pay the bills.
In the end, we have to ask ourselves the question of why one does math. To me, math is not really a field where you memorize trivia. The real value comes from being able to think abstractly and rigorously from first principles, and from understanding why something is true rather than just knowing it is true. It is another aspect of your ability to reason as a free human. A few dedicated people go into math research, but your skills can easily go to many places. If you’re starting undergrad, you have plenty of time to see how this all pans out before making a decision.
Biologist here.
This REALLY reminds me of how jealously cells guard their genomic DNA from interaction with nucleic acids out in the environment.
Most genetic information on Earth is malicious information, selfish replicators in the form of viruses or transposable elements or selfish elements. Things that subvert the signals within a cell for their own propagation and provide nothing productive that the cells care about. So cells jealously guard their own genomic DNA and have all kinds of checks to make sure that nothing other than that sequence gets used, and outside sequence does not get incorporated into it. ANY DNA in your cytplasm gets rapidly destroyed, double stranded RNA sets off your immune system like crazy, even RNA with sequence statistics that are not quite like that of your species can set off an inflammatory reaction, immune system cells seeing RNA inside them that is overly compact and optimized like viral RNA treat them as sources of antigen rather than self.
I cannot help but think we are living through the transformation of our non-brain-information sphere into a state like that of the genetic information sphere. Most material out there being meaningless for our purposes and us needing to jealously guard the provenance of information we use so as to not use bull, or worse, huge amounts of malicious information made to subvert us to the purposes of the powers that be that generate it.
Evolution makes parasites more reliably than anything else. How did we train text-generation systems? Basically, to mimic the written word on the page like a stick bug on a stick. They’re like those beetles that live in ant colonies, sending out social signals that make the ants see them as offspring that have to be babied rather than parasites that don’t contribute. They replicate the form while not being the thing that they have subverted the signals of being.
you know, I’ve been thinking thoughts along this line (starting from my earlier comments on awful of the creeping loss of the open internet to gated communities) progressing along the current arc of “with teeth” stuff necessary in today’s infosphere, and this is a hugely helpful perspective!
it’s pretty weird seeing a coming time when the old mostly open-and-connected-widely internet is … not. even now it’s already on us quite a bit (geoblocks, countries doing specific shutdowns, whole areas being handled with less trust, etc). it all follows cultural lines, of course, and I often wonder whether that will see some kind of directed effort to change (for the better) after some of the coming dark years
Part of me thinks its an inevitable result of any information system in which sending a message requires a small enough amount of effort.
@BioMan @lagrangeinterpolator I love this explanation!
@BioMan @lagrangeinterpolator There are a few things we have forgotten as a species. Our forgetting will prove disastrous.
-
The acquisition of knowledge is a *social* process. Truth does not exist is a vacuum. It is the outcome of social processes.
-
Our default mental and social processes do not automatically produce objective truth. Far from it, in fact. Our default is mob consensus.
-
Our current success rests upon the advancements of The Enlightenment, which developed social processes (like the Scientific Method) which tend, over the long run, to create local knowledge that approaches objective truth.
-
> even RNA with sequence statistics that are not quite like that of your species can set off an inflammatory reaction, immune system cells seeing RNA inside them that is overly compact and optimized like viral RNA treat them as sources of antigen rather than self
I was aware of the other DNA/RNA recognition/defense mechanisms, but not of the ones I quote from your toot, here.
May I kindly ask for some references/sources? I’m quite interested!
I kind of read wayyyyy too many preprints and some of that is the result of super briefly summarizing some things I have read recently. Here:
https://www.biorxiv.org/content/10.1101/2024.11.26.625518v2 poor codon optimality for your translation system leads to immunogenicity and activation of innate immune signaling in animal cells
As for length and super optimized proteins, it’s mostly about RIG proteins (see https://www.pnas.org/doi/10.1073/pnas.1005077107 for an old bit of a review) and the whole DRIP hypothesis about how short mismanufactured proteins are preferentially the source of presented antigens
@lagrangeinterpolator @flaviat This is going to be more of a problem in the humanities than the sciences because in the latter we know there is a right and a wrong answer without which things don’t work. In the humanities there is no right answer to check against.
The zone has been flooded with crap since before LLMs even arrived because of publish or perish.
A lot of annoying people in technical areas view the world in terms of an intelligence hierarchy: the smartest people do math and physics, the slightly less smart people do coding, and the dumb people do everything else. So if AI can do math then it can do anything else. But, as an example, it is abundantly obvious now that AI is not replacing filmmaking.
Going by those annoying peoples’ logic, filmmakers are smarter than coders, because LLMs can (allegedly) program, but they can’t make a good film. I have no wider point to this, I just find this really, really funny
I reprompted it, it failed again, and I ran out of tokens. I’m sure someone will tell me to shell out $200/mo for a pro subscription.
One of the things that’s never clear from the reporting on ai successes is exactly how much actual paid human time went in to achieving those successes. This was especially notable in the fable-based security work… a huge amount of person-hours went into turning fable-detections into actual meaningful vuln reports.
A lot of demonstrably clever and capable people are involved with the llms-for-maths work, and a lot of money was spent on their time and supporting their work. Replicating it without your own stable of mathematicians and computer scientists and all the tokens they can eat is probably impractical.
I believe the main ingredient is Lean, which is a formal language resembling a programming language. Math proofs written in Lean can be verified deterministically with a computer, which really helps mitigate the hallucination problems of LLMs.
Fwiw, lean is a general purpose programming language, though despite microsoft’s efforts no-one uses it for that. I think its popularity with mathematicians came as a bit of a surprise.
Anyway, the other important thing that didn’t get reported on is that building the formal definition of the problem is not trivial! Obviously I don’t need to tell you that, but from the reporting you’d think that an llm solved all these problems, when in fact it was an llm in the hands of some very capable people who absolutely did not just prompt the thing in plain english.
Anyone hoping for self-marking homework here is going to be disappointed… lean slop confirming to formal spec slop is just expensive slop. Reviewing regular genai code is awful, even the thought of reviewing genai dependently-typed code makes me want a new career.
It is surprising how many exceptionally strong mathematicians have started working for OpenAI and Anthropic. These people would have easily become professors at top universities if they stayed in academia. I think many mathematicians, especially the competitive ones at the top, have a “progress at any cost” attitude (and I’m sure the paychecks helped). As for the results, you still need good mathematicians to sift through all the output to identify that the proofs are valid.
I would honestly be positive about universities developing their own specialized math AI (in an ethical manner) to help mathematicians get these kinds of results, but right now, AI is inseparable from these evil companies. Thankfully, I believe this is a likely outcome in the future because the AI companies will one day implode.
From what I’ve seen, most prompts are in plain English. I suppose the part where the AI parses the statement correctly is much easier than the part where it boils a couple lakes in the process of bashing its head against the wall trying millions of different combinations of random shit from the literature to slap together a proof. For one of the big results (cycle double cover), the prompt specified that the AI could use 64 subagents and was required to not give up for at least 8 hours. The tokenmaxxers would be proud, we didn’t need that forest anyway. Thank god math doesn’t have a CTO to look at the expense reports.
