• 1 Post
  • 191 Comments
Joined 3 years ago
cake
Cake day: July 29th, 2023

help-circle
  • Terence Tao observes that an llm may solve a complex mathematical problem without usefully improving the state of mathematics, because the intermediate results of the process themselves can have significant value.

    https://mathstodon.xyz/@tao/117207849921390904

    But such an incomprehensible proof would not be the primary value of the exercise. The process of starting with one ansatz, discovering the precise obstruction preventing it from working, adjusting the ansatz to (partially) eliminate that obstruction, and then iterating, would almost certainly reveal important new insights about fluid mechanics that would not have been feasible to obtain by other means. Crucially, this iteration would only work well at producing such insights if the iterator did not have access to the final ansatz in advance, as this naturally inhibits the exploration of alternate routes to the ansatz that are superficially “dead ends”, but in fact end up being highly instructive in the nature of their failure.

    But there is now a scenario in which an autonomous AI harness, backed by an enormous amount of computational resources, performs this entire iteration internally, and ends up producing the final ansatz, and thence the solution to the Navier-Stokes regularity problem, while the AI company running the harness keeps the process to arrive at that ansatz almost completely out of public view. Technically, one of the most prominent open problems in mathematics would now be solved; but there would be almost no value added to mathematics as a consequence. It is theoretically possible that with some herculean (and heavily AI-assisted) additional effort by a third party, some portion of the process could be reverse-engineered to recover some actual insight and understanding from the solution; but this would be a far less efficient process than if the solution had been obtained via a diverse combination of both human mathematicians and machine assistance as mentioned above.

    https://mathstodon.xyz/@tao/117207855800042681

    He’s not the first to have pointed this out, but I think this was a better summary than similar posts that I’ve come across and at least partially understood.


  • Today’s cognitive surrender news: people asking chatbots to generate them hiking maps and give them advice on route planning and climbing mountains, what to eat on long hikes, and where to camp in national parks. It ends about as well as you’d think.

    https://www.sfchronicle.com/outdoors/article/shasta-climbers-rescued-chatgpt-22414629.php

    (paywalled, but the popovers are easily hidden with browser tools)

    the novice climbers, from Roseville, said they’d entrusted Google’s AI chatbot, Gemini, to help them craft key logistics of their trip — from the best route up the mountain to the types of food they should pack and how much water they should carry with them.

    On a separate incident:

    a man stopped rangers to ask for directions. He showed them a print-out map he’d been following and explained that he’d prompted ChatGPT to produce it for him.

    “It was terrible — terrible — not even close to accurate,” Meyers said. “It had route lines and place names that were totally off — just ridiculous. We were like, dude, you gotta throw that away.”

    The original climbing group were standard idiots, having intended to do most of their navigation by phone app, and then failing to have a backup even a working recharge power pack. They also left litter everywhere at that camp site, the usual mark of thoughtless people.

    I’ve no doubt that they’d have fucked things up in the pre-llm days too… they seem the type. But chatbots are power tools for stupidity and offer whole new ways to abdicate thinking and wander into trivially preventable danger, which in turn means more risk for park rangers and mountain rescue teams 🫤





  • Article on businessinsider with the nauseating title “The modern parenting throuple: Parents are at a breaking point — AI companies see an opportunity” because the age-old silicon valley product design technique of asking what your mother used to do for you, and how it could be sold back to you via a subscription is still very much in use.

    https://www.businessinsider.com/parents-breaking-point-ai-companies-opportunity-2026-8

    I confess I haven’t read the full article (which is why I haven’t given it a proper post) but hypervisible.blacksky.app has kindly pulled the highlights:

    https://bsky.app/profile/hypervisible.blacksky.app/post/3mtupdpfcjs2p

    One of the best bits:

    The most tech-savvy parents have built their own tools. Sarah Baldeo, a cognitive neuroscientist, spent the past year traveling for work, so she built a custom GPT of herself to chat with her 15-year-old son when she’s unavailable. Baldeo tells me she programmed the chatbot with her personality, uploading screenshots of texts between her and her son, along with documents about her values, the family rules, and challenges she felt in parenting.

    Baldeo wanted her son to ask the chatbot advice, and he did prompt it with questions he may have fielded to her, like, How do I diplomatically tell my teacher I’m overloaded with assignments? But he also asked the mombot how to convince his human mom to let him hang out at a coffee shop in Chicago in the evening — something Baldeo had been hesitant to allow.

    The bot told him, as Baldeo recounts, “Your mom very much values new experiences, being around people from different cultures, different socioeconomic backgrounds, and this is how you should frame it to her: that you’re exploring a part of Chicago that is more diverse.” Her son took this tact to Baldeo, and she said yes. It was only later that she saw the chat history that let her know he had gamed the bot to manipulate her.

    🤮

    Why the fuck do these people have children. Why the fuck does any functional human being think it is a good idea to outsource raising their children to sam altman and his chatbot.

    Who on earth thinks that training a chatbot to wear their personality like a rubber mask and manipulate people is going to end well? Honestly, I think this is perhaps the bit that triggers the most revulsion in me. I cannot comprehend wanting to be complicit in my own replacement by this thing, but so many people seem excited by the prospect of doing this to themselves.



  • New book: superlawyers! how claude makes you the best lawyer in the room.

    https://www.amazon.com/dp/B0GHYS5FC2

    The author even shares his prompts for free!

    https://github.com/danlorry/superlawyers-prompts

    Every prompt here is designed for that pattern: Claude multiplies your judgment, never replaces it. The prompts are ready to paste into Claude. The book explains why they are structured the way they are.

    The prompts mostly seem to be structured as “imagine you are an attorney. Now do this homework exercise”. There’s a passingly interesting idea around having the chatbot attempt to form a counter argument to your own claims, which you can then use to make your own position stronger, which seems like it might actually be a genuinely okay idea especially if you expect your opponents to be llm sock puppets, but everything else looks pretty much like garden variety “do my thinking for me”.

    I’ll admit I didn’t read the prompts closely, so may be I’m being uncharitable, but I doubt there are many nuggets of gold hidden in there.

    h/t to rahaeli




  • Nah, that’s just the global-warming-induced warm water they like, the presence of a reactor was incidental… swarms around any sort of suction system are going to be a problem.

    I can offer you a upgrade in the form of an array of radioactive spinning blades if you’d like.

    (in theory you could deploy your data centres in places where nutrient levels are low, so there’s less plankton and so a much reduced risk of jellyfish swarms, but I have absolutely no faith that these cowboys would even stop to think about that for one moment)










  • Salt water is famously a forgiving environment to build in, and oceanic weather is benign and predictable. Not that it would be a problem anyway, because move-fast-and-break-things people can be trusted to do the right thing when it comes to running delicate equipment in places more hostile than mid california, and will not skimp on staffing and maintenance. You can look at all the the successful seasteading operations to see how well this will go.

    There are also no problems I can foresee with putting a bucketload of fissiles in international waters, with no scope for finally uniting old-school piracy with new-school piracy, either. This plan is great, and no-one will have a problem with it.