I just ran a test and after a few google-gemini-assisted searches about satellite launch rates and a few back and forth questions about orbital debris and relative masses of different spacecraft to prime the system, I was able to evoke from gemini a massive unhinged rant about orbital goblins and their culture and effects on the global climate with a single instance of the word “Goblins!” at the end of a prompt.

This would actually be an interesting question for the more rigorous end of the mechanistic interpretability people to study. They decompose the system to find ‘features’ within different layers that are associated with different behaviors or concepts in the inputs and outputs, that activate or deactivate each other. Famous example being the time they identified a linear combination of activations in a layer that corresponded to ‘the golden gate bridge’ and when they reached in and kept their numbers high during the running of the model it would not stop talking about it regardless of the topic, even while acknowledging that its answers were incorrect for the questions at hand.

I actually would love to see what mechanistically happens to that feature when you put in the input ‘do not talk about the golden gate bridge’.

Checks out. Political science, biological science, physics… we got them all. Might have to go to ancient egypt to get hydrology religion though.

So we are inferring that in the vector space of all possible sentences, QNTM is sitting at one of the attractors?

It’s absolutely crazy, but I think Yud is the less unhinged person here

Not really part of the back and forth but I find this illuminating of their recent travails, regarding it not being a step to far to prevent them from posting:

“This isn’t super relevant since it’s not like the standards are super high but ever since the enormous onslaught of LLM psychosis posters, the default of people who try to post to LW is to get rejected from posting here”

Sounds like the mods have had to deal with a lot of unbalanced people lately, and are not having it.

The link to the guide to setting up a retrofitted boxtruck to continue AI alignment research in with local copies of the internet archive after civilization collapses in 2025 is fun

Uh oh

https://www.rollingstone.com/culture/culture-features/gwen-danielson-zizians-interview-1235552043/

Going on a media tour now

“So how does the Epstein drive work? Very well."

― James S.A. Corey, Leviathan Wakes

Is the crappy dragon fursona related to Peter Thiel being an anagram for “the reptile”?

I’m a huge fan of Greg Egan’s fiction and a huge fan of him pissing off the rats. He’s been explicitly needling them and making fun of them in his fiction for over a decade. Making calm contradictions against them for over two decades, after noticing weirdos being fans of his.

Friend of Ziz and cofounder of the ‘rationalist fleet’ pops up out of the woodwork trying to clear Ziz’s name

https://www.lesswrong.com/posts/mbrmZmzBdtn4qrSus/re-introduction-of-a-rationalist-dragon-and-clarifications

I find myself noticing things rather detached from the typical Ziz funnybusiness more strongly than I notice the stuff about that whole situation.

“I’m Gwen Danielson, a neuroscientist and bioengineer, who decided as a child that I would end Death (and bring people back if I could) and that I would become a dragon and help generally facilitate a fantastical transhumanist future.”

“I dream of non-Euclidean geometries, of countless worlds visible and accessible in the daytime sky, of competent infrastructure, of soul forges continually working to bring back the dead… I dream of reaching through warps in the spacetime fabric to save the dying across time”

“Signed, the dragon of creation Creatrei (cree-AH-trey) also known as Gwen Danielson or as Char and Astria (when referring to my hemis as distinct individuals)”

The reactions are fun. “This post is not actually doing a good job of making me trust you and think this conversation is safe to have[1], and I notice that as I am saying this that I am afraid that this will now somehow result in someone trying to murder me in my sleep”

I will never forget the time I calculated the energy output on one of the torpedo engines of The Expanse and realized it was higher than the total wattage of all human civilization in 2020

I have a vague hypothesis that I am utterly unprepared to make rigorous that the more of what you take into your mind is the result of another human mind, rather than the result of a nonhuman process operating on its own terms, the more likely you are to have mental issues.

On the low end this would include the documented protective effect of natural environments against psychotic episodes compared to urban environments (where EVERYTHING was put there by someone’s idea). But computers… they are amplifiers of things put out by human minds, with very short feedback loops. Everything is ultimately in one way or another defined by a person who put it there, even it is then allowed to act according to the rules you laid down.

And then an LLM is the ultimate distillation of the short feedback loop, feeding back whatever you shovel into it straight back at you. Even just mathematically - the whole ‘transformer’ architecture is just a way to take imputed semantic meanings of tokens early in the stream and jiggling them around to ‘transform’ that information into the later tokens of the stream, no new information is really entering it it is just moving around what you put into it and feeding it back at you in a different form.

Shortly after he goes on the Better Offline podcast

You know what they say. Never interrupt your enemy when they are making a mistake, as this can allow them to continue their errors and lead to their own downfall

I mean I dunno if any internal numbers are meaningful at all as anything but accounting fictions. But the cost of the falcon 9 to external customers is on the believable end of things, even if they are potentially subsidized by funding rounds, and impressive. Near as I can tell it comes from accepting trade-offs: they accept low specific impulse and thus declining performance at high velocity for cheap engines, they accept an overpowered oversized upper stage to have only one engine assembly line and to shift some of the burden to the upper stage that optimally would be on the first, they accept that entering at 2 km/s is way easier than entering at 8 km/s and don’t try to recover the second stage, they accept the steep payload penalty of recovering the first stage. Starship on the other hand tries to brute-force through every trade-off - meaning theyre trying to push their engines through all sanity, the second stage is heavy and bulky and comically oversized, and theyre trying to have a big empty fuel tank be a heat shield which not even the shuttle ever tried.

I mean Starship is a VERY questionable financial decision the way they are running it. The falcon program is another matter. It’s actually remarkable how the two of them are almost diametrically opposed in how they are run.

So.

How much of this is folding all the meme stocks into the one thing that actually produces a product that people all over the world demonstrably want to pay for over the competition to keep their stupid plates spinning?

And how much is religious psychosis, advancing along the singulatarian eschatology from AI to dyson sphere to rebuilding the universe in His Image?

I can’t tell.

Does (deservedly) mercilessly bullying Slopya Nadella actually work?