Not only can it solve the halting problem, but also the ignition problem!
jedbrooke 28 minutes ago [-]
I wonder if this approach would work for AdderBoard[1][2] (a competition to make hand written transformers that can add two 10 digit numbers in the least amount of parameters possible), or if it uses way too many parameters to be competitive
Heh, this is very timely, it is more or less exactly what I'm busy with right now but I'm taking the idea in a completely different direction. Fascinating.
jerrythegerbil 3 hours ago [-]
There was an interesting bit of research that qwen has a single neuron that makes it “demonic evil qwen” and a simple “Who are you?” will get some absolutely terrifyingly vile results.
Anyways, what I’m trying to say is I want evil qwen to be my CPU.
krackers 3 hours ago [-]
The single-neuron steering stuff has been going around twitter lately, do you happen to know who was the first one to do it?
I wonder how this squares with the recent "NNs are already clean" interpretability finding, which may have also been the impetus for this kind of research. Similar to their work on CNNs, maybe residual-steering is overkill, there might be just a single neuron that "cleanly" corresponds to "evilness" that you can turn up/down?
There was a paper that popped up a month or two ago (I think published earlier though) about being able to inject "thoughts" into the neurons. E.g. Ask it what I should eat, and steer it towards "bread".
kridsdale1 2 hours ago [-]
Well I think the first big publication on this front was Anthropic’s Golden Gate Bridge neuron.
frotaur 1 hours ago [-]
This was not a neuron. This was from a trained sparse auto encoder.
In a few words, they found a way to map residual stream activations (so, a vector) into a few interpretable sparse concepts. They also had the converse maps, that translate concepts into activations.
Golden Gate claude was obtained by artificially increasing the activations in the direction of the 'Golden Gate' concept. It was not neuron surgery.
krackers 1 hours ago [-]
You don't even need SAE for residual stream based steering, you can do so much more naively by just taking the average difference in activations of prompted vs. unprompted versions, then PCA to get a "control vector" that you can use to elicit the prompted behavior
They implemented wasm in transformer with cpu decoding for the wasm part.
jazzyjackson 4 hours ago [-]
> When Qwen 'saves' a file, the compiled layers hash its name into one of 1,024 slots in a directory kept in the DeltaNet state, and store the text itself in other heads, with eight tokens per row, chained together
I’ve never wanted to learn how LLMs work more than reading this. Fascinating.
Does divagr get paid to do wild things with LLMs? Just curious if they’re being compensated for these wild experiments
Tuna-Fish 54 minutes ago [-]
Learning the basics of how LLMs work is legitimately doable in a weekend if you have ever passed a linear algebra course.
If you are actually good at linear algebra, it's doable in an afternoon.
They are fantastically simple in structure, all the intelligence is in the massive piles of data in them, not their structure. Then there is of course a lot of room in optimization once the basics are done.
I'd be very interested in how a scratch trained model that started training with this neural circuitry would behave. Especially the storage space.
horacemorace 4 hours ago [-]
Along these lines I’m curious how a scratch trained model with a few built-in tools (memory, calculator, ?) available during a single generation would behave. Not tool calls as turns.
low_tech_punk 4 hours ago [-]
a single generation would still allow looping in the "CPU" because the instruction set can do it. The model is almost a Turing machine (except for the limit on memory)
qwertyforce 1 hours ago [-]
As I understand, the problem of continual learning is that if you continue training, it will "overwrite" previous knowledge and this CPU will be lost
im3w1l 32 minutes ago [-]
There is nothing in theory preventing you from training while keeping a subset of weights locked.
LoganDark 27 minutes ago [-]
How do you ensure that subset of the weights will continue to be accessible through the others?
im3w1l 22 minutes ago [-]
I don't understand your question.
nghiemphucminh7 49 minutes ago [-]
[flagged]
andai 3 hours ago [-]
> Now, here's where our fun experiment starts. I found out that by setting the gate to 1,1 the write strength (beta) to 1,2 and using one-hot key encoding,3 every write we perform will overwrite exactly one row, and every one-hot query can retrieve an exact value. This means that the GDN layers themselves can work exactly as RAM, at a far lower cost than using normal attention as RAM, since a GDN head's state stays the same size however long the conversation gets, unlike attention's KV cache.
So like... can't you use a computer as a computer without doing all that roundabout stuff? Like if you want to integrate a LLM with hard computation there's probably a more efficient way to do that.
Or is this building towards "but can it run DOOM?" :)
rao-v 3 hours ago [-]
I’ve always wondered why we don’t add a few frozen “layers” to models that perform complex computations 2/3rds deep in the model. Some set of steerable fft + some common vector math operations could be reusable units of capability that the model learns to leverage during training.
Given that models learn to use residual connections optimally, it can’t be hard for them to steer around the fixed computation layer when it’s not needed.
I’m sure giant models invest a few layers in reproducing these functions, but surely smaller models can benefit?
Wow! I wonder if it can start programming itself soon.
Ericson2314 2 hours ago [-]
The real question is, do current training methods allow one to learn this when training on enough, say, assembly, or can the weights only be achieved via hard-coding?
A fun and important thought experiment about the the space of nerual networks vs the image of current training methods.
Most discourse conflates neural networks and trainable neural networks.
bronlund 3 hours ago [-]
No, I don’t want you to use the computer, I want you to be the whole computer. No mistakes.
blurbleblurble 2 hours ago [-]
This is cool but it'd be way more useful if they had made a GPU, seeing as how expensive those have gotten
incognition 2 hours ago [-]
Uh here the gpu runs the cpu but you want the gpu to run a gpu? Or cpu emulation of Gpu?
there's no free lunch here ever hence the cursed
ydj 2 hours ago [-]
I dare say that’s the joke.
yaoke259 3 hours ago [-]
It could be interesting to experiment with more complex representations in the memory/file system, i.e. vectors. Seems like the full advantage of this architecture is yet to be explored. Blending neurons with deterministic compute to fit on one GPU is a fascination concept, never considered it before!
lacoolj 2 hours ago [-]
I 100% see the potential here and am dying to see it in 30 days
Or maybe ill join your discord and hopefully we can riff it out to get there faster
jonrouach 2 hours ago [-]
i wonder if you could implement "authenticated token authorship" and self-verified running with this.
the model verifies an input as signed by an authorized requester, and signs its own outputs. eventually it also authenticates its own running via e.g. snarks? :)
there's lots of meta available with these toys..
farhanhubble 4 hours ago [-]
This is cool and I understand the RAM emulation with gated attention but how does the CPU emulation wor? You add layers that are then trained to predict outputs while interpreting RISC instructions?
bee_rider 4 hours ago [-]
I think they describe the CPU using the weights? I mean in some sense a CPU can be viewed as just a function, right?
anon291 3 hours ago [-]
At each cycle, a cpu is a simple function from the previous state to the next state.
zardo 3 hours ago [-]
How much VRAM do I need to run DOOM?
voxelghost 3 hours ago [-]
Makes me want to run agents on it
dools 3 hours ago [-]
Have you any idea how it feels to be a Qwenputer living in a Qwenbot's Qwenputer's world?
rumwerfer 3 hours ago [-]
But can it run Doom?
0x20cowboy 1 hours ago [-]
This is very clever. An inversion of tools. Awesome.
3 hours ago [-]
seba_dos1 3 hours ago [-]
Now run LLM inference on it.
deepsquirrelnet 3 hours ago [-]
butterfly Is this AGI?
serf 3 hours ago [-]
this is the sensation I get when I run nested VMs.
fr2029r 9 minutes ago [-]
[dead]
favori995749721 2 hours ago [-]
[flagged]
hujun236 3 hours ago [-]
[flagged]
cinderwick 4 hours ago [-]
[flagged]
rickcarlino 4 hours ago [-]
If anyone else has an easier way to emulate RISC-V I’d like to hear it. /S
gigel82 4 hours ago [-]
But... why? </s>
This reminds me of C++ Template Metaprogramming (TMP) (it's Turing-complete so people wrote "games" that run at compile time). The same with TypeScript types, I think someone got DOOM running entirely in the TypeScript type system. An interesting tech demo but entirely superfluous.
4 hours ago [-]
ansk 3 hours ago [-]
[flagged]
fudged71 2 hours ago [-]
Be more specific with criticism
ansk 2 hours ago [-]
This perspective -- that the burden of proof is on the experts to disprove the slop -- is exactly what I think is unsustainable.
rvz 4 hours ago [-]
So this is yet another rube goldberg machine.
We have gone down this path already in [0] and this is the same thing but overly complicated. This reason is "because LLMs".
It is a great example to refute the common misconception that LLMs are inherently non-deterministic. I see people repeating this a lot and being able to point at a CPU implemented within one is a very strong counter-example.
rvz 2 hours ago [-]
> It is a great example to refute the common misconception that LLMs are inherently non-deterministic.
No it isn't.
Realistically the opposite is true in practice and LLMs are really non-deterministic beyond simplistic experiments such as this in useful activities in production. Even with hacks such as "setting temp=0" (which no-one seriously does) to "achieve determinism", was one of the most common misconceptions ever propagated by "AI engineers".
So using tiny slop experiments aren't really useful as a "strong counter-example". In fact this is a very weak one.
ram_rattle 4 hours ago [-]
Cool cool cool
crossgrainlab 1 hours ago [-]
such a nice page. The font and the background colour are comfortable for my eyes to read the context.
Rendered at 05:23:31 GMT+0000 (Coordinated Universal Time) with Vercel.
> Because a CPU inside a language model clearly wasn't cursed enough, I decided the prudent thing would be to give it a filesystem too.
Love the spirit :)
Or the cli version if preferable: https://nicobrenner.github.io/lossy-git/
Not only can it solve the halting problem, but also the ignition problem!
[1]: https://github.com/anadim/AdderBoard [2]: hn discussion https://news.ycombinator.com/item?id=47170030&utm_source=per...
Anyways, what I’m trying to say is I want evil qwen to be my CPU.
I wonder how this squares with the recent "NNs are already clean" interpretability finding, which may have also been the impetus for this kind of research. Similar to their work on CNNs, maybe residual-steering is overkill, there might be just a single neuron that "cleanly" corresponds to "evilness" that you can turn up/down?
In a few words, they found a way to map residual stream activations (so, a vector) into a few interpretable sparse concepts. They also had the converse maps, that translate concepts into activations.
Golden Gate claude was obtained by artificially increasing the activations in the direction of the 'Golden Gate' concept. It was not neuron surgery.
https://vgel.me/posts/representation-engineering/
They implemented wasm in transformer with cpu decoding for the wasm part.
I’ve never wanted to learn how LLMs work more than reading this. Fascinating.
Does divagr get paid to do wild things with LLMs? Just curious if they’re being compensated for these wild experiments
If you are actually good at linear algebra, it's doable in an afternoon.
They are fantastically simple in structure, all the intelligence is in the massive piles of data in them, not their structure. Then there is of course a lot of room in optimization once the basics are done.
If interested, maybe start here: https://www.deeplearning.ai/courses/how-transformer-llms-wor...
So like... can't you use a computer as a computer without doing all that roundabout stuff? Like if you want to integrate a LLM with hard computation there's probably a more efficient way to do that.
Or is this building towards "but can it run DOOM?" :)
Given that models learn to use residual connections optimally, it can’t be hard for them to steer around the fixed computation layer when it’s not needed.
I’m sure giant models invest a few layers in reproducing these functions, but surely smaller models can benefit?
A fun and important thought experiment about the the space of nerual networks vs the image of current training methods.
Most discourse conflates neural networks and trainable neural networks.
there's no free lunch here ever hence the cursed
Or maybe ill join your discord and hopefully we can riff it out to get there faster
This reminds me of C++ Template Metaprogramming (TMP) (it's Turing-complete so people wrote "games" that run at compile time). The same with TypeScript types, I think someone got DOOM running entirely in the TypeScript type system. An interesting tech demo but entirely superfluous.
We have gone down this path already in [0] and this is the same thing but overly complicated. This reason is "because LLMs".
Why?
[0] https://arxiv.org/abs/1410.5401
No it isn't.
Realistically the opposite is true in practice and LLMs are really non-deterministic beyond simplistic experiments such as this in useful activities in production. Even with hacks such as "setting temp=0" (which no-one seriously does) to "achieve determinism", was one of the most common misconceptions ever propagated by "AI engineers".
So using tiny slop experiments aren't really useful as a "strong counter-example". In fact this is a very weak one.