Well, TinyChat. Still mind-boggling. From the video description:
I built a small language model in Minecraft using no command blocks or datapacks!
The model has 5,087,280 parameters, trained in Python on the TinyChat dataset of basic English conversations. It has an embedding dimension of 240, vocabulary of 1920 tokens, and consists of 6 layers. The context window size is 64 tokens, which is enough for (very) short conversations. Most weights were quantized to 8 bits, although the embedding and LayerNorm weights are stored at 18 and 24 bits respectively. The quantized weights are linked below; they are split into hundreds of files corresponding to the separate sections of ROM in the build.
The build occupies a volume of 1020x260x1656 blocks. Due to its immense size, the Distant Horizons mod was used to capture footage of the whole build; this results in distant redstone components looking strange as they are being rendered at a lower level of detail.
It can produce a response in about 2 hours when the tick rate is increased using MCHPRS (Minecraft High Performance Redstone Server) to about 40,000x speed.
jweather 4 days ago [-]
It would take just over 9 years for a response at the normal redstone tick rate... sounds about right.
throwup238 1 days ago [-]
The Minecraft AI Challenge: can your LLM company reach profitability before Minecraft can predict a token?
athrowaway3z 23 hours ago [-]
This can be achieved today.
Microsoft owns Minecraft; so they'll just swirl the numbers together with Minecraft sales to show their AI is already profitable.
rightbyte 22 hours ago [-]
> no command blocks
Oh. I thought this would be some cheesy command block curl to Chat GPT.
lhamil64 20 hours ago [-]
As far as I'm aware there's no way to do an HTTP request via command blocks, at least in vanilla.
datadrivenangel 19 hours ago [-]
Through Lua, all things are possible
charcircuit 16 hours ago [-]
The scripting language for Minecraft is Typescript. Lua would require a 3rd party mod.
>The @minecraft/server-net module contains types for executing HTTP-based requests. This module can only be used on Bedrock Dedicated Server. These APIs do not function within the Minecraft game client or within Minecraft Realms.
ianhawes 16 hours ago [-]
* In Bedrock Edition. Minecraft purists only use Java Edition.
sd9 21 hours ago [-]
I originally didn’t click on this post because it was obviously clickbait - the title isn’t possible.
But no. The author actually embedded a small LLM in Minecraft using hundreds of millions of blocks, that generates 1 token per 2h at 40,000x speed.
Bravo. I wouldn’t have even thought to try.
cosmic_quanta 19 hours ago [-]
One of my fondest memories was buying the book "The Elements of Computing Systems" by Nisan and Schocken, and implementing a 4-bit CPU in Minecraft.
4-bit is small enough that you can build it manually, without the use of external tools (which I don't think existed at the time anyways).
Highly recommended for children interesting in computing!
andrehacker 18 hours ago [-]
>> One of my fondest memories was buying the book "The Elements of Computing Systems"
>> by Nisan and Schocken, and implementing a 4-bit CPU in Minecraft.
You confused me there, the book doesn't cover Minecraft, you did that yourself after reading the book, got it.
The book is absolutely fantastic, it is the basis for the "From Nand to Tetris" courses:
https://www.nand2tetris.org/
I haven't digested it in full and with a title like that and the boring cover I always have to scramble to find it when I got a few minutes (What is that "Nand to Tetris" book called again?)
iterance 19 hours ago [-]
I'm glad I'm not the only one. Built a tape drive and bootloader to load programs off it & everything...
kqr 15 hours ago [-]
> (which I don't think existed at the time anyways)
This surprised me at first because I remember using Python to create Minecraft maps early on, or at least in the beta.
But it seems like redstone was added in the alpha, and the earliest commits of pymclevel (which I think I used) also dates back to the alpha. So there might indeed have been a time window of a few months in which redstone was available but not tooling for creating maps.
cosmic_quanta 15 hours ago [-]
I remember starting playing Minecraft Beta, that was around 2011 maybe?
Even if tools to programmatically create maps were available, I wouldn't have known how to program. So this is more about my lack of knowledge of tools of the time.
12 hours ago [-]
2OEH8eoCRo0 16 hours ago [-]
Funny, I bought that book and studied it because I heard about it in a Minecraft video. Was that you? Nand2tetris helped me land my first programming job.
cosmic_quanta 16 hours ago [-]
It was not me! Maybe we watched the same video though? I can't recall who or what gave me the idea
sram1337 12 hours ago [-]
Just wanted to say I think we all watched the same video! I remember doing this one summer in high school
- It had Jit backends before, but seems they have been removed
unleaded 20 hours ago [-]
Here is an earlier, longer video on someone explaining how they built a neural network to recognise handwritten digits in Minecraft: https://www.youtube.com/watch?v=DQ0lCm0J3PM
It should answer some of the questions/clear up some of the confusion raised here (e.g. how they get the weights in).
aubanel 4 days ago [-]
That title looked like clickbait until... Oh well they did actually did it
dude250711 1 days ago [-]
It was probably done by an individual.
zooi 23 hours ago [-]
"They" can also refer to an individual
dude250711 19 hours ago [-]
If they did refer to an individual Minecraft expert for an advice, then it counts as a team effort.
taneq 10 hours ago [-]
What if the individual Minecraft expert was themselves? I mean, having been singlehandedly responsible for team efforts in the past, I get it. :D
Jotalea 21 hours ago [-]
wait what?
burkaman 19 hours ago [-]
"Someone is at the door asking for you."
"What do they want?"
This is the standard way to refer to an unknown person in English, anything else sounds awkward. "What does the person want", "what does he or she want", "what does this 'someone' want", none of these will sound natural to a native speaker.
npteljes 21 hours ago [-]
It's called "singular they". It's used like: "We have a new joiner in our team! They became the talk of the town very fast". I love this feature of the English language, while a bit confusing at first, I think it works much better than "he/she" or "s/he" when talking about someone with an unspecified, unknown or unrelated gender.
OkayPhysicist 17 hours ago [-]
This really shouldn't be some surprise. You probably do it on a regular basis without noticing. Shakespeare uses it in Hamlet:
There's not a man I meet but doth salute me
As if I were their well-acquainted friend
English has always used the singular they, especially (but not always) when the gender of the target of the pronoun is unknown.
tempaccountabcd 15 hours ago [-]
ƿrong. "they" is norþmannisċ. It sċuld be he, forðat hē and hēo ƿuld'fe melded into he, and forðat Englisċ is an Indo-Europisċ tung, ƿere ƿerelie is ðe first kin.
It is a bit of a clickbait since they used commandblocks, not just redstone. But it's still impressive
AndrewDucker 24 hours ago [-]
From the video description:
"I built a small language model in Minecraft using no command blocks or datapacks!"
utf_8x 24 hours ago [-]
From the video description:
I built a small language model in Minecraft using no command blocks or datapacks!
nomilk 24 hours ago [-]
Can you explain the difference (for non-minecrafters)
thrance 24 hours ago [-]
Command blocks can only be obtained by cheating in normal gameplay, they are used to execute server commands automatically. Using them to build a computer in the game kind of defeats the purpose of the exercise, since instead of using the game's physics to build your device, you're now mostly doing scripting with minecraft commands. The author explicitly said they didn't use any in their build.
promiseofbeans 22 hours ago [-]
The confusion might come from the author using commands / external software to generate and assemble parts of the redstone machine. The final machine doesn’t use any command blocks as part of it’s operation, but the description is a bit ambiguous here
19 hours ago [-]
NSPG911 18 hours ago [-]
there were no clickbaits there at all. no command blocks were used at all. if you were so certain, why dont you download the world and try it yourself?
fuzzy_biscuit 21 hours ago [-]
I think you missed the 'no' in there. They did not use command blocks.
dangoodmanUT 19 hours ago [-]
Redstone computing theory: Everything technological breakthrough in computing will eventually be rebuilt in redstone
nonethewiser 19 hours ago [-]
quantum computing
redman25 19 hours ago [-]
I guess redstone still somewhat follows the physics of electricity. Maybe a quantum simulator?
sbuttgereit 17 hours ago [-]
Or just use Bedrock edition.
laszlokorte 23 hours ago [-]
Are these kind of projects actually build manually inside of minecraft block by block or is there some verilog/vhdl to minecraft-level compiler toolchains used?
baobun 21 hours ago [-]
They did not manually place hundreds of millions of blocks <:
buzzy_hacker 22 hours ago [-]
It’s been awhile since I’ve played Minecraft, but when I built large redstone projects before, I built out each circuit manually and then used mods to copy/paste it within the game.
squigz 22 hours ago [-]
And nowadays you can use Axiom, which is a much better experience than WorldEdit (WE still has its uses though!)
There are libraries for e.g. Python to make Minecraft maps entirely programmatically, as well as convenient tools for editing maps by hand.
jalk 22 hours ago [-]
Hoping the answer is: we used ChatGPT to create the build :)
TrackerFF 23 hours ago [-]
Is there some kind of building automation do this? Surely it would take forever to plan out, and actually build blocks like this by hand.
promiseofbeans 22 hours ago [-]
There are libraries that let you generate minecraft structure files programmatically [0]. These can then be pasted into the world.
You still need to come up with the circuitry for all the maths and figure out how to represent all the state with red stone, but once you get the little bits done it’s quite a quick process filling in weights and duplicating logic sections
Seems to be a routing problem. If you can place a digital design in an fpga, you can also do it in minecraft.
There are a few projects out there for vhdl and verilog to redstone.
buzzy_hacker 22 hours ago [-]
There are mods that let you copy/paste sections of blocks.
sbuttgereit 17 hours ago [-]
If you like thinking about the possibilities of the worlds of Minecraft and LLMs colliding I recommend the YouTube channel @EmergentGarden (https://www.youtube.com/@EmergentGarden) it's not exclusively agents and Minecraft, but there's a fair amount of stuff tested over time.
Invokes shades of Iain M. Banks' "Surface Detail" Culture novel, where virtual Hells are a major plot point.
mittermayr 21 hours ago [-]
Aside from the technical feat, which is amazing in and of itself, I was completely mesmerized by that video! So well done, incredibly entertaining and perfectly presented.
WillieCubed 8 hours ago [-]
I suppose it's now only a matter of time until someone trains an LLM in Minecraft, right?
alexc05 19 hours ago [-]
when people do these complex CPU designs in minecraft are they laying the blocks individually in real time and in 3d space - or are they scripting some sort of algorithm that instantiates the system in one go?
It's impressive either way but the manual version seems ... impossible.
jtokoph 19 hours ago [-]
There are external tools which essentially let you script the generation of a save file of the world.
derleyici 19 hours ago [-]
So, Minecraft is Turing complete?
OkayPhysicist 17 hours ago [-]
Any system that can represent a NAND gate is Turing complete. In Minecraft, connecting two inputs to the same branch of redstone wire is an OR gate, and a redstone torch on a block acts as a NOT gate. Two inputs, both inverted, into an OR gate gives a NAND.
kibwen 15 hours ago [-]
It's even slightly simpler than that, NOR is also functionally complete, so you can just negate the OR.
kaffekaka 18 hours ago [-]
Yes, Notch said somewhere that was by design.
ThrowawayTestr 17 hours ago [-]
Turing completeness is actually pretty easy to implement. The game Baba is You is also turing complete.
jcmontx 18 hours ago [-]
These people are insane, it has to be created programmatically, isn't it?
nom 18 hours ago [-]
I suspect a human life isn't long enough to place that many blocks.
jerf 17 hours ago [-]
"(1020 * 260 * 1656) seconds in years" [1] comes out to 14 years. Solid, of course, no breaks or sleep, no time for training or planning or debugging. And one block per second is a pretty good clip for a technical build. It doesn't take many multiplicative factors before it's more than one lifetime.
You would probably also need another lifetime just to fix all the errors once you are finished with laying out all the pieces...
16th_hop 16 hours ago [-]
Fascinating. How exactly is the model trained? I see this is an inference tool, is the model trained external to minecraft or in the game itself?
junon 13 hours ago [-]
Appears based on the video to be trained externally and then just the weights being added into the game.
__fst__ 12 hours ago [-]
Can't wait for an LLM implementation in the Game of Life too.
aeneas_ory 18 hours ago [-]
Can‘t wait to test out the world download, this is so cool and also scary how much time this must have taken! Did you build it with some schematic editor?
sim7c00 4 days ago [-]
tbh great vid also funny editing for some reason it looked so satisfying..the leap onto the enter key etc. besides well done on the project, presentation is classy :D looks a lot of fun!
runeblaze 24 hours ago [-]
Of course it seems to be the same person to have done minecraft inside minecraft using redstone :)). peak!! done it again
tshanmu 21 hours ago [-]
this reminds me of permutation city and the question of what is time...
waltbosz 21 hours ago [-]
It reminded me of the human computer in "Three Body Problem"
I would like to know more about how this was generated. I'm sure there must have been code to generate the blocks/structures.
Surely this wasn't all done in a block editor by hand?
There was some code shown at beginning. Was that placing the blocks to build each section?
And, more info on the system that could run this.
OkayPhysicist 17 hours ago [-]
It'd be possible to build something like this just with large-scale copy-paste, but there are compilers from code to redstone.
codergautam 22 hours ago [-]
The code at the beginning was for training the 5M model I believe
pcunite 15 hours ago [-]
And I did it over the weekend.
/s
zkmon 23 hours ago [-]
This is outrageous! I see people build some basic circuits and logic gates using red stone, but this is unimaginable. Is Minecraft becoming a digital clone of the real world?
bamboozled 23 hours ago [-]
I hope there are people like this working on genetic research for (currently) incurable diseases because that would kick ass.
Rendered at 08:55:54 GMT+0000 (Coordinated Universal Time) with Vercel.
Microsoft owns Minecraft; so they'll just swirl the numbers together with Minecraft sales to show their AI is already profitable.
Oh. I thought this would be some cheesy command block curl to Chat GPT.
https://learn.microsoft.com/en-us/minecraft/creator/scriptap...
>The @minecraft/server-net module contains types for executing HTTP-based requests. This module can only be used on Bedrock Dedicated Server. These APIs do not function within the Minecraft game client or within Minecraft Realms.
But no. The author actually embedded a small LLM in Minecraft using hundreds of millions of blocks, that generates 1 token per 2h at 40,000x speed.
Bravo. I wouldn’t have even thought to try.
4-bit is small enough that you can build it manually, without the use of external tools (which I don't think existed at the time anyways).
Highly recommended for children interesting in computing!
>> by Nisan and Schocken, and implementing a 4-bit CPU in Minecraft.
You confused me there, the book doesn't cover Minecraft, you did that yourself after reading the book, got it.
The book is absolutely fantastic, it is the basis for the "From Nand to Tetris" courses: https://www.nand2tetris.org/
I haven't digested it in full and with a title like that and the boring cover I always have to scramble to find it when I got a few minutes (What is that "Nand to Tetris" book called again?)
This surprised me at first because I remember using Python to create Minecraft maps early on, or at least in the beta.
But it seems like redstone was added in the alpha, and the earliest commits of pymclevel (which I think I used) also dates back to the alpha. So there might indeed have been a time window of a few months in which redstone was available but not tooling for creating maps.
Even if tools to programmatically create maps were available, I wouldn't have known how to program. So this is more about my lack of knowledge of tools of the time.
- Re-implements parts of Minecraft
- Runs 512x512 plots in different threads
- Compiles Redstone applying different kind of optimization passes https://github.com/MCHPR/MCHPRS/blob/master/docs/Redpiler.md
- It had Jit backends before, but seems they have been removed
It should answer some of the questions/clear up some of the confusion raised here (e.g. how they get the weights in).
"What do they want?"
This is the standard way to refer to an unknown person in English, anything else sounds awkward. "What does the person want", "what does he or she want", "what does this 'someone' want", none of these will sound natural to a native speaker.
There's not a man I meet but doth salute me As if I were their well-acquainted friend
English has always used the singular they, especially (but not always) when the gender of the target of the pronoun is unknown.
https://modrinth.com/mod/axiom
You still need to come up with the circuitry for all the maths and figure out how to represent all the state with red stone, but once you get the little bits done it’s quite a quick process filling in weights and duplicating logic sections
[0]: E.g. https://github.com/BenBenBenB/nbt-structure-utils
There are a few projects out there for vhdl and verilog to redstone.
Most recently: Can AI (actually) beat Minecraft? - https://www.youtube.com/watch?v=Wh4abvcUj8Q
https://www.youtube.com/watch?v=FCnQvdypW_I
Invokes shades of Iain M. Banks' "Surface Detail" Culture novel, where virtual Hells are a major plot point.
It's impressive either way but the manual version seems ... impossible.
[1]: https://www.google.com/search?q=%281020+*+260+*+1656%29+seco...
Bravo to you, sir.
Surely this wasn't all done in a block editor by hand?
There was some code shown at beginning. Was that placing the blocks to build each section?
And, more info on the system that could run this.