NHacker Next
  • new
  • past
  • show
  • ask
  • show
  • jobs
  • submit
WebAssembly (WASM) arch support for the Linux kernel (github.com)
s-macke 19 hours ago [-]
That’s fast. Buggy, but fast. I’m totally impressed! Especially because I researched the necessary steps to do the same thing 10 years ago based on [0]. The patches required for this hack touch LLVM, libc, Linux kernel, BusyBox, ... and total approximately 15,000 lines of code.

I ran a small performance test with 'bc -lq' and compared with [0]:

  scale=1000
  4*a(1)
This WASM architecture compilation completely blows away my old emulation setup, which only managed around 200 MIPS. Maybe this approach can be generalized. Running a full Linux distribution at near-native speed right in the browser would be awesome.

[0] https://github.com/s-macke/jor1k

p0w3n3d 7 minutes ago [-]
nice benchmark. comparing to fabrice bellard's jslinux (https://bellard.org/jslinux/) it's roughly 20x faster (if arm on arm) and 64x faster (if x86 on arm)
Imustaskforhelp 12 hours ago [-]
Your project was also really nice to play around with. I think it was one of the few which actually had an interesting idea including (blink), (copy.sh)

I generally preferred copy.sh more to be really honest. I have actually used it sometimes as a poor man's qemu. If I may ask, what are your thoughts on copy.sh as I found that its performance on busybox or (tinycore linux with gui) was so brilliant (the only downside was that the internet speed was abysmally slow, like for me really really slow.)

Y_Y 9 hours ago [-]
What results did your benchmark get?
s-macke 3 hours ago [-]
By a factor of about 170. But this is more of a micro benchmark that gives you a rough idea. It's not a definitive figure.
syrusakbary 2 hours ago [-]
Really impressive work. Would love to see it progress.

Some ways I can see it could improve:

  1. setjmp/longjmp could implemented via Wasm Exceptions (this is how we do it on WASIX) - no need to wait on stack switching proposal
  2. fork could work easily with asyncify (start/resume), per binary compiled
  3. JIT could work via dlopen/dlsym (compiling the Wasm and linking it), even with runtime patching (using memory spaces on tables and updating them as you go to newly compiled code).
In general, I recommend taking an inspiration from WASIX [1] for those things, as we have spend quite a bit of time to make things work as much as possible!

[1] https://wasix.org/

lalitmaganti 18 hours ago [-]
Tried running:

  ~ # du -h 
  (...)
  [Runner sh (2390656)]: Wasm crash: RuntimeError: operation does not support unaligned accesses
  [Main]: Stopping CPU 0
  [Main]: Stopping CPU 1
  [Main]: Stopping CPU 2
  Kernel panic - not syncing: Aiee, killing interrupt handler!
  [Runner sh (2390656)]: Kernel panic: Aiee, killing interrupt handle
> Due to a bug in LLVM's build system, building LLVM a second time fails when building runtimes (complaining that clang fails to build a simple test program). A workaround is to build it yet again (it works each other time, i.e. the 1st, 3rd, 5th etc. time).

I'm incredibly curious what this bug might be!

bionade24 13 hours ago [-]
Unrelated to this issue but I've had a race condition with Automake which while run oin 2-4 threads occured exactly every 2nd run. With -j48 it was obvious it's a race condition. No idea how cache invalidation works in the automake stack, but that must have caused it to fail exactly 50% of the time.
moi2388 13 hours ago [-]
I just yesterday read about obfuscated Trojan quines in compilers, and a good test being building the compiler twice and it being the same, and now I hear this.

Spooked me for a sec xD

edubart 18 hours ago [-]
This is cool because it avoids emulation. However I think it has many shortcomings today which could all be solved by emulating a real CPU architecture (e.g memory protection support, ecosystem with tooling and Linux distributions).

By the way I have developed a similar project, WebCM, a RISC-V emulator capable of running full Alpine Linux that can be embedded in the Web browser and can reach up to 500 MIPS for some users, which I think is pretty fast despite the emulation, you can try at https://edubart.github.io/webcm/. Booting is also fast, it always boots from scratch when you open the page, so you can boot fast even with emulation.

someone_jain_ 5 hours ago [-]
That indeed feel fast, awesome stuff!
emmelaich 9 hours ago [-]
That is excellent!
virajk_31 47 minutes ago [-]
This is great, I started working on this during the early days of WASM, however left coz of busy schedule... Happy to see someone making progress ;)
ktpsns 2 hours ago [-]
I think lightweight wasm OS runtimes can have real use cases for instance in "cloud terminals" (such as in Rancher or at hyperscalers) which currently just connect a web terminal to some k8s pod. Or in more snappy Jupyter scientific notebook ecosystems. But people have to meet a sweep spot of downloading <1MB runtime, realizing networking and enough tools within the VM.
westurner 3 days ago [-]
How does this compare to the c2w container2wasm approach?

container2wasm/container2wasm: https://github.com/container2wasm/container2wasm :

> container2wasm is a container-to-wasm image converter that enables to run the container on WASM.

> Converts a container to WASM with emulation by Bochs (for x86_64 containers), TinyEMU (for riscv64 containers) and QEMU.

> Runs on WASI runtimes (e.g. wasmtime, wamr, wasmer, wasmedge, wazero)

> Runs on browser

> x86_64, riscv64 or AArch64 containers are recommended.

/? container2wasm: https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...

ktock/vscode-container-wasm https://github.com/ktock/vscode-container-wasm :

> Containers on VSCode for the Web [ https://vscode.dev ]

ktock/vscode-container-wasm-gcc-example: https://github.com/ktock/vscode-container-wasm-gcc-example

JupyterLite works without install on Chromebooks.

JupyterLite still lacks a Terminal e.g. with BusyBox Ash in WASM, with a file system integrated with the Jupyter-xeus kernel file system.

This appears to load much more quickly than other Linux and I think even just bash in WASM demos I've seen.

mappu 3 days ago [-]
That requires an ISA emulation layer, this new implementation doesn't - here, every binary is compiled as wasm, and every child process runs as a new Wasm WebWorker, and the Kernel ABI is exposed as Wasm export functions.

Removing the ISA translation layer has the potential to be massively faster for full-system environments. At the expense of maybe some new bugs.

The performance should ultimately be similar to compiling your userspace application directly as Wasm, but you now get to take advantage of the full kernel ABI instead of just the minimal shims that Emscripten give you / whatever DOM glue you create yourself.

westurner 2 days ago [-]
One less layer of translation!

Shouldn't browser tabs and/or origins get their own SELinux contexts like all Android apps since Android 4.4, like container-selinux and openshift's k8s? https://news.ycombinator.com/item?id=45418918#45421242

uutils/coreutils, findutils, diffutils, and Toybox are written in Rust which IIRC has a cleaner compile to WASM: https://news.ycombinator.com/item?id=45495100

RustPython may for may not also have a faster loading time than CPython compiled to WASM, though there are already some patches to CPython for WASM.

Where are the tests for the post-patch bugs this finds? Are they're expected behaviors that are not yet in tests which specify?

seanw265 18 hours ago [-]
Very cool! I'm curious as to how it compares with WASIX in terms of both compatibility and performance.

Also tangentially related: I'd love to see a performant build of Node.js compatible with this runtime (or really any flavor of WASM), but I think you'd run into the same issues that I have with WASIX. Namely build headaches, JIT, and wasm(-in-wasm) support. I'd explore it myself but I've already sunk way more time than is reasonable on that endeavor.

hardwaresofton 16 hours ago [-]
Could this work with https://github.com/webassembly/wasi-libc ?

It seems like OP put together their own musl-based libc which is awesome, but being able to compile against WASI would open up a lot of possibilities.

This also reminds me of the recent thread on user-mode linux -- how easy it would be to compile to WASM was definitely on my mind.

koolala 14 hours ago [-]
This is crazy cool. 8,000 CPUs. I wonder if any types of programs would ever make 10k tasks in their normal runtime behavior.

"One important difference is that there is no way to suspend execution of a task. There is a way around this though: Linux supports up to 8k CPUs (or possibly more...). We can just spin up a new CPU dedicated to each user task (process/thread) and never preempt it. Each task is backed by a Web Worker, which is in practice backed by a thread in the host OS (through the WebAssembly implementation). "

VladVladikoff 5 hours ago [-]
So next step they run a browser in this Linux in WASM in the browser. And then run Doom in that browser using WASM.
philipwhiuk 19 hours ago [-]

   ~ # ping 8.8.8.8
   PING 8.8.8.8 (8.8.8.8): 56 data bytes
   ping: can't create raw socket: Function not implemented
   [Runner sh (18823808)]: Wasm crash: RuntimeError: memory access out of bounds
darn
kro 18 hours ago [-]
Afaik wasm cannot open network sockets.

The segfault is unfortunate though

s-macke 17 hours ago [-]
You can write a network device driver, which exports the network packages into JavaScript. The author already wrote a console device. So, not much of a deal.

https://github.com/joelseverin/linux-wasm/blob/master/patche...

seanw265 17 hours ago [-]
Doable for http and https, but if you're running it in a browser environment, you'll eventually run into issues with CORS and other protocols. To get around this you need a proxy server running elsewhere that exposes the lower layers of the network stack.
s-macke 17 hours ago [-]
This is exactly what [0] does. Try it out. If you know the IP you can even log in to another open browser window via telnet.

[0] https://github.com/s-macke/jor1k

seanw265 16 hours ago [-]
Aha! Now I see I'm talking to the expert on the topic ;) Thanks for the link. I'll check this out.
Imustaskforhelp 12 hours ago [-]
This is such a brilliant thing.

I am not a person involved in building anything like this but I am a person who frequently used copy.sh/v86 and was actually building a tinycore fork which added jujutsu so that people can run jujutsu to try it out in their browser. It was a project which made me understand so much more about building linux from scratch, what isos are, and everything.

One surprising thing was that I was able to make the iso actually have jujutsu and it could run on something like qemu but not on tinycore due to some minor issue

Basically I believe that some really interesting quick-tries of some software can happen if we can have customized linux with our apps directly in the browser

This is such an interesting project really and it feels very snappy to me.

I have a quick suggestion as someone trying to make a custom iso linux / binary apps just work on linux in browser: Can you please, please, create a docker image where you can give static applications as an input in the docker and it would automatically generate the html page with wasm or the wasm output containing that static application in the /usr/bin

You mention something like this this

docker run -it -name full-linux-wasm linux-wasm-contained:dev /linux-wasm/linux-wasm.sh all

I haven't read the project more but I do find it incredibly minimalist and I genuinely hope you can add the ability to add static/hopefully some day glibc as well but I am mentioning it because there are a lot of golang tools which are statically linked and it would be so interesting to running them in browser if possible via their binaries

Yes I know that they could probably run via compiling into wasm itself but that just felt so much messy to me on how I could provide other tools like busybox with it or how it would actually feel like a linux environment to test it out as an example

I have to admit, your project is really really cool. Good luck on this project! Starred, and have a nice day.

nilslice 12 hours ago [-]
Very cool.

If anyone's curious to see what's packed in here at a glance: https://modsurfer.dylibso.com/module?hash=3fa6b28252b0d72c82...

utopiah 20 hours ago [-]
How does it compare to https://xrsh.isvery.ninja (ignoring the XR aspect)?
shevy-java 20 hours ago [-]
Hopefully this will make WASM more popular. I tried to get into it but lack of documentation was already one reason to not invest too much; speed concerns mentioned by other bloggers also amplified this issue recently. For some reason WebAssembly is not really "breaking through" right now. Perhaps it is inertia, perhaps another reason.
whizzter 19 hours ago [-]
Wasm is used in a lot of nooks and crannies, apart from games, Figma already uses it in the core and Wasm-GC has just started to become viable so we will se a lot of server-side languages get better web support.

Using Wasm as an end-all system was never the main intention even if we're heading that way now thanks to all the work people has put in.

I'd say that it's probably used where it's made sense so far.

Ray20 19 hours ago [-]
I hope so too. Websites that load runtimes for various programming languages are too slim; they need to load the entire operating system, otherwise why do we need all these powerful home computers?.
charcircuit 16 hours ago [-]
There's already a problem of downloading 10s of megabytes of web assembly. We don't need to download gigabytes for a single page.
cogman10 15 hours ago [-]
Yes, but then how else could I run a docker container in the browser?
evanjrowley 3 days ago [-]
I hope the situation gets better for Firefox.
embedding-shape 20 hours ago [-]
What situation exactly? Tried the demo (https://joelseverin.github.io/linux-wasm/), seems to run fine. There isn't any benchmarking programs/scripts available inside of it, so can't really give out any numbers, but it doesn't seem to work worse than any other "Linux-in-a-browser-tab" I've tried earlier. Using a 5950x with Firefox on Linux 6.17.6-2 FWIW.
evanjrowley 19 hours ago [-]
Sorry, I should have been more clear in my comment. I was referring to the statement from the project about debug capabilities:

> I recommend Chromium-based browsers over Firefox, as the latter does not work very well when debugging Wasm projects of this size.

frizlab 20 hours ago [-]
killed by the fork bomb

    :(){ :|:& };:
Diederich 17 hours ago [-]
How did that look on the host system CPU/memory wise?
cogman10 15 hours ago [-]
Mine jumped up to ~3gb and then the vm crashed. It happened fast enough that I didn't really see the CPU spike too much. Firefox FTR
littlestymaar 13 hours ago [-]
The Gary Bernhardt prophecy is still alive.

(https://www.destroyallsoftware.com/talks/the-birth-and-death...)

u8080 16 hours ago [-]
~ # wget

[Runner sh (18815616)]: Wasm crash: RuntimeError: abort

Illegal instruction

acdbddh 12 hours ago [-]
when docker ;)
12 hours ago [-]
3 days ago [-]
jgoode19 18 hours ago [-]
[flagged]
17 hours ago [-]
OsrsNeedsf2P 18 hours ago [-]
Thanks, ChatGPT
throwaway031125 19 hours ago [-]
$ rm -rf /

and it's gone

s-macke 19 hours ago [-]
Not quite right. Try the following.

  echo *
  cd /proc
  echo *
  while read line; do echo $line; done < /proc/cpuinfo
The last line should work and print the entire file, but it seems there's a bug.
throwaway031125 14 hours ago [-]
Well, it should not surprise you that the virtual file systems of the kernel remain.
lxgr 15 hours ago [-]
Only on 127.0.0.1.
throwaway031125 14 hours ago [-]
Thank you, Captain Obvious.
iberator 20 hours ago [-]
What's the point ?! It's insane. Who on earth would use that?
phendrenad2 13 minutes ago [-]
To elicit surprised cries of delight from the HN readers, of course!
MomsAVoxell 18 hours ago [-]
These questions are the number two most important questions to ask, in software. The sanity/insanity part is not so relevant, but it is necessary to point out that, pretty much a huge percentage of software any of us uses on a daily basis, started off with someone having a random insanity, answering those two questions with a working binary, and thus setting the idea towards becoming normal and thus sane.

Soon enough, WASM may just well be the #1 platform upon which to run a Linux on a Desktop ..

iamnothere 19 hours ago [-]
I can think of a few uses quite easily:

- Testing a distro or specific software without downloading it

- Educational use (teaching Linux basics on Chromebooks etc)

- Bypassing restrictions on installing certain software

MBCook 12 hours ago [-]
Ah, thank you!

I think this is really cool but I was struggling to think of a way it could be useful. Your last two suggestions seem especially pertinent.

tracker1 17 hours ago [-]
Because someone can... While I don't see a practical use myself, beyond educational or experimental, that doesn't mean nobody else could, should or would.

In the end, it's kinda cool.

afavour 20 hours ago [-]
I don’t think it’s that big a stretch of the imagination to see how this could be used in smaller pieces than the entirety of Linux.
trollbridge 16 hours ago [-]
Yeah. Basically any code you can run on Linux you can now run on a browser with a lot less work.
lxgr 13 hours ago [-]
Realistically, with quite a bit more work (compared to e.g. v86), but at much higher performance.
kbelder 12 hours ago [-]
Like Lynx?
VladVladikoff 5 hours ago [-]
Yo dog
Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact
Rendered at 10:19:13 GMT+0000 (Coordinated Universal Time) with Vercel.