One great thing about this book is the 'stuff I didn't do' part.
Layout is really hard. Just tables by themselves are hard, even without any css around them. CSS makes layout impossibly difficult. I challenge anyone to keep the whole CSS spec and its associated behaviors in their head.
At this point css + html + javascript have become a dynamic PDL, and probably is one of the most complex pieces of software today.
As an aside, video decoding is offloaded onto hardware, so it's not as battery intensive as it used to be.
btown 62 days ago [-]
For the absolutely massive amount of code one needs to implement for production-grade CSS layout, the Servo source code is illustrative and IMO quite cool to see. For instance, this file just implements block and inline contexts; there's a bit of Rust boilerplate here, but the vast majority of lines are "business logic" around various parts of the specification. And there's a whole folder of these. https://github.com/servo/servo/blob/main/components/layout/f...
But implementing a layout engine is doable. CSS is not magic; there's a spec that can be (meticulously) transformed into code. I've occasionally showed code like this to people frustrated that CSS seems arbitrary, just to show them that there is a logic to the execution environment. Granted, you're not going to regularly click into it the way you'd click into the implementation of a library, but it's no different from something like React in that regard. I think it helps!
tannhaeuser 62 days ago [-]
FWIW, Pavel, one of the authors, has devoted considerable time into what is one of the very, very few attempts at a formal specification for CSS (the static/float layout fragment cf [1]). It's a Racket program generating Z3 SMT solver code for verifying an instance layout (which also looks like Scheme) so it's not for the faint-hearted ;) but maybe just what an FP fan on HN is looking for as a challenge.
Wow, thanks, you always suspect no one has actually read the papers :) That was a crazy project... I eventually got it passing almost all of the WPT css2 fragment.
I'm still working on CSS layout, with hopefully another paper coming soon.
bloopernova 62 days ago [-]
For what it's worth, I'm just a devops person and I found that article on How CSS Floats Work to be very understandable :) Thank you for writing all this great stuff!
userbinator 61 days ago [-]
Since you're still working on it, have you considered this edge-case with floats: if a float is added long after the last breakpoint, and the current line grows long enough that it (with the float) reaches the right edge, do you take that breakpoint (and thus the float that came after moves to the next line due to the rule that a float cannot be put higher than its occurrence in the flow), or "defer" the float to the next line, freeing more space on the current line? All the browsers I've tested recently seem to do the former, but if you read the CSS rules for floats carefully, the rule that says floats are to be placed as high as possible suggests that deferring the current float, and thus possibly allowing more content on the current line, may result in future floats being higher than if the current float was not deferred, despite the current float ending up at the same height.
tannhaeuser 62 days ago [-]
In that case, you've got at least one avid reader ;)
pavpanchekha 62 days ago [-]
Yes, layout is difficult, especially because (I think):
1. The most "core" parts of layout, like CSS 2 stuff, is pretty poorly considered with a bunch of weird features that interact in strange ways. (Floats and clearance? Margin collapsing?) Some parts of this "core" were intended to be universal even though they're a bad fit for other layout modes. (Margin and padding, for example, don't have a clear purpose for say grid elements.)
2. It's not well-modularized the way JS APIs are. A JS API can often be implemented fairly stand-alone, but each layout module interacts with every other layout module since they can be nested in various ways. I think newer specs like grid are trying to be stricter with this but there are fundamental challenges: the actual 2D screen is a shared resource that different layout modes must split up.
fouric 62 days ago [-]
Layout is so difficult that it made me quit using Common Lisp and ncurses to build my passion project and become the very thing I swore to destroy (a React developer).
I can't be the only one who wants a simpler layout language than CSS that's designed with two decades of hindsight to provide the maximum simplicity-expressiveness product. Are there any serious projects to engineer something like this, or has everyone given up and either embraced CSS3 (waiting for the LLVM backend) or gone back to plain text?
pavpanchekha 62 days ago [-]
Author here, and I also teach web dev, including CSS, at the University of Utah (including this semester). Newer parts of CSS, like flex-box layout are both simple and powerful. Just use those! I think it's important to start thinking about learning all of the Web Platform like you'd think about learning all of the Windows APIs or all of the Linux system calls or all of your favorite programming language's features. People rarely do! (I have 15 years of Python experience, and I do not understand metaclasses or async.) There are lots of weird obscure corners, but you don't need to know those to build websites.
fouric 60 days ago [-]
Thanks for the response!
Are there any resources on learning to design simpler layout systems (like flexbox + any other important parts) without having to adjust the design to compensate for older systems (e.g. if you were to try to implement CSS).
meindnoch 62 days ago [-]
Constraint-based layouts. The world's most sophisticated UI system uses that (Apple's UIKit).
amelius 62 days ago [-]
I was playing with Solvespace a few weeks ago, and the thought occurred to me that the constraint-based modeling approach is exactly what I want in a layout system, and it extends to 3d even. We're stuck with CSS for now, but this must be the future.
LLVM backend for CSS3? (This must a joke, right??)
intelVISA 62 days ago [-]
;)
DanielHB 62 days ago [-]
For React Native the facebook engineers just gave up and were like "all you get is flexbox layout" and people were quite okay with that (although some people grumble about lack of display grid)
It works great for small devices, but I prefer ios constraint layout (and I think android got it too). No need for spacers.
erikpukinskis 58 days ago [-]
When do you need spacers in flexbox?
vbezhenar 62 days ago [-]
This Babylonian tower will crumble one day.
Layout does not have to be so complex. There are dozens of GUI frameworks with simpler layout system. Those are enough for applications everyone uses.
PaulDavisThe1st 62 days ago [-]
Actually, almost every GUI toolkit's scheme for layout has issues, and none of them are perfect.
The ones that use absolute pixel positioning fail when using different resolution displays.
The ones that use box packing fail when you need to deal with different sized displays.
The ones that use constraint programming fail when you need to layout hundreds or thousands of widgets.
CSS-style layout has its own pros and cons, but there is no alternative to it that is clearly better under all circumstances. If you doing layout and want to be resolution-independent, function on everything from phones to giant displays and have thousands of things to layout, CSS is actually likely better than any alternative.
peoplefromibiza 61 days ago [-]
possibly, but videogames have been doing layout for ages without CSS and they can vary a lot from game to game while most desktop applications have basically the same 2-3 layouts: input and edit some text, fill in some form, display some data in a table, a tree or a chart
PaulDavisThe1st 60 days ago [-]
DAWs (digital audio workstations)
Video editors.
Image editors.
Web browsers.
Circuit, interior and architectural design.
None of these are anything like your "most desktop applications".
peoplefromibiza 52 days ago [-]
Just to be clear: when you name niche desktop applications you are literally saying that they are not "most of them"
BTW all of those applications you mentioned use many of the same techniques video games use (for the same reason: custom layouts), existed long before the web and are doing just fine without CSS
psd1 61 days ago [-]
Games lay out one game.
Document readers lay out every possible document.
peoplefromibiza 61 days ago [-]
> Document readers lay out every possible document.
you mean like vector graphic software, DTPs, word processors etc etc?
Flash did resolution independent graphics and layouts on the web back in the late '90s, we killed it - not without reasons - but we still haven't created a decent alternative to it
mardifoufs 62 days ago [-]
And they all have massive issues, or just provide a worse version of CSS (QT's qss, for example, as it's just a less well documented, non standard and very sparsely talked about CSS implementation. Oh and it doesn't work for everything in QT)
msie 62 days ago [-]
Pixel positioning is so nice! I remember how easy it was to layout UIs with VB.
echoangle 62 days ago [-]
How does that work when someone resizes a window? I'm genuinely asking because pixel positioning to me sounds like hardcoding positions. Do you have to manually code the reflow when the window becomes narrow for example?
vbezhenar 61 days ago [-]
Pixel positioning usually anchors to a specific side. Like you can anchor label to the right side and it'll move as expected. And you also need to specify minimum width/height to keep thing sane.
Although it works poorly with translations, so it's probably not the best approach, but certainly one of the simplest. If you don't need translations, it works surprisingly well. Basically all Delphi applications used it and there were used a lot in private companies.
maxwell 62 days ago [-]
Which do you recommend?
vbezhenar 60 days ago [-]
I've used Java Swing, Cocoa, Qt, Android. They are pretty good and were enough to build any kind of complex UI that I needed. I'm not sure what's the best one in the world, but generally I've found most GUI layout managers pretty good.
Even basic tools like those which were found in old Delphi, with anchoring components to the parent side, were sufficient for many UI-s, although some tasks like internalisation could case some pain.
In the end my opinion is that there should be a balance between usability and ease of implementation. Especially for technologies which are supposed to have many implementations.
khana 62 days ago [-]
[dead]
throwup238 62 days ago [-]
Modern CSS implementations are full blown geometric constraint solvers now. The only things approaching their algorithmic complexity are now other geometric constraint solvers like CAD kernels and silicon layout software.
hinkley 62 days ago [-]
I did layout for a feature phone browser. WAP and mercifully XHTML-basic’s ideas of layout was simple enough that I could treat spans as a concave octagon - a rectangle with a bite out of the upper left and lower right corner. It made it at least an order of magnitude faster to paint and scroll a long document, and much simpler to think about.
In the years since I’ve used a lot of tricks for web app CSS that I’m not sure I’m smart enough to figure out. But then I’ve never thought as long and as hard about typesetting as I did at the time so who knows.
userbinator 62 days ago [-]
Tables are trivial in comparison to floats. If you think the latter are easy --- there are many (literal) edge-cases to consider.
pstrateman 62 days ago [-]
> As an aside, video decoding is offloaded onto hardware, so it's not as battery intensive as it used to be.
This is technically but not usefully true with most videos on the web today.
The video decode itself is accelerated, but each frame passes through JavaScript to be composited.
The only time video is fully hardware decoded is when it's a simple video element to a static video file.
pjc50 62 days ago [-]
> The video decode itself is accelerated, but each frame passes through JavaScript to be composited
I don't think that's true, and it's even less true once DRM video is involved - it becomes very difficult to get other software on the machine to even see the video, at least on Windows. You can very occasionally see bugs where the hardware accelerated playback ends up in a different place to where the browser thinks the video should have been put, too.
What does happen is the video data gets reassembled in Javascript (e.g. Video.js) because the native player doesn't support HLS. Not quite the same thing. It's just reformatting MPEG-TS to the similar but not identical MP4. Oddly, the browser in my LG TV does play HLS video natively, and I think Safari does?
incrudible 62 days ago [-]
> each frame passes through JavaScript to be composited
What do you mean by that? There is no Javascript doing the actual compositing, and the actual compositing is (usually) hardware accelerated.
afavour 62 days ago [-]
> not usefully true with most videos on the web today
> The only time video is fully hardware decoded is when it's a simple video element to a static video file.
These seem in disagreement to me. The vast majority of videos on the web are simple video elements going to static video files. It is not usual for each frame to pass through JavaScript before being displayed.
pstrateman 62 days ago [-]
Most of the things you think of as static video files (say youtube) are actually video segments operating more or less exactly the way live streams work.
afavour 61 days ago [-]
Yes, I know. But the frames still aren't passed through JavaScript.
Sesse__ 62 days ago [-]
The decoded frames do not pass through JavaScript. The compressed data may (e.g. if you are using the Media Source API).
pavpanchekha 62 days ago [-]
I think by "JavaScript" here you mean rendering—that's partially true. In macOS and Windows these days (also I think Linux with GTK4 on Wayland, though only in a limited way), the window manager is itself composited and a window can send a small display list to that window manager for it to composite. In that case, it's possible to actually have the video decoding to happen entirely in hardware and never have the browser directly interact with decoded video bits. That said usually the window manager compositor is pretty limited and the browser will only do this when the stars align. The sort of things that can break it are any kind of weird clipping, transparency, or effects applied to the videos.
paulddraper 62 days ago [-]
> I challenge anyone to keep the whole CSS spec and its associated behaviors in their head.
Lol, no way.
People are always "guess what JS does, wut."
Doesn't hold a candle to Cascading Stylesheets.
sylware 62 days ago [-]
yep, namely noscript/basic (x)html.
jm4 62 days ago [-]
This looks awesome. About 15 years ago, I started working on a headless browser and maintained it for several years. It used SpiderMonkey as the js interpreter and had a custom DOM implementation. It ran all the modern js from the time, AJAX, etc. Later, I added a custom Flash runtime. It basically did everything but draw to the screen. That project was a lot of fun.
I'm definitely interested in going through this book.
aitchnyu 62 days ago [-]
Umm, if you wanted/want to draw to the screen, what library will you use?
pavpanchekha 62 days ago [-]
The book uses Tk (via Python's tkinter library) for the first 10 chapters and then switches to Skia, which is used by maybe all of the browsers now (I believe Webkit on Linux just switched to it from Cairo). It seems to be by far the most common high-performance 2D graphics library.
skeeterbug 62 days ago [-]
I believe Chrome uses Skia
lesuorac 62 days ago [-]
Yes [4].
> [1] The library is used as of 2023 in Google Chrome, ChromeOS, ChromiumOS, Mozilla Firefox, Mozilla Thunderbird, Android, Firefox OS, Flutter,[5] Avalonia (from Alpha 4), LibreOffice (from version 7.0) and RAD Studio[6](since version 12.0).
> [2] Changes to the Skia repository will be rolled into Chromium by the AutoRoll bot several times per day.
> [3] It serves as the graphics engine for Google Chrome and ChromeOS, Android, Flutter, and many other products.
Whoa. Somehow I have not heard of this. Can this be used to make cross-platform GUI apps?
Rohansi 62 days ago [-]
Sure you can, it's a 2D graphics library. It's more like the JS Canvas API though instead of a UI framework.
PaulDavisThe1st 62 days ago [-]
Which rather importantly means that you still need to find something else to do:
* layout
* event handling
which are not exactly trivial for a "real" application (whatever that means).
jm4 62 days ago [-]
I didn’t have a clue. That wasn’t part of my skillset. All we needed was a headless browser that was automated. It was crawling a few million pages a day. I had a debug console where I could see cached pages, headers, cookies, etc.
currygen 62 days ago [-]
It's refreshing that browser engineering seems to become a "trend" now. The ecosystem is quite sparse with basically only Google, Apple and Mozilla defining it. I'd like to see forward into a future with more independent browser engines.
mike_hearn 62 days ago [-]
I don't think it's worth trying to write a rendering engine for HTML. You will never finish - HTML is a spec fully owned by Google and Apple at this point and it's just too complex to implement from scratch.
The interesting space is really post-HTML UI/document tech. There's another thread running about Typst which is a sort of better LaTeX. Markdown was highly impactful. There's a lot of scope for people to do interesting things in this space that are "HTML but better". It doesn't even have to be a markup format - Typst and React HTML both blur the lines between code and data. Jetpack Compose shows how to use Kotlin's DSL features to make something that looks a bit like a UI description but which is actually code.
Of course it means you have to then either distribute a 'browser' for your format, or find a way to display it in the browser. But compiling down to some JS/HTML/WASM thing is certainly possible. You can also use portable GUI toolkits like JavaFX; that also gives you accessibility. Or do both!
Once you define your own UI language there's a lot of scope to try things that HTML doesn't do well. An obvious one is separation of content and style. HTML tried and never really got there. XSL:T tried harder but was a weird pure functional language with XML as its syntax. React does quite well with going JSON->boxes but the underlying protocols are always ad-hoc and tacked on, so you can't really write useful tooling on top of that.
Another idea would be a format that's natively immune to XSS.
berkes 62 days ago [-]
> I don't think it's worth trying to write a rendering engine for HTML. You will never finish - HTML is a spec fully owned by Google and Apple at this point and it's just too complex to implement from scratch.
This keeps being repeated. But it leans on three false assumptions.
- That is has to be "finished" at all. For many use-cases, a subset (of a subset) might just be fine. The screen in my refrigerator, or the information display in a train, might want to render some HTML, but when the HTML is controlled and constrained, there's no need for "everything".
- That is has to adhere to "the spec". See above, but also if the HTML+CSS+JS is less controlled, quite a few use-cases it's fine to ignore lots of the quirks or even large parts of the specs. Even Chrome and FF don't implement "all", whatever "the spec" might be in the first place. But a browser in a TV set-top box, my e-reader, some dedicated wikipedia-device, or the "help section of an app" are fine if they break on complex sites.
- That is must be implemented from scratch. Even if you forego the big rendering engines, JS VMs and so forth, there's a lot of libs that do DOM handling, CSS parsing, JS runtime etc. There's a lot of shoulders to stand on, aside from "just run chrome headless".
By repeating this mantra that its not worth "building a new browser" or "rendering engine", we only cement the status quo further. And promote the idea that your car, refrigerator, test-runner, help-section, dashboard, e-reader and whatnot must run either a full chrome or firefox. We stiffle innovation.
mike_hearn 61 days ago [-]
Sure, if you're willing to write a renderer for something that looks like HTML but isn't then that's a lot more tractable. But at that point why not just do something better than HTML? Most of the effort is in the engine anyway.
berkes 61 days ago [-]
> something that looks like HTML
This is surprisingly easy. Because the "this is HTML" is both vague and limited. HTML5 is remarkably vague and permissive.
Also, HTML has nothing to do with CSS nor with JS, nor with any DOM apis and not even with how stuff has to be rendered or if it has to be rendered at all.
There are numerous libraries and projects that can parse the WHATWG version of HTML. I'm not saying these are trivial or simple. But far simpler than a full rendering engine will ever be.
btown 62 days ago [-]
Another thing you can feasibly do is implement flexbox or a similar useful subset of layout! https://www.yogalayout.dev/ is one such library that powers React Native. Letting people bring CSS intuition when writing greenfield code for a simpler engine can be a great way to onboard users.
niutech 62 days ago [-]
> I don't think it's worth trying to write a rendering engine for HTML. You will never finish - HTML is a spec fully owned by Google and Apple at this point and it's just too complex to implement from scratch.
Andreas Kling, the author of Ladybird browser, proves otherwise.
Andreas hasn't finished. Also, I question to what extent it's worth it.
UniverseHacker 62 days ago [-]
I was an early user of KDE back in 2000 and thought they were absolutely insane for trying to write their own web browser engine when it was controlled by Microsoft and Netscape. The web was just too complex and nothing worked in it, there was just no way a tiny open source project like that could make any useful headway with browser technology.
Of course, jump forward 24 years and the KDE browser engine is basically the only game in town- the basis of both Chrome and Safari. Absolutely no way I saw that coming.
7crow 62 days ago [-]
> You will never finish - HTML is a spec fully owned by Google and Apple at this point and it's just too complex to implement from scratch.
sounds like a skill issue
01HNNWZ0MV43FF 62 days ago [-]
Something that uses less RAM would be nice. Other than that and the spyware from Capital-G Google Chrome and Capital-M Mozilla Firefox, I don't have a problem with it being sparse. It's millions of hours of de-duplicated work.
I'd like an alternative to HTML though. If I was to make a browser maybe I'd focus on replacing HTML because I can't stand it, and replacing js just because the runtime is heavy.
Like, a browser that only runs wasm and has nearly no JS runtime would make me giggle
niutech 62 days ago [-]
For less RAM, try: Dillo, Netsurf, Otter Browser, K-Meleon, Pale Moon, Basilisk, Ultralight.
As an alternative for HTML, why not QML with Canonic browser?
PaulDavisThe1st 62 days ago [-]
> Like, a browser that only runs wasm
That's not a browser.
More or less by definition, a browser is an application that can use HTTP (and potentially other protocols) to make requests to other systems and retrieve stuff described using HTML (and possibly other formats).
Sure, a tool that just loads wasm and executes it would be fun (and probably exists, at least for the local case). But it's not a web browser.
01HNNWZ0MV43FF 62 days ago [-]
As opposed to current browsers that run wasm and JS I mean
Yes there would be a DOM in addition
pmarreck 62 days ago [-]
Or perhaps an entirely new platform/protocol, since this one is completely saturated with complexity at this point.
amjoshuamichael 62 days ago [-]
I keep coming back to this idea as the (albeit ideal) future of the web. HTML keeps morphing and changing to fit the increasingly complex requirements of modern web apps. I mean the W3C spec is 114 million words (1). I think that web apps as a concept are a good idea, but I just can't believe that HTML/CSS/JS are the best technologies to fill that out. I'd love to see someone tackle a new, "micro-sized app format", with a much simpler document format, and something like a FORTH as a scripting language. Uxn (2) and Decker (3) are good examples of this, but obviously a proper implementation would have to be built with the full range of possible UI and accessibility in mind, not just monochrome bitmap displays.
A web standard so simple, anyone can implement it!
Reflowing UIs to suit varying aspect ratios is a challenging puzzle that Decker presently doesn't touch, though I do have some ideas for the future. If decks were embedded as inline "applets" of interactive content within a fluid document container (markdown, gemtext, etc) I think reflow would be considerably less important.
RodgerTheGreat 61 days ago [-]
I think multiple smaller formats and protocols would be a better approach than one big new standard, and we already have lots of simple, broadly-supported options to choose from.
There's nothing to say that a Gemini browser (for example) can't choose to display gemtext links to other resources inline; images are a natural choice, but imagine a browser displaying CSV files as nice inline tables with standard affordances like sorting and line highlighting. No special browser support? No problem; lots of tools can work with CSV, and it's not completely unusable when displayed as raw text.
Some things don't currently have great options. We desperately need a standardized vector image format that doesn't have the cancerous complexity of SVG or EPS. A rich, declarative format for describing forms and their validation could be handy. Perhaps a simple self-contained format for interactive visualizations and games, like a stripped-down equivalent of Flash? (I have some ideas there.) As long as new formats are designed to degrade as gracefully as possible without special support and the means of composing formats is flexible enough, an ecosystem as a whole can grow and evolve while each component remains simple enough for one person to understand.
The present web and our universe of broadly supported technologies is very close to this ideal, if developers had a bit more restraint.
have you by any chance looked into alternative browser engines such as servo, ladybird, goanna, netsurf, sciter, flow etc?
pavpanchekha 62 days ago [-]
One of the authors here—thank you all for the nice words. Happy to answer questions!
SomeWan 62 days ago [-]
Thank you for this amazing book! I always wanted to learn more about the technological foundations that I rely on as web developer (or engineer, as you put it).
I am just curious, what was the process that lead you to decide to use Python to implement the browser? I feel like JavaScript via node would have offered a more related programming experience.
writing it in python makes me actually want to read the book, in fact i definitely will give it a read. if it was done in rust or go id probably skip it, and c++ is just too hard to look at for a fun project. will come back after reading and hopefully have something more meaningful to say about it.
_benj 62 days ago [-]
It is so exciting to see material like this being made!
Browsers seem like mysterious, undecipherable black boxes, which is very likely how G wants them to be perceived, but that is cracking by seeing the efforts/results of such projects like ladybird and others!
I hope to one day be able to jump in and contribute to break that moat! And this books looks like an amazing start!
wmanley 62 days ago [-]
> I hope to one day be able to jump in and contribute to break that moat!
The moat isn't caused by a lack of non-chrome browser engines, it's because so few people use a non-chrome browser engine. Firefox already exists - it's just that ~no-one uses it and for websites that don't work with it those users have learnt to just open up chrome.
I'd love for the moat to be broken, and contributing to a browser engine like ladybird would be fun - but it doesn't contribute to breaking the moat. I'd love to know what would.
DavidPiper 62 days ago [-]
I'm one of those ~nobodies. Firefox is actually quite good these days, I use it at home and at work, 100% of the time - i.e. no Chrome or Safari fallback needed.
If anyone's looking for a reason to try a switch again, consider this your sign.
mattlondon 62 days ago [-]
Is the performance hit sorted yet when opening a page? For me Firefox used to hang for a second or two doing something? Just a blank screen with the progress bar paused around 20% or so and apparently nothing happening...(DNS? HTTPS handshake?) and then it would kick off and load normally. Happened on mobile(android) and desktop(windows, Linux, macros).
On chrome-based browsers the same pages on the same computers on the same network would load in within the blink of an eye, with no pause.
I eventually gave up and went back to chrome after Firefox being my daily for years. I prefer the dev tools in chrome anyway TBH
kroltan 62 days ago [-]
Did you happen to have used uBlock Origin during your stay?
It's the most common source of browser-load latency, as it by default blocks the main page request until it is able to load its blocklists, so when you open the browser afresh, or reopen a window, it takes a while until the browser gets to continue loading the thing you asked.
I think by default it comes as enabled in Firefox, go to the "uBlock settings > Filter Lists > Suspend network activity until all filter lists are loaded", though of course it is a tradeoff.
> In Firefox-based browsers, this setting is enabled by default. Disabling it gives the option to potentially speed up page load at browser launch, at the cost of possibly not properly filtering network requests as per filter lists or rules.
> In Chromium-based browsers, this setting is disabled by default, since Chromium-based browsers do not support natively suspending network requests.2 Enabling this setting in Chromium-based browsers may lead to negative side-effects at browser launch.
01HNNWZ0MV43FF 62 days ago [-]
Sorry to say but I've never seen that bug
abudimir 62 days ago [-]
Same here.
I have Safari and Chrome only for cross browser testing.
wmanley 62 days ago [-]
I use Firefox too, and I agree it is great - which goes to show that having a great browser engine is not enough to break the moat.
pmarreck 62 days ago [-]
Firefox is great. I use Nightly. Sync bookmarks and everything. Chrome completely unnecessary.
wmanley 61 days ago [-]
To clarify: My point is not that Firefox sucks, my point is that Firefox is great - better at handling the web than these new other web-browser engines will be for a long time, and yet it still hasn't broken the moat. It's not enough to make a great non-chrome web-browser - we already have one. It wouldn't make any difference toward that goal if we had 4 of them.
What is needed to break the moat are users - and enough for website operators to care. This is what Apple/Safari has. This is what Firefox increasingly lacks.
For and for that someone will have to do something other than make a great browser.
lewisjoe 62 days ago [-]
> which is very likely how G wants them to be perceived
One of the authors of the book is Chris, who leads the Blink rendering team at G :)
pavpanchekha 62 days ago [-]
I'm one of the book's two authors (the other is the head of Blink Rendering!) and I've talked to a number of people on the Chrome team. None of them have struck me as trying to keep browsers mysterious! On the contrary, folks who work on Chrome, Firefox, Safari, and Ladybird all seem incredibly excited to talk browsers and discuss how they work. The world of browser development is surprisingly small, the engineers often move between companies, and I think it'd be tough to keep a "conspiracy" going.
But I do think there's a real lack of teaching material (why I wrote the book) and even "common vocabulary" to discuss browser internals, especially for the core phases like layout and raster, which is something Chris and I are hoping to create with the book.
awesomekling 62 days ago [-]
Can confirm, am incredibly excited to talk browsers and discuss how they work!
(Hi Pavel, love the book!)
pradmatic 62 days ago [-]
I've been looking for a fun project to start and I'm already thoroughly enjoying this book. Kudos for making the writing particularly engaging.
I've been levelling up on browser internals, and this book is awesome. It helps build up intuition on how browsers work, without going through the millions of lines of chrome code.
mananaysiempre 63 days ago [-]
[flagged]
kaptainkarl 63 days ago [-]
As a native English speaker, yes that is kinda weird to be "incredibly irritated" by a sentence that's completely normal and well constructed.
mmkos 62 days ago [-]
With all due respect, this feels better suited as a journal entry rather than a comment on a HN thread.
mananaysiempre 62 days ago [-]
Perhaps, but it would have been of little use as a question there.
Once you acquire enough of a feeling for a foreign language that not every bit of your skill in it comes from somebody telling you that a particular thing is said in a particular way, you are doomed to live with a constant suspicion that your feeling is somehow off in a way you don’t recognize. Usually it can be suppressed and ignored, but sometimes it can’t, and occasionally it has to become a question. (I expect this is a fairly common experience.) This was one of those.
jgwil2 62 days ago [-]
I'm not sure about your point about "leveling" vs "leveling up", because for me, "leveling" means making a surface level/flat and does not have any video game-related meaning, but the "on" is probably coming from the phrase "to read up on" something (meaning to study/read about something; there's also a slangier variation, "to bone up on" something).
wslh 62 days ago [-]
Is there a promotional code for HN? I was a happy user of HTMLUnit [1] with Jython [2] in the past and am very interested in a future where we can automatically generate portions of browser code using code generation and verification techniques. I've never felt as comfortable with tools like Playwright/Cypress/Selenium as I did with HTMLUnit (with all due respect to both).
The book isn’t out yet, so no promo code, but the whole thing is free online.
wslh 62 days ago [-]
Thank you for your prompt response. I understand the item isn't available, but I clicked the "Add to Cart" button thinking it would either place me on a waitlist or ship once available. However, I was then prompted to enter a promo code, which caused the confusion.
pavpanchekha 62 days ago [-]
Ah, weird! We can ask our publisher. This HN post was a surprise (Chis & I didn’t put it up) so we weren’t prepared.
julenx 62 days ago [-]
I was able to pre-order the book just fine — it didn't prompt me for any promo codes.
austin-cheney 62 days ago [-]
Nice book. I would recommend splitting chapter 9 into two separate chapters where executing JavaScript via Duktape is one chapter and then interacting with the DOM and events are a separate later chapter.
pavpanchekha 62 days ago [-]
It might be best read in two sittings! The chapters get longer as the book goes on and tackles more advanced topics, and I do recommend following along in code as you progress through the book.
mannyv 62 days ago [-]
Some major problems that browsers have to deal with have to do with UI. Specifically:
1. Users expect the UI to update in realtime,
2. Developers expect the UI to update in realtime,
3. UIs tend to be single threaded, and
4. The principles of software engineering and the reality of software stress encapsulation, but the realtime nature of UI requires visibility into these processes.
And of course this impacts the Javascript layer, since this is yet another environment with the same constraints as above.
bberrry 63 days ago [-]
I'm so incredibly thankful that there are people like Pavel and Chris putting effort into articles like this. You are truly the best of us
bloopernova 62 days ago [-]
This is wonderful!
I had an opportunity to run a tutorial on basic command line usage for newer software engineers. It's always fun to see people's expressions or read their reactions to seeing me telnet to port 25 and 80.
This is amazing, I just want to drop everything and start digging through this. Well done!
adhamsalama 62 days ago [-]
Would be nice to have the option to download it as an epub to read it on my e-reader.
abixb 62 days ago [-]
My thoughts precisely. I wish there was a service that could convert book-length webpages into neatly formatted ePUB document. I did find a 'converter', [0] but the service has tons of room for improvement.
The website has a single HTML page version, I downloaded it using SingleFile extension on Firefox and sent it to my Kindle using Caliber after converting it to AZW3. The result is great!
rjurney 62 days ago [-]
What an awesomely comprehensive resource, reminds me of MITRE ATT&CK.
systems 62 days ago [-]
why python, why not a system programming language like
C, OCaml or Go (or newer languages like zig or odin)
Are web browsers, not considered to be "system software"
dvektor 62 days ago [-]
I think the point is to demonstrate how things work and are designed, and python is easy for everyone to understand. I don't think the author is recommending trying to write a production web browser in python. (or probably at all ;)
Basically, performance isn't a big focus, Python is very widely known and doesn't have too many quirks for non programmers to deal with, and systems languages emphasize error handling that, for expedience, we often need to skip.
nindalf 62 days ago [-]
Going to take a wild guess that maybe they're going to rely on the excellent, extensive standard library of Python which C and Zig can't compete with. The second constraint was probably that they want to keep the number of lines of code low to encourage more people to buy the book. That's where Python does better than Go - you can do a lot with list comprehensions and you don't have if err != nil every few lines.
pavpanchekha 62 days ago [-]
Definitely the second reason, but we actually try hard not to use too much of the standard library, for easy porting. But it's nice that sockets & ssl are standard, plus a (bad) GUI library.
FartyMcFarter 62 days ago [-]
They definitely are system software, since they include compilers and interpreters, software libraries and other such things which AFAIK have always been considered system software.
Browsers these days are about as complex as any operating system, or perhaps more complex when you consider all the non-systems stuff in them.
adhamsalama 62 days ago [-]
Looks very cool, will definitely read it! Thanks!
pmarreck 62 days ago [-]
I hope the AI gets good enough to dynamically translate from one language to another with high reliability, in case not everyone is a fan of Python
cyral 62 days ago [-]
It is, ask claude to translate a file and it can do it pretty flawlessly.
udev4096 62 days ago [-]
Of course it does. LLMs are only good for small scale tasks
mvesto 62 days ago [-]
This is awesome! Nice work
keepamovin 63 days ago [-]
hey, this book looks cool! well done :)
pavpanchekha 62 days ago [-]
Thanks!
62 days ago [-]
wai-dang-loveme 62 days ago [-]
[flagged]
bafatik870 62 days ago [-]
[flagged]
adr1an 62 days ago [-]
Care to elaborate? I've just noticed those are links to comments that are spam and the account was deleted on github.
Rendered at 00:06:03 GMT+0000 (Coordinated Universal Time) with Vercel.
Layout is really hard. Just tables by themselves are hard, even without any css around them. CSS makes layout impossibly difficult. I challenge anyone to keep the whole CSS spec and its associated behaviors in their head.
At this point css + html + javascript have become a dynamic PDL, and probably is one of the most complex pieces of software today.
As an aside, video decoding is offloaded onto hardware, so it's not as battery intensive as it used to be.
But implementing a layout engine is doable. CSS is not magic; there's a spec that can be (meticulously) transformed into code. I've occasionally showed code like this to people frustrated that CSS seems arbitrary, just to show them that there is a logic to the execution environment. Granted, you're not going to regularly click into it the way you'd click into the implementation of a library, but it's no different from something like React in that regard. I think it helps!
[1]: https://pavpanchekha.com/blog/css-floats.html
I'm still working on CSS layout, with hopefully another paper coming soon.
1. The most "core" parts of layout, like CSS 2 stuff, is pretty poorly considered with a bunch of weird features that interact in strange ways. (Floats and clearance? Margin collapsing?) Some parts of this "core" were intended to be universal even though they're a bad fit for other layout modes. (Margin and padding, for example, don't have a clear purpose for say grid elements.)
2. It's not well-modularized the way JS APIs are. A JS API can often be implemented fairly stand-alone, but each layout module interacts with every other layout module since they can be nested in various ways. I think newer specs like grid are trying to be stricter with this but there are fundamental challenges: the actual 2D screen is a shared resource that different layout modes must split up.
I can't be the only one who wants a simpler layout language than CSS that's designed with two decades of hindsight to provide the maximum simplicity-expressiveness product. Are there any serious projects to engineer something like this, or has everyone given up and either embraced CSS3 (waiting for the LLVM backend) or gone back to plain text?
Are there any resources on learning to design simpler layout systems (like flexbox + any other important parts) without having to adjust the design to compensate for older systems (e.g. if you were to try to implement CSS).
https://github.com/facebook/yoga
Layout does not have to be so complex. There are dozens of GUI frameworks with simpler layout system. Those are enough for applications everyone uses.
The ones that use absolute pixel positioning fail when using different resolution displays.
The ones that use box packing fail when you need to deal with different sized displays.
The ones that use constraint programming fail when you need to layout hundreds or thousands of widgets.
CSS-style layout has its own pros and cons, but there is no alternative to it that is clearly better under all circumstances. If you doing layout and want to be resolution-independent, function on everything from phones to giant displays and have thousands of things to layout, CSS is actually likely better than any alternative.
Video editors.
Image editors.
Web browsers.
Circuit, interior and architectural design.
None of these are anything like your "most desktop applications".
BTW all of those applications you mentioned use many of the same techniques video games use (for the same reason: custom layouts), existed long before the web and are doing just fine without CSS
Document readers lay out every possible document.
you mean like vector graphic software, DTPs, word processors etc etc?
Flash did resolution independent graphics and layouts on the web back in the late '90s, we killed it - not without reasons - but we still haven't created a decent alternative to it
Although it works poorly with translations, so it's probably not the best approach, but certainly one of the simplest. If you don't need translations, it works surprisingly well. Basically all Delphi applications used it and there were used a lot in private companies.
Even basic tools like those which were found in old Delphi, with anchoring components to the parent side, were sufficient for many UI-s, although some tasks like internalisation could case some pain.
In the end my opinion is that there should be a balance between usability and ease of implementation. Especially for technologies which are supposed to have many implementations.
In the years since I’ve used a lot of tricks for web app CSS that I’m not sure I’m smart enough to figure out. But then I’ve never thought as long and as hard about typesetting as I did at the time so who knows.
This is technically but not usefully true with most videos on the web today.
The video decode itself is accelerated, but each frame passes through JavaScript to be composited.
The only time video is fully hardware decoded is when it's a simple video element to a static video file.
I don't think that's true, and it's even less true once DRM video is involved - it becomes very difficult to get other software on the machine to even see the video, at least on Windows. You can very occasionally see bugs where the hardware accelerated playback ends up in a different place to where the browser thinks the video should have been put, too.
What does happen is the video data gets reassembled in Javascript (e.g. Video.js) because the native player doesn't support HLS. Not quite the same thing. It's just reformatting MPEG-TS to the similar but not identical MP4. Oddly, the browser in my LG TV does play HLS video natively, and I think Safari does?
What do you mean by that? There is no Javascript doing the actual compositing, and the actual compositing is (usually) hardware accelerated.
> The only time video is fully hardware decoded is when it's a simple video element to a static video file.
These seem in disagreement to me. The vast majority of videos on the web are simple video elements going to static video files. It is not usual for each frame to pass through JavaScript before being displayed.
Lol, no way.
People are always "guess what JS does, wut."
Doesn't hold a candle to Cascading Stylesheets.
I'm definitely interested in going through this book.
> [1] The library is used as of 2023 in Google Chrome, ChromeOS, ChromiumOS, Mozilla Firefox, Mozilla Thunderbird, Android, Firefox OS, Flutter,[5] Avalonia (from Alpha 4), LibreOffice (from version 7.0) and RAD Studio[6](since version 12.0).
> [2] Changes to the Skia repository will be rolled into Chromium by the AutoRoll bot several times per day.
> [3] It serves as the graphics engine for Google Chrome and ChromeOS, Android, Flutter, and many other products.
[1]: https://en.wikipedia.org/wiki/Skia_Graphics_Engine
[2]: https://skia.org/docs/dev/chrome/
[3]: https://skia.org/
[4]: https://github.com/chromium/chromium/tree/main/skia
[0]: https://blogs.igalia.com/carlosgc/2024/02/19/webkit-switchin...
* layout
* event handling
which are not exactly trivial for a "real" application (whatever that means).
The interesting space is really post-HTML UI/document tech. There's another thread running about Typst which is a sort of better LaTeX. Markdown was highly impactful. There's a lot of scope for people to do interesting things in this space that are "HTML but better". It doesn't even have to be a markup format - Typst and React HTML both blur the lines between code and data. Jetpack Compose shows how to use Kotlin's DSL features to make something that looks a bit like a UI description but which is actually code.
Of course it means you have to then either distribute a 'browser' for your format, or find a way to display it in the browser. But compiling down to some JS/HTML/WASM thing is certainly possible. You can also use portable GUI toolkits like JavaFX; that also gives you accessibility. Or do both!
Once you define your own UI language there's a lot of scope to try things that HTML doesn't do well. An obvious one is separation of content and style. HTML tried and never really got there. XSL:T tried harder but was a weird pure functional language with XML as its syntax. React does quite well with going JSON->boxes but the underlying protocols are always ad-hoc and tacked on, so you can't really write useful tooling on top of that.
Another idea would be a format that's natively immune to XSS.
This keeps being repeated. But it leans on three false assumptions.
- That is has to be "finished" at all. For many use-cases, a subset (of a subset) might just be fine. The screen in my refrigerator, or the information display in a train, might want to render some HTML, but when the HTML is controlled and constrained, there's no need for "everything".
- That is has to adhere to "the spec". See above, but also if the HTML+CSS+JS is less controlled, quite a few use-cases it's fine to ignore lots of the quirks or even large parts of the specs. Even Chrome and FF don't implement "all", whatever "the spec" might be in the first place. But a browser in a TV set-top box, my e-reader, some dedicated wikipedia-device, or the "help section of an app" are fine if they break on complex sites.
- That is must be implemented from scratch. Even if you forego the big rendering engines, JS VMs and so forth, there's a lot of libs that do DOM handling, CSS parsing, JS runtime etc. There's a lot of shoulders to stand on, aside from "just run chrome headless".
By repeating this mantra that its not worth "building a new browser" or "rendering engine", we only cement the status quo further. And promote the idea that your car, refrigerator, test-runner, help-section, dashboard, e-reader and whatnot must run either a full chrome or firefox. We stiffle innovation.
This is surprisingly easy. Because the "this is HTML" is both vague and limited. HTML5 is remarkably vague and permissive.
Also, HTML has nothing to do with CSS nor with JS, nor with any DOM apis and not even with how stuff has to be rendered or if it has to be rendered at all.
There are numerous libraries and projects that can parse the WHATWG version of HTML. I'm not saying these are trivial or simple. But far simpler than a full rendering engine will ever be.
Andreas Kling, the author of Ladybird browser, proves otherwise.
There is also QML as an alternative for HTML - see Canonic browser: https://www.canonic.com
Of course, jump forward 24 years and the KDE browser engine is basically the only game in town- the basis of both Chrome and Safari. Absolutely no way I saw that coming.
sounds like a skill issue
I'd like an alternative to HTML though. If I was to make a browser maybe I'd focus on replacing HTML because I can't stand it, and replacing js just because the runtime is heavy.
Like, a browser that only runs wasm and has nearly no JS runtime would make me giggle
As an alternative for HTML, why not QML with Canonic browser?
That's not a browser.
More or less by definition, a browser is an application that can use HTTP (and potentially other protocols) to make requests to other systems and retrieve stuff described using HTML (and possibly other formats).
Sure, a tool that just loads wasm and executes it would be fun (and probably exists, at least for the local case). But it's not a web browser.
Yes there would be a DOM in addition
A web standard so simple, anyone can implement it!
One can dream...
(1) https://drewdevault.com/2020/03/18/Reckless-limitless-scope.... (2) https://100r.co/site/uxn.html (3) https://internet-janitor.itch.io/decker
EDIT: There is Project Gemini (https://geminiprotocol.net/), but it doesn't support styling or scripting.
Reflowing UIs to suit varying aspect ratios is a challenging puzzle that Decker presently doesn't touch, though I do have some ideas for the future. If decks were embedded as inline "applets" of interactive content within a fluid document container (markdown, gemtext, etc) I think reflow would be considerably less important.
There's nothing to say that a Gemini browser (for example) can't choose to display gemtext links to other resources inline; images are a natural choice, but imagine a browser displaying CSV files as nice inline tables with standard affordances like sorting and line highlighting. No special browser support? No problem; lots of tools can work with CSV, and it's not completely unusable when displayed as raw text.
Some things don't currently have great options. We desperately need a standardized vector image format that doesn't have the cancerous complexity of SVG or EPS. A rich, declarative format for describing forms and their validation could be handy. Perhaps a simple self-contained format for interactive visualizations and games, like a stripped-down equivalent of Flash? (I have some ideas there.) As long as new formats are designed to degrade as gracefully as possible without special support and the means of composing formats is flexible enough, an ecosystem as a whole can grow and evolve while each component remains simple enough for one person to understand.
The present web and our universe of broadly supported technologies is very close to this ideal, if developers had a bit more restraint.
There is QML (see https://www.canonic.com) and Gemini (see https://gmi.skyjake.fi/lagrange/) as alternatives.
I am just curious, what was the process that lead you to decide to use Python to implement the browser? I feel like JavaScript via node would have offered a more related programming experience.
Apparently some of it now runs in the browser ("in the book itself") by compiling Python to JS?
https://browserbook.substack.com/p/compiling-python-to-js
Browsers seem like mysterious, undecipherable black boxes, which is very likely how G wants them to be perceived, but that is cracking by seeing the efforts/results of such projects like ladybird and others!
I hope to one day be able to jump in and contribute to break that moat! And this books looks like an amazing start!
The moat isn't caused by a lack of non-chrome browser engines, it's because so few people use a non-chrome browser engine. Firefox already exists - it's just that ~no-one uses it and for websites that don't work with it those users have learnt to just open up chrome.
I'd love for the moat to be broken, and contributing to a browser engine like ladybird would be fun - but it doesn't contribute to breaking the moat. I'd love to know what would.
If anyone's looking for a reason to try a switch again, consider this your sign.
On chrome-based browsers the same pages on the same computers on the same network would load in within the blink of an eye, with no pause.
I eventually gave up and went back to chrome after Firefox being my daily for years. I prefer the dev tools in chrome anyway TBH
It's the most common source of browser-load latency, as it by default blocks the main page request until it is able to load its blocklists, so when you open the browser afresh, or reopen a window, it takes a while until the browser gets to continue loading the thing you asked.
I think by default it comes as enabled in Firefox, go to the "uBlock settings > Filter Lists > Suspend network activity until all filter lists are loaded", though of course it is a tradeoff.
From the Wiki: https://github.com/gorhill/uBlock/wiki/Dashboard:-Filter-lis...
> In Firefox-based browsers, this setting is enabled by default. Disabling it gives the option to potentially speed up page load at browser launch, at the cost of possibly not properly filtering network requests as per filter lists or rules.
> In Chromium-based browsers, this setting is disabled by default, since Chromium-based browsers do not support natively suspending network requests.2 Enabling this setting in Chromium-based browsers may lead to negative side-effects at browser launch.
What is needed to break the moat are users - and enough for website operators to care. This is what Apple/Safari has. This is what Firefox increasingly lacks.
For and for that someone will have to do something other than make a great browser.
One of the authors of the book is Chris, who leads the Blink rendering team at G :)
But I do think there's a real lack of teaching material (why I wrote the book) and even "common vocabulary" to discuss browser internals, especially for the core phases like layout and raster, which is something Chris and I are hoping to create with the book.
(Hi Pavel, love the book!)
This comic book about how Chrome works is also a great place to get started: https://www.google.com/googlebooks/chrome/med_00.html
Once you acquire enough of a feeling for a foreign language that not every bit of your skill in it comes from somebody telling you that a particular thing is said in a particular way, you are doomed to live with a constant suspicion that your feeling is somehow off in a way you don’t recognize. Usually it can be suppressed and ignored, but sometimes it can’t, and occasionally it has to become a question. (I expect this is a fairly common experience.) This was one of those.
[1] https://htmlunit.sourceforge.io/
[2] https://www.jython.org/
1. Users expect the UI to update in realtime,
2. Developers expect the UI to update in realtime,
3. UIs tend to be single threaded, and
4. The principles of software engineering and the reality of software stress encapsulation, but the realtime nature of UI requires visibility into these processes.
And of course this impacts the Javascript layer, since this is yet another environment with the same constraints as above.
I had an opportunity to run a tutorial on basic command line usage for newer software engineers. It's always fun to see people's expressions or read their reactions to seeing me telnet to port 25 and 80.
https://news.ycombinator.com/item?id=28898157 (409 points | Oct 19, 2021 | 63 comments)
Web Browser Engineering: Scheduling tasks and threads - https://news.ycombinator.com/item?id=30336408 - Feb 2022 (3 comments)
Web Browser Engineering - https://news.ycombinator.com/item?id=28898157 - Oct 2021 (63 comments)
How Browsers Lay Out Web Pages - https://news.ycombinator.com/item?id=26707368 - April 2021 (50 comments)
Web Browser Engineering (A Book) - https://news.ycombinator.com/item?id=24055748 - Aug 2020 (1 comment)
[0] https://www.freeconvert.com/webpage-to-epub
Are web browsers, not considered to be "system software"
Basically, performance isn't a big focus, Python is very widely known and doesn't have too many quirks for non programmers to deal with, and systems languages emphasize error handling that, for expedience, we often need to skip.
Browsers these days are about as complex as any operating system, or perhaps more complex when you consider all the non-systems stuff in them.