Whatever expertise you need to prune a working system is less than the expertise you'll need to create a whole new one and then also prune it as it grows old

(I'm not including the C++ Standard Library, as I didn't implement it.)

It does take a huge amount of work to write and maintain. Typically the authors are not technical, so it also relies on the designers being available to answer questions as well. Then there’s a choice of how it’s written: narrative and potentially imprecise but readable, or terse and precise but hard to read. There’s both styles in the same document, terse for register descriptions.

I'm not sure large traditional silicon vendors like Intel, TI, et al re-evaluate the documentation requirements (and costs) on a chip by chip basis. It's probably done by chip class and for companies who've been selling chips by the millions over many decades to industries as diverse as defense, aerospace, automotive, etc there are classes of chips where robust, complete documentation is not only expected but often a required part of the RFP, compliance or conformance processes.

While this level of effort probably isn't needed for every chip in that class, it could be hard to reliably predict when a general purpose chip is still in the design phase which customers may be interested in it during its life (which for some of these chips might be decades). Many chips which conform to MIL-SPEC or other similar standards which can require extensive documentation are simply enhanced versions of standard chips, so the docs exist anyway. Finally, there's the organizational capabilities and culture aspect. Once the org needs to maintain the systemic ability to generate serious documentation at scale, you end up with a lot of managers and staff who think this way.

For comparison, a data sheet for a single transistor can be around 12 to 30 pages. A data sheet for a tiny microcontroller is probably a few hundred pages.

I once wrote a driver for a flash chip and that had a data sheet of around 80 pages.

Regards to (1), if you don't publish this information you're not selling a CPU, you're selling a very expensive chunk of sand. There is simply no way that a customer can guess at what your implementation looks like. Additionally, Intel barely has IP in the traditional sense. They hold patents, but their only real competitor in making x86 processors, AMD, has a long-standing mutual non-enforcement agreement wrt patents.

Regards to (2), I'm guessing a majority of this PDF can be generated sort of like you generate API documentation from doxygen comments.

[1]: https://www.ti.com/lit/ug/spruh73q/spruh73q.pdf?ts=175651560...

Assumptions, fair or not, about (1) seems more likely somehow.

Getting it to work on GCP meant properly driving something called the Intel PIIX4 controller which was emulated into the VM.

Separately from the OS being able to turn itself off, the OS needs to process a signal received by the hypervisor on this controller to support the hypervisor gracefully shutting it down. Otherwise GCP will wait 90 seconds after it has sent the shut down signal to give up and terminate the VM itself.

The problem I was trying to solve was (a) OS can shut itself down in GCP (b) restarts in GCP from the GCP console would be instant, rather than take 90+ seconds

Not the parent, but of course they're wasting their time... That's the point of a hobby OS.

I'm working on a hobby OS, and I have no illusions that it's most likely fewer than 10 people will ever run it, and less than 100 will hear about it, but it lets me explore some interesting (to me) ideas, and forces me to learn a little more about random pieces of computing. If I ran on GCP, I'd want the reboot button to work. That sounds useful.

On the topic, I don't see why anyone would want to build a general purpose OS. There's enough already and even with the shrinking of hardware variety, there's a lot of stuff to support to make a general purpose OS work on enough hardware for people to consider using it. You can take Linux or a BSD and hack it up pretty good to explore a lot of OS ideas. Chances are you're going to borrow some of their drivers anyway, and then you'll end up with at least some similarity... may as well start there and save a lot of time. (My hobby OS has a custom kernel and custom drivers, but I only support a bare minimum of devices... (pc) console i/o, one real NIC, and virtio-net... that's all I need; I might add support for more NICs and more consoles later)

This is not impossible, but the effort and costs required are substantial and often lose out on a priority basis to just fixing or improving the product itself.

This way, good documentation is priced into my estimate for the project. I don't have a work item "spend a few days documenting." Nope, if I'm doing a foo then that includes documenting a foo at the same time.

    > Look at Apple's documentation

What you describe is probably necessary for getting _fast_ Linux compatibility. However, if you are willing to take the overhead of a few layers of indirection, you can probably sandbox the Linux land somewhere, and not have it impact the rest of your design much.

Most hardware access doesn't have to be particularly efficient. And, yes, for the few pieces of hardware that you do want to support efficiently (eg your storage devices or networking, whatever you want to concentrate on in your design) these you can handle natively.

Btw, I would suggest that most people these days should write their toy operating systems to run as a VM on a hypervisor like Xen or similar. The surface to the outside world is smaller that way.

It has a rump kernel architecture which makes reusing the drivers almost trivial compared to reusing linus drivers with a new kernel.

  > you're never going to get there off of recycling linux's device drivers because that forces you to design your new OS as a linux clone in which cade you definitely did not need to write an entire new kernel from scratch.

probably it would have to be very general in design, but something along the lines of driverkit or iokit might work?

    > custom Wifi board

So please don't mock the spend. Big spends fail sometimes, and at least people were paid to do the work.

Why not, though? We used to have e.g. glass teletypes with microprocessors (8080/8051) in them that exposed a serial bus with very neat command protocol that we still use nowadays, that could boot up, init and self-test all on their own.

It helps to have a concept to guide you too, but you can certainly make some progress on the basics before you figure out what you really want to do.

[1] https://wiki.osdev.org/User:Zesterer/Bare_Bones

[2] https://osdev.wiki/wiki/Multiboot1_Bare_Bones

John Carmack, David Cutler, Tom West, Cameron Zwarich, etc. There are about maybe 50 of them.

I remember this guy wanted $20 million to build AGI a year ago (did he get that money?), and people here thought he would go into isolation for a few weeks and come out with AGI because he made some games like that. It’s just embarrassing as a community.

I am sure Carmack himself encourages debates and discussions. Lionizing one person can't be expected of every employee (unless that person is also the founder or the company is tiny).

Not some new OS so you can start making the glasses 2 years later.

Just like when you want to bake a cake, you dont start by designing an oven, or creating an universe from scratch

and also, it wasn't enough to "win" against a den-of-wolves place filled with power-players like meta.

XROS was an org that hired for specific specialist positions (as opposed to the usual "get hired into FB, go through the bootcamp, and find your place within the company"). At one point we got two separate requests from the recruiting execs: - Your tech screen pass rate is way too low compared to other teams at FB. Please consider making your tech screen easier to expand the pool of candidates. - Your interview-to-offer rate is way too low compared to other teams at FB. Please consider making your tech screen more difficult to reduce time that engineers spend on interviewing and writing feedback.

Anyway, IMO it was a very strong team in a very wrong environment. Most of the folks on the team hated the Facebook culture, despised the PSC process (despite having no problems with delivering impact in a greenfield project), had very little respect for non-technical managers coming from FB proper (the XROS team saw themselves as part of Oculus), and the majority I believe fled to other companies as soon as the project was scrapped. The pay was good however, and the work was very interesting. My overall impression was that most people on the team saw XROS as a journey, not a destination, and it was one of the reasons why it was destined to never ship.

If you want to be the dominant player in the market in 10-15 years, build the OS and keep funding it.

It doesn't sound like he's concerned with waste. It sounds like it's a typical Carmack argument - distilled and hyper logical, and his conclusion is more to do with the pointlessness of it. He actually concedes the point that the project may have been highly efficient (which it may or may not have been, he was steelmanning).

His main points seemed to be:

If every cycle matters and efficiency is paramount, just make the project monolithic C++ code. If every cycle matters, that is somewhat incompatible with general purpose OSs, and if it doesn't, the existing landscape is more than good enough. Presumably, he's calling out the absurdity of counter-arguments which are being unrealistic about the objectives of creating a new general purpose OS, while also focusing on extreme efficiency. He states that the requirements to fully achieve these objectives would require a "monastic coding enclave" like Plan 9 OS, and it wasn't realistic even with the high talent in Meta.

And that plays into the second point, which seems to essentially be "new OSs aren't a draw for developers, they are a burden". This is painfully obvious when looking at the history of OSs and software, and it's the obvious reason why "let the kids build their operating system ffs" should result in a reflexive "noooo..." from the greybeards. The deeper point though is that if A. is achieved, the B. Burden on devs will be even more onerous. Therefore unless the entire project is committed to truly moving crowds to new paradigms (good luck, literally billions have been lost here), just use the proven, faily high performance options that have widespread support.

The conclusion is "on balance, it's a bad idea." He's arguing it sharply (although I understand a Carmack steelman is intimidating to attack), but in essence it's a fairly banal and conservative conclusion, backed with strong precedent.

the problem was, it was holding back products. Because if youre going to make your own OS, it changes what chips you put in. If you don't know what chipset you're going to have, you don;t know what your pixel budget is, you can't plan features.

It takes about 2 years to get hardware out the door, and another 1.5 years to iron out the bugs and get a "finished" product.

And the Sr was generally a very nice person who did not give much weight to levels, willing to engage with anyone.

(Especially if it's an "API" persisted to disk)

Exactly! I picture the choice being grandfathering in compatibility with existing OSes (having the promised performance of their product in fact indirectly modulated by the output of all other teams of world's smartest throughout computing history and present day), vs wringing another OS-sized piece of C++ tech debt upon unsuspecting humanity. In which case I am thankful to Carmack for making the call.

I can understand how "what you're doing is fundamentally pointless" is something they can only afford to hear from someone who already has their degree of magnitude of fuck-you money. Furthermore in a VC-shaped culture it can also be a statement that's to many people fundamentally incomprehensible

Damn, I'd pay to work in some serious OS/Compiler teams, but hey why should they hire me? Oh well...Yeah I'm doing a bit of projects on my side but man I'm so burnt out by my 9am-5pm $$ job + 5pm-10pm kid job that I barely have any large chunk of time to work on those.

I'm not saying that there's no space for direct communication and that everyone needs to be formal and socially polite during every interaction. But I've met many people who act like you describe John does who very much do not appreciate getting it back, implying some level of awareness that their directness is hurtful on occasion.

I've only met a few direct people who can take it as well.

It makes me want to reframe this a little with your statement 'understand the person and adapt accordingly.' As someone who has learned their social skills later, I think it's usually more of a responsibility of the abrasive person to adapt their communication style and know when it is best used.

Specifically, I think abrasive and direct works great in high trust environments. It has served me well as well. It does sometimes relate to autism for me, ymmv.

Anyway the reason why it doesn't work outside of high trust environments is that people have feelings, and their feelings matter. Ultimately you do have a responsibility to try and be considerate. So like, for me I try to separate the high trust and low trust environments in my life, and keep the part of me that's direct and abrasive (often among peers in technical context) less vocal in the low trust environment.

When I intentionally want to push back in a low trust environment, I try to check in more with the person, look to where they seem uncomfortable, and double check I understand what their insecurities might be in a certain context as that often increases defensiveness.

Sometimes in low trust environments I might not notice, or I might identify it as low trust and just not care. In those contexts yeah I'll be the disgruntled aspie ;) but in other contexts I want to connect to people and really think through the impact of my words not the righteousness.

I mean maybe, but maybe Carmac is just an ass hole... He can be a "legend" in the software development world and also just not be a super great person socially. The two things aren't mutually exclusive.

I don't disagree with you entirely, but being "direct" isn't a get-out-of-jail-free card for poor interpersonal skills. It's not always about "fragile egos" or "entitlement", it's about basic professionalism and communication.

If I’m kinda sensitive but also hyper-ambitious, I acknowledge that Facebook has

1. Some of the highest pay in the industry. 2. Ultra-competitive environment. 3. Low moral principles.

Seems like the strategy would be to use every lever at your disposal to manipulate your environment, rather than leave and risk getting paid less.

In Doom Guy, though, Romero says that after he left iD, he heard from others still working there that the company had become something of a dictatorship under Carmack, and that within X months (I forget how many), half the company had quit. Romero also qualifies several times that they were all in their early/mid-20s and didn't have the requisite life experience to be handling business situations well

FWIW I like carmack from what I have seen publically (and Romero, who I have interacted with) but I wouldn’t pretend to know who either of them really are from my observation of them.

And in order to make it happen it also required writing the already-launched HTML-based UI in Flutter/Dart. Again ... why? What for? There wasn't even a working "native" Flutter at the time, despite promises, and there certainly wasn't a working accessibility stack -- no screen reader, no magnification, nothing -- so that all had to be kludged in. It was everything wrong with the "rewrites considered harmful" distilled.

Not to mention terrible for morale, execution, planning, budget, customer satisfaction.

I was just a lowly SWE 3 "IC" just in the trenches, not nearly as "important" as all that, so my opinion mattered not at all. But to me it violated every sound engineering / project planning principle I'd learned in the 15 years of my career up to that point. Just another event that led to me becoming quite cynical about the ability of leadership at Google to actually manage anything of significant complexity that wasn't ads/search related.

Again, Fuchsia .. very neat. But it didn't belong there.

It was much easier to cripple your competition back when there were several orders of magnitude less software engineers in the world.

Your sub's officers also need to constantly be aware of what to communicate to whom and in which language. Your superiors certainly kept you in the dark about a ton of concerns that were on their plate because simply mentioning them to subordinates would have been too distracting.

If you are building for a single abstraction, code gets much simpler, instead of building a platform that multiple abstractions can then be built on top of.

Those boards were always accompanied by horrific binary blobs glued to a kernel form a stone age. Or Windows.

5 Millions alone for the AMD graphic driver.

(Actually I just realised you meant "buy")

It’s completely tangential at this point. Meta dreams of a metaverse but most of their investments have gone to solving foundational issues with VR.

If Meta wants to pay for that, well, I’m not going to bite the hands that feeds VR research. My starry-eyed teenage years are thankfully far behind me.

But my enthusiasm diminished after learning that they worked thousands people to death.

Interesting exemple because the biggest investors in renewable by far are big oil companies.

So what should people do? Go or not go? Sadly, the world is never black and white.

> it only went down after you know who passed away.

I mean that's kind of ironic. You believe that Jobs was a great and decent person who led Apple to be roses and rainbows. Outside this cult of personality it was a company that engaged in unethical and monopolistic business practices just as anyone else did, walled-garden lock-in of their platform, intentionally making their devices impossible to repair, planned obsolescence, slave labor conditions in manufacturing, etc. etc.

So the point is untrue, its the benevolent dictator delusion, just like there are no benevolent political authoritarian leaders so are there no benevolent CEOs either. Because the system itself is the issue, authoritarianism/capitalism. If you are a conservative/authoritarian you believe in the former if you are a liberal/libertarian/capitalist you believe in the latter.

I'm not familiar of any platform that calls for collective ownership of the means of production, land redistribution and other far left positions.

I don't think it's just about legality. Whether the recommendation letter is included in the application is at the distinction of the applicant. When you want it to reach the next company, you must write is so, that the former employee considers it to be a good recommendation.

And me saying cool off is madness? You must live in a mad, mad world. Good luck going forward.

But in any case, almost all interesting projects are likely to fail. Of course it is objectively unlikely that a project to write a new OS will succeed. I expect the people working on it were aware of that.