One has to add that from the 218 UB in the ISO C23, 87 are in the core language. From those we already removed 26 and are in progress of removing many others. You can find my latest update here (since then there was also some progress): https://www.open-std.org/jtc1/sc22/wg14/www/docs/n3529.pdf
tialaramex 22 hours ago [-]
A lot of that work is basically fixing documentation bugs, labelled "ghosts" in your text. Places where the ISO document is so bad as a description of C that you would think there's Undefined Behaviour but it's actually just poorly written.
Fixing the document is worthwhile, and certainly a reminder that WG21's equivalent effort needs to make the list before it can even begin that process on its even longer document, but practical C programmers don't read the document and since this UB was a "ghost" they weren't tripped by it. Removing items from the list this way does not translate to the meaningful safety improvement you might imagine.
There's not a whole lot of movement there towards actually fixing the problem. Maybe it will come later?
uecker 21 hours ago [-]
Fixing the actual problems is work-in-progress (as my document also indicates), but naturally it is harder.
But the original article also complains about the number of trivial UB.
taneq 19 hours ago [-]
> practical C programmers don't read the document and since this UB was a "ghost" they weren't tripped by it
I would strongly suspect that C compiler implementers very much do read the document, though. Which, as far as I can see, means "ghosts" could easily become actual UB (and worse, sneaky UB that you wouldn't expect.)
tialaramex 19 hours ago [-]
The previous language might cause a C compiler developer to get very confused because it seems as though they can choose something else but what it is isn't specified, but almost invariably eventually they'll realise oh, it's just badly worded and didn't mean "should" there.
It's like one of those tricky self-referential parlor box statements. "The statement on this box is not true"? Thanks I guess. But that's a game, the puzzles are supposed to be like that, whereas the mission of the ISO document was not to confuse people, so it's good that it is being improved.
uecker 17 hours ago [-]
Most of the "ghosts" are indeed just cleaning up the wording. But compiler writers historically often used any excuse that the standard is not clear to justify aggressive optimization. This starts with an overreaching interpretation of UB itself, to wacky concepts such as time-travel, wobbly numbers, incorrect implementation of aliasing (e.g. still in clang), and pointer-to-integer round trips.
tialaramex 13 hours ago [-]
I'm sure the compiler authors will disagree that they were "using any excuse". From their point of view they were merely making transformations between equivalent programs, and so any mistake is either that these are not in fact equivalent programs because they screwed up - which is certainly sometimes the case - or the standard should not have said they were equivalent but it did.
One huge thing they have on their side is that their implementation is concrete. Whatever it is that, say, GCC does is de facto actually a thing a compiler can do. The standards bodies (and WG21 has been worse by some margin, but they're both guilty) may standardize anything, but concretely the compiler can only implement some things. "Just do X" where X isn't practical works fine on paper but is not implementable. This was the fate of the Consume ordering. Consume/ Release works fine on paper, you "just" need to have whole program analysis to implement it. Well of course that's not practical so it's not implemented.
uecker 12 hours ago [-]
They sometimes screwed up, sometimes just because of bugs, or because different optimization passes had different assumptions that are inconsistent. This somehow contradicts your second point. Compiler have something things implemented which may be concrete on some sense (because it is in a compiler), but still not really a "thing" because it is a mess nobody can formalize using a coherent set of rules.
But then, they also sometimes misread the standard in ways I can't really understand. This often can be seen when the "interpretation" changes over time. Earlier compilers (or even earlier parts of the same compiler) implement the standard as written, some new optimization pass has some creative interpretation.
tialaramex 10 hours ago [-]
Certainly compiler developers are only human, and many of them write C++ so they're humans working with a terrible programming language, I wouldn't sign up for that either (I have written small contributions to compilers, but not in C++). I still don't see "any excuses". I see more usual human laziness and incompetence, LLVM for example IMNSHO doesn't work hard enough to ensure their IR has coherent semantics and to deliver on those semantics.
But it seems like it's just that everybody fucked this up in similar ways, that's two different major compiler backends! I wouldn't be surprised if Microsoft (whose code we can't see) find that they don't get this quite right either.
Sharlin 17 hours ago [-]
If I understand correctly, the "ghosts" are vacuously UB. As in, the standard specifies that if X, then UB, but X can in fact never be true according to the standard.
ncruces 23 hours ago [-]
And yet, I see P1434R0 seemingly trying to introduce new undefined behavior, around integer-to-pointer conversions, where previously you had reasonably sensible implementation defined behavior (the conversions “are intended to be consistent with the addressing structure of the execution environment").
Pointer provenance already existed before, but the standards were contradictory and incomplete. This is an effort to more rigorously nail down the semantics.
i.e., the UB already existed, but it was not explicit had to be inferred from the whole text and the boundaries were fuzzy. Remember that anything not explicitly defined by the standard, is implicitly undefined.
Also remember, just because you can legally construct a pointer it doesn't mean it is safe to dereference.
ncruces 20 hours ago [-]
The current standard still says integer-to-pointer conversions are implementation defined (not undefined) and furthermore "intended to be consistent with the addressing structure of the execution environment" (that's a direct quote).
I have an execution environment, Wasm, where doing this is pretty well defined, in fact. So if I want to read the memory at address 12345, which is within bounds of the linear memory (and there's a builtin to make sure), why should it be undefined behavior?
And regarding pointer provenance, why should going through a pointer-to-integer and integer-to-pointer conversions try to preserve provenance at all, and be undefined behavior in situations where that provenance is ambiguous?
The reason I'm using integer (rather than pointer) arithmetic is precisely so I don't have to be bound by pointer arithmetic rules. What good purpose does it serve for this to be undefined (rather than implementation defined) beyond preventing certain programs to be meaningfully written at all?
I'm genuinely curious.
uecker 17 hours ago [-]
I fully agree with your analysis but compilers writers did think the could bend the rules, hence it was necessary to clarify that pointer-to-integer casts do work as intended. This still not in ISO C 23 btw because some compiler vendors did argue against it. But it is a TS now. If you are, please file bugs against your compilers.
gpderetta 15 hours ago [-]
Do you fully agree? I finally went and read n3005.pdf. The important item there is that a cast to integer exposes the pointer and now the compiler must be conservative and assume that the pointed object might be changed via non trackable pointers. This seems quite a reasonable compromise to make existing code work without affecting the vast majority of objects whose address is never cast to an integer. But ncruces wants defined semantics for arbitrary forged pointers.
uecker 12 hours ago [-]
You are right, I wasn't thinking straight. I do not fully agree. Creating arbitrary pointers can not work. Forging pointers to implementation-defined memory region would be ok though.
ncruces 11 hours ago [-]
Why can't it work though?
And I'm taking about both things.
Integer arithmetic that produces pointers that are just out of bounds of an object. Why can't this work? Why can't the compiler assume that, since I explicitly converted a pointer to an integer, the pointed-to object can't be put into a register, or made to go out or scope early?
Second, fabricating pointers. If I have a pointer to mmap/sbrk memory, shouldn't I be allowed to “fabricate” arbitrary pointers from integers that point into that area? If not, why not?
Finally Wasm. The linear memory is addressable from address 0 to __builtin_wasm_memory_size * PAGESIZE. Given this, and except maybe the address at zero, why should it be undefined behavior to dereference any other address?
What's the actual advantage to making these undefined behavior? What to we gain in return?
SkiFire13 16 hours ago [-]
> I have an execution environment, Wasm, where doing this is pretty well defined, in fact. So if I want to read the memory at address 12345, which is within bounds of the linear memory (and there's a builtin to make sure), why should it be undefined behavior?
How would you define it? Especially in a way that is consistent with the rest of the language and allows common optimizations (remember that C supports variables, which may or may not be stored in memory)?
ncruces 15 hours ago [-]
Just read whatever is at address 12345 of the linear memory. Doesn't matter what that is. If it's an object, if it was malloc'ed, if it's the "C stack", a "global".
It's the only way to interpret *(uint64_t*)(12345) when the standard says that a integer-to-pointer conversion is "intended to be consistent with the addressing structure of the execution environment".
There exists an instruction to do that load in Wasm, there's a builtin to check that 12345 points to addressable memory, the load is valid at the assembly level, the standard says the implementation should define this to be consistent with the addressing structure of the execution environment, why the heck are we playing games and allowing the compiler to say, "nope, that's not valid, so your entire program is invalid, and we can do what ever we want, no diagnostic required"?
vlovich123 8 hours ago [-]
If a newer version of that value is also stored in a register and not yet flushed to memory, should the compiler know to insert that flush for your or is reading a stale value ok?
For what it’s worth there’s a reason you’re supposed to do this kind of access through memcpy, not by dereferencing made up pointers.
> There exists an instruction to do that load in Wasm, there's a builtin to check that 12345 points to addressable memory, the load is valid at the assembly level, the standard says the implementation should define this to be consistent with the addressing structure of the execution environment, why the heck are we playing games and allowing the compiler to say, "nope, that's not valid, so your entire program is invalid, and we can do what ever we want, no diagnostic required"?
Because the language standard is defined to target a virtual machine as output, not any given implementation. That virtual machine is then implemented on various platforms, but the capabilities of the underlying system aren’t directly accessible - they are only there to implement the C virtual machine. That’s why C can target so many different target machines.
jcranmer 10 hours ago [-]
In a compiler, you essentially need the ability to trace all the uses of an address, at least in the easy cases. Converting a pointer to an integer (or vice versa) isn't really a deal-breaker; it's essentially the same thing as passing (or receiving) a pointer to an unknown external function: the pointer escapes, whelp, nothing more we can do in that case for the most part.
But converting an integer to a pointer creates a problem if you allow that pointer to point to anything--it breaks all of the optimizations that assumed they could trace all of the uses of an address. So you need something like provenance to say that certain back-conversions are illegal. The most permissive model is a no-address-taken model (you can't forge a pointer to a variable whose address was never taken). But most compilers opt instead for a data-dependency-based model: essentially, even integer-based arithmetic of addresses aren't allowed to violate out-of-bounds at the point of dereference. Or at least, they claim to--the documentation for both gcc and llvm have this claim, but both have miscompilation bugs because they don't actually allow this.
The proposal for pointer provenance in C essentially looks at how compilers generally implement things and suggests a model that's closer to their actual implementation: pointer-to-integer exposes the address such that any integer-to-pointer can point to it. Note this is more permissive than the claimed models of compilers today--you're explicitly able to violate out-of-bounds rules here, so long as both objects have had their addresses exposed. There's some resistance to this because adhering to this model also breaks other optimizations (for example, (void*)(uintptr_t)x is not the same as x).
As a practical matter, pointer provenance isn't that big of a deal. It's not hard to come up with examples that illustrate behaviors that cause miscompilation or are undefined specifically because of pointer provenance. But I'm not aware of any application code that was actually miscompiled because the compiler implemented its provenance model incorrectly. The issue gets trickier as you move into systems code that exists somewhat outside the C object model, but even then, most of the relevant code can ignore their living outside the object model since resulting miscompiles are prevented by inherent optimization barriers anyways (note that to get a miscompile, you generally have to simultaneously forge the object's address, have the object's address be known to the compiler already, and have the compiler think the object's address wasn't exposed by other means).
bcrl 17 hours ago [-]
It is important to understand why undefined behaviour has proliferated over the past ~25 years. Compiler developers are (like the rest of us) under pressure to improve metrics like the performance of compiled code. Often enough that's because a CPU vendor is the one paying for the work and has a particular target they need to reach at time of product launch, or there's a new optimization being implemented that has to be justified as showing a benefit on existing code.
The performance of compilers is frequently measured using the SPEC series of CPU benchmarks, and one of the main constraints of the series SPEC series of tests is that the source code of the benchmark cannot be changed. It is static.
As a result, compiler authors have to find increasingly convoluted ways to make it possible for various new compiler optimizations to be applied to the legacy code used in SPEC. Take 403.gcc: it's based on gcc version 3.2 which was released on August 14th 2002 -- nearly 23 years ago.
By making certain code patterns undefined behaviour, compiler developers are able to relax the constraints and allow various optimizations to be applied to legacy code in places which would not otherwise be possible. I believe the gcc optimization to eliminate NULL pointer checks when the pointer is dereferenced was motivated by such a scenario.
In the real world code tends to get updated when compilers are updated, or when performance optimizations are made, so there is no need for excessive compiler "heroics" to weasel its way into making optimizations apply via undefined behaviour. So long as SPEC is used to measure compiler performance using static and unchanging legacy code, we will continue to see compiler developers committing undefined behaviour madness.
The only way around this is for non-compiler developer folks to force language standards to prevent compilers from using undefined behaviour to do that which normal software developers considers to be utterly insane code transformations.
uecker 16 hours ago [-]
Language standards have much less power than people think and compiler-vendors are of course present in the standard working groups. Ultimately, the users need to put pressure on the compiler vendors. Please file bugs - even if this often has no effect, it takes away the argument "this is what our users want". Also please support compilers based on how they deal with UB and not on the latest benchmark posted somewhere.
bcrl 15 hours ago [-]
Language standards have plenty of power over compiler vendors, however, very few people that are not involved in writing compilers tend to participate in the standards process. Standards bodies bend to the will of those participating.
pjmlp 17 hours ago [-]
Dr. Dobbs used to have articles with those benchmarks, here are a couple of examples,
Pointer provenance was certainly not here in the 80s. That's a more modern creation seeking to extract better performance from some applications at a cost of making others broken/unimplementable.
It's not something that exists in the hardware. It's also not a good idea, though trying to steer people away from it proved beyond my politics.
jcranmer 19 hours ago [-]
Pointer provenance probably dates back to the 70s, although not under that name.
The essential idea of pointer provenance is that it is somehow possible to enumerate all of the uses of a memory location (in a potentially very limited scope). By the time you need to introduce something like "volatile" to indicate to the compiler that there are unknown uses of a variable, you have to concede the point that the compiler needs to be able to track all the known uses within a compiler--and that process, of figuring out known uses, is pointer provenance.
As for optimizations, the primary optimization impacted by pointer provenance is... moving variables from stack memory to registers. It's basically a prerequisite for doing any optimization.
The thing is that traditionally, the pointer provenance model of compilers is generally a hand-wavey "trace dataflow back to the object address's source", which breaks down in that optimizers haven't maintained source-level data dependency for a few decades now. This hasn't been much of a problem in practice, because breaking data dependencies largely requires you to have pointers that have the same address, and you don't really run into a situation where you have two objects at the same address and you're playing around with pointers to their objects in a way that might cause the compiler to break the dependency, at least outside of contrived examples.
JonChesterfield 8 hours ago [-]
My grievance isn't with aliasing or dataflow, it's with a pointer provenance model which makes assumptions which are inconsistent with reality, optimises based on it, then justifies the nonsense that results with UB.
When the hardware behaviour and the pointer provenance model disagree, one should change the model, not change the behavior of the program.
jcranmer 3 hours ago [-]
Give me an example of a program that violates pointer provenance (and only pointer provenance) that you think should be allowed under a reasonable programming model.
gpderetta 20 hours ago [-]
I'm not a compiler writer, but I don't know how you would be able to implement any optimization while allowing arbitrary pointer forging and without whole-program analysis.
JonChesterfield 8 hours ago [-]
It's an interesting question.
Say you're working with assembly as your medium, on a von neumann machine. Writing to parts of the code section is expected behaviour. What can you optimise in such a world? Whatever cannot be observed. Which might mean replacing instructions with sequences of the same length, or it might mean you can't work out anything at all.
C is much more restricted. The "function code" isn't there, forging pointers to the middle of a function is not a thing, nor is writing to one to change the function. Thus the dataflow is much easier, be a little careful with addresses of starts of functions and you're good.
Likewise the stack pointer is hidden - you can't index into the caller's frame - so the compiler is free to choose where to put things. You can't even index into your own frame so any variable whose address is not taken can go into a register with no further thought.
That's the point of higher level languages, broadly. You rule out forms of introspection, which allows more stuff to change.
C++ has taken this too far with the object model in my opinion but the committee disagrees.
ncruces 20 hours ago [-]
Why? What specific optimization do you have in mind that prevents me from doing an aligned 16/32/64-byte vector load that covers the address pointed to by a valid char*?
gpderetta 20 hours ago [-]
Casting a char pointer to a vector pointer and doing vector loads doesn't violate provenance, although it might violate TBAA.
Regarding provenance, consider this:
void bar();
int foo() {
int * ptr = malloc(sizeof(int));
*ptr = 10;
bar();
int result = *ptr;
free(ptr);
return result;
}
If the compiler can track the lifetime of the dynamically allocated int, it can remove the allocation and covert this function to simply
int foo() {
bar();
return 10;
}
It can't if arbitrary code (for example inside bar()) can forge pointers to that memory location. The code can seem silly, but you could end up with something similar after inlining.
torstenvl 19 hours ago [-]
> It can't if arbitrary code (for example inside bar()) can forge pointers to that memory location.
Yes. It absolutely can. What are you even talking about?
C is not the Windows Start Menu. This habit of thinking it needs to do what it thinks I might expect instead of what I told it is deeply psychotic.
gpderetta 19 hours ago [-]
I litterally have no idea what are you trying to say. Do you mean that bar should be allowed to access *ptr with impunity or not?
torstenvl 16 hours ago [-]
I'm not trying to say anything. I said and meant exactly what I said. No more, no less. Your logic is obviously flawed. There is nothing preventing that optimization in the presence of a forged pointer in bar().
gpderetta 16 hours ago [-]
Either there is no provenance, forging is allowed and the optimization is disallowed; or there is provenance and forging the pointer and attempting to inspect (or modify) the value of *ptr in bar() is UB.
ncruces 11 hours ago [-]
You never converted ptr to an integer. If you did, if the pointer escapes, yes, I claim that then the allocation can't be optimized away. Why is that so bad?
torstenvl 15 hours ago [-]
Attempting to inspect or modify the value of *ptr in bar() through a forged pointer was always UB. You are saying absolutely nothing meaningful.
ncruces 19 hours ago [-]
Can't reply to the sibling comment, for some reason.
If you don't know the extents of the object pointed to by the char*, using an aligned vector load can reach outside the bounds of the object. Keeping provenance makes that undefined behavior.
Using integer arithmetic, and pointer-to-integer/integer-to-pointer conversions would make this implementation defined, and well defined in all of the hardware platforms where an aligned vector load can never possibly fail.
So you can't do some optimizations to functions where this happens? Great. Do it. What else?
As for why you'd want to do this. C makes strings null-terminated, and you can't know their extents without strlen first. So how do you implement strlen? Similarly your example. Seems great until you're the one implementing malloc.
But I'm sure "let's create undefined behavior for a libc implemented in C" is a fine goal.
gpderetta 19 hours ago [-]
[when there is no reply button, you need to click on the date (i.e. N minutes ago) to get the reply box]
I think your example would fall foul of reading beyond the end of an object in addition to pointer provenance. In your case the oob read is harmless as you do not expect any meaningful values for the extra bytes, but generally the compiler would not be able to give any guarantees about the content of the additional memory (or that the memory exists in the first place).
This specific use case could be addressed by the standard, but vectors are already out of the standard, so in practice you use whatever extension you have to use and abide to whatever additional rule the compiler requires (of course this is often underspecified). For example, on GCC simd primitives already have carve-outs for TBAA.
FWIW, a libc implementation in practice already must rely on compiler specific, beyond the standard behaviour anyway.
tialaramex 19 hours ago [-]
> [when there is no reply button, you need to click on the date (i.e. N minutes ago) to get the reply box]
As an off-topic aside here that might help anybody who is wondering: HN deliberately doesn't provide "Reply" for very recent comments to try to dissuade you from having the sort of urgent back-and-forth you might reasonably do in a real time chat system, and less reasonably attempt (and likely regret) on platforms like Twitter.
A brief window to think about the thing you just read might cause you to write something more thoughtful, and even to realise that it wasn't saying what you had thought in the first place.
My favourite example was an example where somebody said a feature means "less typing" and another comment insisted it did not, and I was outraged until I realised all that's happening is that one person thinks "Typing" means "You know, pressing keys on your keyboard" and the other person thinks "Typing" means "You know, why an integer is different from a float in C" and so they're actually not even disagreeing the conflict is purely syntax!
gpderetta 18 hours ago [-]
Allegedly. Instead I like to think it is a reality check to remind me I'm wasting too much time on HN and should I do something productive :D
tialaramex 20 hours ago [-]
> It's not something that exists in the hardware
This is sort of on the one hand not a meaningful claim, and then on the other hand not even really true if you squint anyway?
Firstly the hardware does not have pointers. It has addresses, and those really are integers. Rust's addr() method on pointers gets you just an address, for whatever that's worth to you, you could write it to a log maybe if you like ?
But the Morello hardware demonstrates CHERI, an ARM feature in which a pointer has some associated information that's not the address, a sort of hardware provenance.
lmkg 20 hours ago [-]
It very much is something that exists in hardware. One of the major reasons why people finally discovered the provenance UB lurking in the standard is because of the CHERI architecture.
pjmlp 16 hours ago [-]
People keep forgetting that SPARC ADI did it first with hardware memory tagging for C.
AnimalMuppet 19 hours ago [-]
So it's something that exists in some hardware. Are you claiming that it exists in all hardware, and we only realized that because of CHERI? Or are you claiming that it exists in CHERI hardware, but not in others.
If it only exists in some hardware, how should the standard deal with that?
lmkg 16 hours ago [-]
> If it only exists in some hardware, how should the standard deal with that?
Generally seems to me the C standard makes things like that UB. Signed integer overflow, for example. Implemented as wrapping two's-complement on modern architectures, defined as such in many modern languages, but UB in C due to ongoing support for niche architectures.
The issues around pointer provenance are inherent to the C abstract machine. It's a much more immediate show-stopper on architectures that don't have a flat address space, and the C abstract machine doesn't assume a flat address space because it supports architecture where that's not true. My understanding is that reflects some oddball historical architectures that aren't relevant anymore, nowadays that includes CHERI.
uecker 16 hours ago [-]
Historically, the reason was was often niche architectures. But sometimes certain behavior dies out and we can make semantics more strict. For example, two's complement is now a requirement for C. Still, we did not make signed overflow defined. The reasons are optimization and - maybe surprising for some - safety. UB can be used to insert the compile-time checks we need to make things safe, but often we can not currently require everyone to do this. At the same time, making things defined may make things worse. For example, finding wraparound bugs in unsigned arithmetic - though well-defined - is a difficult and serious problem. For signed overflow, you use a compiler flag and this is not exploitable anymore (could still be a DoS).
kazinator 23 hours ago [-]
Undefined behavior only means that ISO C doesn't give requirements, not that nobody gives requirements. Many useful extensions are instances where undefined behavior is documented by an implementation.
Including a header that is not in the program, and not in ISO C, is undefined behavior. So is calling a function that is not in ISO C and not in the program. (If the function is not anywhere, the program won't link. But if it is somewhere, then ISO C has nothing to say about its behavior.)
Correct, portable POSIX C programs have undefined behavior in ISO C; only if we interpret them via IEEE 1003 are they defined by that document.
If you invent a new platform with a C compiler, you can have it such that #include <windows.h> reformats all the attached storage devices. ISO C allows this because it doesn't specify what happens if #include <windows.h> successfully resolves to a file and includes its contents. Those contents could be anything, including some compile-time instruction to do harm.
Even if a compiler's documentationd doesn't grant that a certain instance of undefined behavior is a documented extension, the existence of a de facto extension can be inferred empirically through numerous experiments: compiling test code and reverse engineering the object code.
Moreover, the source code for a compiler may be available; the behavior of something can be inferred from studying the code. The code could change in the next version. But so could the documentation; documentation can take away a documented extension the same way as a compiler code change can take away a de facto extension.
Speaking of object code: if you follow a programming paradigm of verifying the object code, then undefined behavior becomes moot, to an extent. You don't trust the compiler anyway. If the machine code has the behavior which implements the requirements that your project expects of the source code, then the necessary thing has been somehow obtained.
throw-qqqqq 22 hours ago [-]
> Undefined behavior only means that ISO C doesn't give requirements, not that nobody gives requirements. Many useful extensions are instances where undefined behavior is documented by an implementation.
True, most compilers have sane defaults in many cases for things that are technically undefined (like take sizeof(void) or do pointer arithmetic on something other than a char). But not all of these cases can be saved by sane defaults.
Undefined behavior means the compiler can replace the code with whatever. So if you e.g. compile optimizing for size, the compiler will rip out the offending code, as replacing it with nothing yields the greatest size optimization.
Snippets of software exhibiting undefined behavior, executing e.g. both the true and the false branch of an if-statement or none etc. UB should not be taken lightly IMO...
eru 21 hours ago [-]
> [...] undefined behavior, executing e.g. both the true and the false branch of an if-statement or none etc.
Or replacing all you mp3s with a Rick Roll. Technically legal.
(Some old version of GHC had a hilarious bug where it would delete any source code with a compiler error in it. Something like this would technically legal for most compiler errors a C compiler could spot.)
pjmlp 22 hours ago [-]
Unfortunely it also means that when the programmer fails to understand what undefined behaviour is exposed on their code, the compiler is free to take advantage of that to do the ultimate performance optimizations as means to beat compiler benchmarks.
The code change might come in something as innocent as a bug fix to the compiler.
account42 18 hours ago [-]
Ah yes, the good old "compiler writers only care about benchmarks and are out to hurt everyone else" nonsense.
I for one am glad that compilers can assume that things that can't happen according to the language do in fact not happen and don't bloat my programs with code to handle them.
adwn 18 hours ago [-]
> I for one am glad that compilers can assume that things that can't happen according to the language do in fact not happen and don't bloat my programs with code to handle them.
Yes, unthinkable happenstances like addition on fixed-width integers overflowing! According to the language, signed integers can't overflow, so code like the following:
int new_offset = current_offset + 16;
if (new_offset < current_offset)
return -1; // Addition overflowed, something's wrong
can be optimized to the much leaner
int new_offset = current_offset + 16;
Well, I sure am glad the compiler helpfully reduced the bloat in my program!
account42 17 hours ago [-]
Garbage in, garbage out. Stop blaming the compiler for your bad code.
adwn 2 hours ago [-]
You're objectively wrong. This code isn't bad, it's concise and fast (even without the compiler pattern-matching it to whatever overflow-detecting machine instructions happen to be available), and it would be valid and idiomatic for unsigned int. Stop blaming the code for your bad language spec.
account42 9 minutes ago [-]
The language spec isn't bad just because it doesn't allow you to do what you want. Are you also upset that you need to add memory barriers where the memory model of the underlying platform doesn't need them?
Again, this isn't undefined behavior to fuck you over and compilers don't use it for optimizations because they hate you. It's because it makes a real difference for performance which is the primary reason low level languages are used.
If you for some reason want less efficient C++ then compilers even provide you flags to make this particular operation defined. There is no reason the rest of us have to suffer worse performance because of you.
Personally I would prefer if unsigned ints had the same undefined behavior by default with explicit functions for wrapping overflow. That would make developer intent much clearer and give tools a chance to diagnose unwanted overflow.
titzer 17 hours ago [-]
Moral hazard here. The rest of us, and all of society, now rests on a huge pile of code written by incorrigible misers who imagined themselves able to write perfect, bug-free code that would go infinitely fast because bad things never happen. But see, there's bugs in your code and other people pay the cost.
kazinator 16 hours ago [-]
There is an incredible amount of C out there relative to how the sky basically isn't falling.
titzer 15 hours ago [-]
Ransomware attacks against hospitals and a dark extortion economy churning tens if not hundreds of billions of dollars a year in losses and waste.
What would the "sky falling" look like to you? If you're expecting dramatic movie scenes like something out of Mr Robot, I'm afraid the reality is more mundane, just a never-ending series of basic programming errors that turn into remote code execution exploits because of language and compiler choices by people who don't pay the costs.
kazinator 13 hours ago [-]
To completely eliminate the possibility of ransomware attack, you need an incredibly locked down platform, and users who are impervious to social engineering.
Vulnerabilities to ransomware (and other forms of malware) can be perpetrated without a single bad pointer being dereferenced.
For instance, a memory-safe e-mail program can automatically open an attachment, and the memory-safe application which handles the attachment can blindly run code embedded in the document in a leaky sandbox.
There is an incredible amount of infrastructure out there that depends on C. Embedded devices, mobile devices, desktops, servers. Network stacks, telephony stacks, storage, you name it. Encryption, codecs, ...
Sky is falling would mean all of it would be falling down so badly that, for instance, you would have about a 50% chance of connecting a server that is more than four hops away.
account42 17 hours ago [-]
There's bugs in your code without undefined behavior too. Go use a different language if you don't care about performance, there are many to choose from.
pjmlp 16 hours ago [-]
Not only do I care about performance, the languages I use, are able to delivery both safety and performace at the level required for project delivery.
Unfortunely too many folks still pretend C is some kind of magic portable Assembly language that no other language on Earth is able to achieve the same.
Also if I care enough about ultimate performace, like anyone that actually cares about performance, I dust off my Assembly programming skills, alongside algorithms, datastructures and computer organisation.
quietbritishjim 21 hours ago [-]
> Including a header that is not in the program, and not in ISO C, is undefined behavior.
What is this supposed to mean? I can't think of any interpretation that makes sense.
I think ISO C defines the executable program to be something like the compiled translation units linked together. But header files do not have to have any particular correspondence to translation units. For example, a header might declare functions whose definitions are spread across multiple translation units, or define things that don't need any definitions in particular translation units (e.g. enum or struct definitions). It could even play macro tricks which means it declares or defines different things each time you include it.
Maybe you mean it's undefined behaviour to include a header file that declares functions that are not defined in any translation unit. I'm not sure even that is true, so long as you don't use those functions. It's definitely not true in C++, where it's only a problem (not sure if it's undefined exactly) if you ODR-rule use a function that has been declared but not defined anywhere. (Examples of ODR-rule use are calling or taking the address of the function, but not, for example, using sizeof on an expression that includes it.)
kazinator 20 hours ago [-]
> I can't think of any interpretation that makes sense
Start with a concrete example. A header that is not in our program, or described in ISO C. How about:
#include <winkle.h>
Defined behavior or not? How can an implementation respond to this #include while remaining conforming? What are the limits on that response?
> But header files do not have to have any particular correspondence to translation units.
A header inclusion is just a mechanism that brings preprocessor tokens into a translation unit. So, what does the standard tell us about the tokens coming from #include <winkle.h> into whatever translation unit we put it into?
Say we have a single file program and we made that the first line. Without that include, it's a standard-conforming Hello World.
quietbritishjim 19 hours ago [-]
Do you just meant an attempt to include a file path that couldn't be found? That's not a correct usage of the term "program" – that refers to the binary output of the compilation process, whereas you're taking about the source files that are the input to the compilation. That sounds a bit pedantic but I really didn't understand what you meant.
I just checked, and if you attempt to include a file that cannot be found (in the include path, though it doesn't use that exact term) then that's a constraint violation and the compiler is required to stop compilation and issue a diagnostic. Not undefined behaviour.
kazinator 16 hours ago [-]
Yes; we are more interested in the other case: it happens to be found.
What are the requirements then?
quietbritishjim 14 hours ago [-]
I don't get your point then. If the file is found then there is no undefined behaviour in the process of the file being included. There might be undefined behaviour in the overall translation unit after the text has been substituted in, but that's nothing to do with the preprocessor.
kazinator 13 hours ago [-]
> If the file is found then there is no undefined behaviour in the process of the file being included.
Correct; but processing doesn't stop there.
> There might be undefined behaviour in the overall translation unit
But what does that mean; how do you infer that there might be undefined behavior?
Does ISO C define the behavior, or does it not?
ISO C has nothing to say about what is in #include <winkle.h> if such a header is found and didn't come from the program.
Without having anything to say about what is in it, if it is found at all, ISO C cannot be giving a definition of behavior of the tokens that are substituted for that #include.
im3w1l 19 hours ago [-]
I think we are slowly getting closer to the crux of the matter. Are you saying that it's a problem to include files from a library since they are "not in our program"? What does that phrase actually mean? What is the bounds of "our program" anyway? Couldn't it be the set {main.c, winkle.h}
kazinator 16 hours ago [-]
> What is the bounds of our program?
N3220: 5.1.1.1 Program Structure
A C program is not required to be translated in its entirety at the same time. The text of the program
is kept in units called source files, (or preprocessing files) in this document. A source file together
with all the headers and source files included via the preprocessing directive #include is known
as a preprocessing translation unit. After preprocessing, a preprocessing translation unit is called a
translation unit. Previously translated translation units may be preserved individually or in libraries.
The separate translation units of a program communicate by (for example) calls to functions whose
identifiers have external linkage, manipulation of objects whose identifiers have external linkage, or
manipulation of data files. Translation units may be separately translated and then later linked to
produce an executable program.
> Couldn't it be the set {main.c, winkle.h}
No; in this discussion it is important that <winkle.h> is understood not to be part of the program; no such header is among the files presented for translation, linking and execution. Thus, if the implementation doesn't resolve #include <winkle.h> we get the uninteresting situation that a constraint is violated.
Let's focus on the situation where it so happens that #include <winkle.h> does resolve to something in the implementation.
quietbritishjim 14 hours ago [-]
The bit of the standard that you've quoted says that the program consists of all files that are compiled into it, including all files that are found by the #include directive. So, if <winkle.h> does successfully resolve to something, then it must be part of the program by definition because that's what "the program" means.
Your question about an include file that isn't part of the program just doesn't make any sense.
(Technically it says that those files together make up the "program text". As my other comment says, "program" is the binary output.)
kazinator 13 hours ago [-]
I see what you are getting at. Programs consist of materials that are presented to the implementation, and also of materials that come from the implementation.
So what I mean is that no file matching <winkle.h> has been presented as part of the external file set given to the implementation for processsing.
I agree that if such a file is found by the implementation it becomes part of the program, as makes sese and as that word is defined by ISO C, so it is not right terminology to say that the file is not part of the program, yet may be found.
If the inclusion is successful, though, the content of that portion of that program is not defined by ISO C.
quietbritishjim 11 hours ago [-]
It still seems like you have invented some notion of "program" that doesn't really exist. Most suspicious is when you say this:
> So what I mean is that no file matching <winkle.h> has been presented as part of the external file set given to the implementation for processsing.
The thing is, there is no "external file set" that includes header files, so this sentence makes no sense.
Note that when the preprocessor is run, the only inputs are the file being preprocessed (i.e., the .c file) and the list of directories to find include files (called the include path). That's not really part of the ISO standard, but it's almost universal in practice. Then the output of the preprocessor is passed to the compiler, and now it's all one flat file so there isn't even a concept of included files at this point. The object files from compilation are then passed to the linker, which again doesn't care about headers (or indeed the top-level source files). There are more details in practice (especially with libraries) but that's the essence.
I wonder if your confusion is based on seeing header files in some sort of project-like structure in an IDE (like Visual Studio). But those are just there for ease of editing - the compiler (/preprocessor) doesn't know or care which header files are in your IDE's project, it only cares about the directories in the include path. The same applies to CMake targets: you can add include files with target_sources(), but that's just to make them show up in any generated IDE projects; it has no effect on compilation.
Or are you just maybe saying that the developer's file system isn't part of the ISO C standard, so this whole textual inclusion process is by some meaning not defined by the standard? If so, I don't think that matches the conventional meaning of undefined behaviour.
If it's neither of those, could you clarify what exactly you mean by "the external file set given to the implementation for processing"?
kazinator 9 hours ago [-]
Let's drop the word "program" and use something else, like "project", since the word "program" is normative in ISO C.
The "project" is all the files going into a program supplied other than by the implementation.
C programs can contain #include directives. Those #include directives can be satisfied in one of three ways: they can reference a standard header which is specified by ISO C and hence effectively built into the hosted language, such as <stdio.h>.
C programs can #include a file from the project. For instance someone's "stack.c" includes "stack.h". So yes, there is an external file set (the project) which can have header files.
C programs can also #include something which is neither of the above. That something might be not found (constraint violation). Or it might be found (the implementation provides it). For instance <sys/mmap.h>: not in your project, not in ISO C.
My fictitious <winkle.h> falls into this category. (It deliberately doesn't look like a common platform-specific header coming from any well-known implementation---but that doesn't matter to the point).
> Or are you just maybe saying that the developer's file system isn't part of the ISO C standard, so this whole textual inclusion process is by some meaning not defined by the standard?
Of course, it isn't, no I'm not saying that. The C standard gives requirements as to how a program (project part and other) is processed by the implementation, including all the translation phases that include preprocessing.
To understand what the requirements are, we must consider the content of the program. We know what the content is of the project parts: that's in our files. We (usually indirectly) know the content of the standard headers, from the standard; we ensure that we have met the rules regarding their correct use and what we may or may not rely on coming form them.
We don't know the content of successfully included headers that don't come from our project or from ISO C; or, rather, we don't know that content just from knowing ISO C and our project. In ISO C, we can't find any requirements as to what is supposed to be there, and we can't find it in our project either.
If we peek into the implementation to see what #include <winkle.h> is doing (and such a peeking is usually possible), we are effectively looking at a document, and then if we infer from that document what the behavior will be, it is a documented extension --- standing in the same place as what ISO C calls undefined behavior. Alternatively, we could look to actual documentation. E.g. POSIX tells us what is in <fcntl.h> without us having to look for the file and analyze the tokens. When we use it we have "POSIX-defined" behavior.
#include <winkle.h> is in the same category of thing as __asm__ __volatile__ or __int128_t or what have you.
#include <winkle.h> could contain the token __wipe_current_directory_at_compile_time which the accompanying compiler understands and executes as soon as it parses the token. Or __make_demons_fly_out_of_nose. :)
Do you see the point? When you include a nonstandard header that is not coming from your project, and the include succeeds, anything can happen. ISO C no longer dictates the requirements as to what the behavior will be. Something unexpected can happen, still at translation time.
Now headers like <windows.h> or <unistd.h> are exactly like <winkle.h>: same undefined behavior.
quietbritishjim 7 hours ago [-]
> The "project" is all the files going into a program supplied other than by the implementation.
Most of my most recent comment is addressing the possibility that you meant this.
As I said, there is no such concept to the compiler. It isn't passed any list of files that could be included with #includr, only the .c files actually being compiled, and the directories containing includable files.
The fact that your IDE shows project files is an illusion. Any header files shown there are not treated differently by the compiler/preprocessor to any others. They can't be, because it's not told about them!
It's even possible to add header files to your IDE's project that are not in the include path, and then they wouldn't be picked up by #include. That's how irrelevant project files are to #include.
kazinator 7 hours ago [-]
There is no "compiler", "IDE" or "include path" in the wording of the ISO C standard. A set of files is somehow presented to the implementation in a way that is not specified. Needless to say, a file that is included like "globals.h" but is not the base file of a translation unit will not be indicated to the implementation as the base of a translation unit. Nevertheless it has to be somehow present, if it is required.
It doesn't seem as if you're engaging with the standard-based point I've been making, in spite of detailed elaboration.
> Any header files shown there are not treated differently by the compiler/preprocessor to any others.
This is absolutely false. Headers which are part of the implementation, such as standard-defined headers like <stdlib.h> need not be implemented as files. When the implementation processes #include <stdlib.h>, it just has to flip an internal switch which makes certain identifiers appear in their respective scopes as required.
For that reason, if an implementation provides <winkle.h>, there need not be such a file anywhere in its installation.
gpderetta 18 hours ago [-]
You are basically trying to explain the difference between a conforming program and a strictly conforming one.
safercplusplus 23 hours ago [-]
A couple of solutions in development (but already usable) that more effectively address UB:
i) "Fil-C is a fanatically compatible memory-safe implementation of C and C++. Lots of software compiles and runs with Fil-C with zero or minimal changes. All memory safety errors are caught as Fil-C panics."
"Fil-C only works on Linux/X86_64."
ii) "scpptool is a command line tool to help enforce a memory and data race safe subset of C++. It's designed to work with the SaferCPlusPlus library. It analyzes the specified C++ file(s) and reports places in the code that it cannot verify to be safe. By design, the tool and the library should be able to fully ensure "lifetime", bounds and data race safety."
"This tool also has some ability to convert C source files to the memory safe subset of C++ it enforces"
tialaramex 22 hours ago [-]
Fil-C is interesting because as you'd expect it takes a significant performance penalty to deliver this property, if it's broadly adopted that would suggest that - at least in this regard - C programmers genuinely do prioritise their simpler language over mundane ideas like platform support or performance.
The resulting language doesn't make sense for commercial purposes but there's no reason it couldn't be popular with hobbyists.
eru 21 hours ago [-]
Well, you could also treat Fil-C as a sanitiser, like memory-san or ub-san:
Run your test suite and some other workloads under Fil-C for a while, fix any problems report, and if it doesn't report any problems after a while, compile the whole thing with GCC afterwards for your release version.
safercplusplus 20 hours ago [-]
Right. And of course there are still less-performance-sensitive C/C++ applications (curl, postfix, git, etc.) that could have memory-safe release versions.
But the point is also to dispel the conventional wisdom that C/C++ is necessarily intrinsically unsafe. It's a tradeoff between safety, performance and flexibility/compatibility. And you don't necessarily need to jump to a completely different language to get a different tradeoff.
Fil-C sacrifices some performance for safety and compatibility. The traditional compilers sacrifice some safety for performance and flexibility/compatibility. And scpptool aims to provide the option of sacrificing some flexibility for safety and performance. (Along with the other two tradeoffs available in the same program). The claim is that C++ turns out to be expressive enough to accommodate the various tradeoffs. (Though I'm not saying it's always gonna be pretty :)
eru 6 hours ago [-]
Even with UB holes plugged, C (and C++) are still unsafe, because there are many assumptions you might want to make that you can not encode in the language.
To get an example that's easy to understand: before the introduction of the 'const' keyword, you just couldn't express that some variable should never be changed. And no amount of UB sanitisers would have fixed this for you: you just couldn't express the concept. There's lots of other areas of these languages that are still in a similar state.
Eg there's no way to express that a function should be pure, ie not have side effects (but is allowed to use mutation internally).
safercplusplus 4 hours ago [-]
Yeah, but C++ now supports "user-defined" annotations which effectively allow you to add the equivalent of any keyword you need, right? (Even if it's not the prettiest syntax.) For example, the scpptool static analyzer supports (and enforces) lifetime annotations with similar meaning to Rust's lifetime annotations.
I believe gcc actually does support `__attribute__ ((pure))` to indicate function purity. (I assume it doesn't actually enforce it, but presumably it theoretically could at some point.)
fattah25 1 days ago [-]
Rust here rust there. We are just talking about C not rust. Why we have to using rust. If you talking memory safety why there is no one recommends Ada language instead of rust.
We have zig, Hare, Odin, V too.
ViewTrick1002 1 days ago [-]
> Ada language instead of rust
Because it never achieved mainstream success?
And Zig for example is very much not memory safe. Which a cursory search for ”segfault” in the Bun repo quickly tells you.
More accurately speaking, Zig helps spatial memory safety (e.g. out-of-bound access) but doesn't help temporal memory safety (e.g. use-after-free) which Rust excels at.
pjmlp 22 hours ago [-]
Which is something that even PL/I predating C already had.
ViewTrick1002 23 hours ago [-]
As long as you are using the "releasesafe" build mode and not "releasefast" or "releasesmall".
johnisgood 21 hours ago [-]
> Because it never achieved mainstream success?
And with this attitude it never will. With Rust's hype, it would.
pjmlp 22 hours ago [-]
None of them solve use after free, for example.
Ada would rather be a nice choice, but most hackers love their curly brackets.
the__alchemist 19 hours ago [-]
Even within the rust OSS community it's irritating. They will try to cancel people for writing libs using `unsafe`, and makes APIs difficult to use by wrapping things in multiple layers of traits, then claim using other patters are unsafe/unsound/UB. They make claims that things like DMA are "advanced topics", and "We haven't figured it out yet/found a good solution yet". Love rust/hate the Satefy Inquisition. Or say things like "Why use rust if you don't use all the safety-features and traits"... which belittles rust as a one-trick lang!
laauraa 23 hours ago [-]
>Uninitialized data
They at least fixed this in c++26.
No longer UB, but "erroneous behavior".
Still some random garbage value (so an uninitialized pointer will likely lead to disastrous results still), but the compiler isn't allowed to fuck up your code, it has to generate code as if it had some value.
tialaramex 22 hours ago [-]
It won't be a "random garbage value" but is instead a value the compiler chose.
In effect if you don't opt out your value will always be initialized but not to a useful value you chose. You can think of this as similar to the (current, defanged and deprecated as well as unsafe) Rust std::mem::uninitialized()
There were earlier attempts to make this value zero, or rather, as many 0x00 bytes as needed, because on most platforms that's markedly cheaper to do, but unfortunately some C++ would actually have worse bugs if the "forgot to initialize" case was reliably zero instead.
eru 21 hours ago [-]
What are these worse bugs?
tialaramex 21 hours ago [-]
The classic thing is, we're granting user credentials - maybe we're a login proces, or a remote execution helper - and we're on Unix. In some corner case we forget to fill out the user ID. So it's "random noise". Maybe in the executable distributed to your users it was 0x4C6F6769 because the word "Login" was in that memory in some other code and we never initialized it so...
Bad guys find the corner case and they can now authenticate as user 0x4C6F6769 which doesn't exist and so that's useless. But - when we upgrade to C++ 26 with the hypothetical zero "fix" now they're root instead!
kazinator 23 hours ago [-]
C also fixed it in its way.
Access to an uninitialized object defined in automatic storage, whose address is not taken, is UB.
Access to any uninitialized object whose bit pattern is a non-value, likewise.
Otherwise, it's good: the value implied by the bit pattern is obtained and computation goes on its merry way.
account42 18 hours ago [-]
That's unfortunate.
agalunar 15 hours ago [-]
A small nit: the development of Unix began on the PDP-7 in assembly, not the PDP-11.
(The B language was implemented for the PDP-7 before the PDP-11, which are rather different machines. It’s sometimes suggested that the increment and decrement operators in C, which were inherited from B, are due to the instruction set architecture of the PDP-11, but this could not have been the case. Per Dennis Ritchie:¹
> Thompson went a step further by inventing the ++ and -- operators, which increment or decrement; their prefix or postfix position determines whether the alteration occurs before or after noting the value of the operand. They were not in the earliest versions of B, but appeared along the way. People often guess that they were created to use the auto-increment and auto-decrement address modes provided by the DEC PDP-11 on which C and Unix first became popular. This is historically impossible, since there was no PDP-11 when B was developed. The PDP-7, however, did have a few “auto-increment” memory cells, with the property that an indirect memory reference through them incremented the cell. This feature probably suggested such operators to Thompson; the generalization to make them both prefix and postfix was his own.
Another person puts it this way:²
> It's a myth to suggest C’s design is based on the PDP-11. People often quote, for example, the increment and decrement operators because they have an analogue in the PDP-11 instruction set. This is, however, a coincidence. Those operators were invented before the language [i.e. B] was ported to the PDP-11.
In any case, the PDP-11 usually gets all the love, but I want to make sure the other PDPs get some too!)
In C, using uninitialized data is undefined behavior only if:
- it is an automatic variable whose address has not been taken; or
- the uninitialized object' bits are such that it takes on a non-value representation.
VivaTechnics 4 days ago [-]
We switched to Rust.
Generally, are there specific domains or applications where C/C++ remain preferable? Many exist—but are there tasks Rust fundamentally cannot handle or is a weak choice?
pjmlp 22 hours ago [-]
Yes, all the industries where C and C++ are the industry standards like Khronos APIs, POSIX, CUDA, DirectX, Metal, console devkits, LLVM and GCC implementation,....
Not only you are faced with creating your own wrappers, if no one else has done it already.
The tooling, for IDEs and graphical debuggers, assumes either C or C++, so it won't be there for Rust.
Ideally the day will come where those ecosystems might also embrace Rust, but that is still decades away maybe.
bluetomcat 23 hours ago [-]
Rust encourages a rather different "high-level" programming style that doesn't suit the domains where C excels. Pattern matching, traits, annotations, generics and functional idioms make the language verbose and semantically-complex. When you follow their best practices, the code ends up more complex than it really needs to be.
C is a different kind of animal that encourages terseness and economy of expression. When you know what you are doing with C pointers, the compiler just doesn't get in the way.
eru 21 hours ago [-]
Pattern matching should make the language less verbose, not more. (Similar for many of the other things you mentioned.)
> When you know what you are doing with C pointers, the compiler just doesn't get in the way.
Alas, it doesn't get in the way of you shooting your own foot off, too.
Rust allows unsafe and other shenanigans, if you want that.
bluetomcat 20 hours ago [-]
> Pattern matching should make the language less verbose, not more.
In the most basic cases, yes. It can be used as a more polished switch statement.
It's the whole paradigm of "define an ad-hoc Enum here and there", encoding rigid semantic assumptions about a function's behaviour with ADTs, and pattern matching for control-flow. This feels like a very academic approach and modifying such code to alter its opinionated assumptions isn't funny.
eru 6 hours ago [-]
How is encoding all the assumptions and invariants badly in eg a bunch of booleans and nullable pointers any better?
za_creature 16 hours ago [-]
> When you know what you are doing with C pointers, the compiler just doesn't get in the way.
Tell me you use -fno-strict-aliasing without telling me.
Advantages of C are short compilation time, portability, long-term stability, widely available expertise and training materials, less complexity.
IMHO you can today deal with UB just fine in C if you want to by following best practices, and the reasons given when those are not followed would also rule out use of most other safer languages.
simonask 22 hours ago [-]
This is a pet peeve, so forgive me: C is not portable in practice. Almost every C program and library that does anything interesting has to be manually ported to every platform.
C is portable in the least interesting way, namely that compilers exist for all architectures. But that's where it stops.
snovymgodym 18 hours ago [-]
> C is not portable in practice. Almost every C program and library that does anything interesting has to be manually ported to every platform.
I'm guessing you mean that every cross-platform C codebase ends up being plastered in cascading preprocessor code to deal with OS and architecture differences. Sure that's true, you still have to do some porting work regardless of the language you chose.
But honestly, is there any language more portable than C? I struggle to come up with one.
If someone told me "I need a performant language that targets all major architectures and operating systems, but also maybe I want to run it on DOS, S390X, an old Amiga I have in my closet, and any mystery-meat microcontroller I can find." then really wouldn't have a better answer for them than C89.
If C isn't portable then nothing is.
simonask 17 hours ago [-]
If "portability" to you has to include incredibly esoteric architectures in 2025, then what C has to offer is probably the best you can do, but my point is it doesn't do any better on mainstream platforms either.
If you are targeting any recent platform, both Rust and Zig do what you want.
pjmlp 20 hours ago [-]
Back in the 2000's I had lots of fun porting code across several UNIX systems, Aix, Solaris, HP-UX, Red-Hat Linux.
A decade earlier I also used Xenix and DG/UX.
That is a nice way to learn how "portable" C happens to be, even between UNIX systems, its birthplace.
uecker 21 hours ago [-]
Compilers existing is essential and not trivial (and also usually then what other languages build on). The conformance model of C also allows you to write programs that are portable without change to different platforms. This is possible, my software runs on 20 different architectures without change. That one can then also adopt it to make use of specific features of different platforms is quite natural in my opinion.
simonask 17 hours ago [-]
It is essential and nontrivial, but it's also the extremely bare minimum.
You cannot write portable code without platform-specific and even environment-specific adaptations, like handling the presence of certain headers (looking at you, stdint.h and stddef.h), and let's not even start about interacting with the OS in any way.
uecker 17 hours ago [-]
There may be platforms that are not conforming to the C standard. But I doubt those then have comprehensive implementations of other languages either.
lifthrasiir 24 hours ago [-]
> short compilation time
> IMHO you can today deal with UB just fine in C if you want to by following best practices
In the other words, short compilation time has been traded off with wetware brainwashing... well, adjustment time, which makes the supposed advantage much less desirable. It is still an advantage, I reckon though.
uecker 21 hours ago [-]
I do not understand what you are tying to say, but it seems to be some hostile rambling.
lifthrasiir 20 hours ago [-]
Never meant to be hostile (if I indeed were, I would have question every single word), but sorry for that.
I mean to say that best practices do help much but learning those best practices take much time as well. So short compilation time is easily offseted by learning time, and C was not even designed to optimize compilation time anyway (C headers can take a lot to parse and discard even when unused!). Your other points do make much more sense and it's unfortunate that first points are destructively interfering each other, hence my comment.
uecker 17 hours ago [-]
Sorry, maybe I misread your comment. There are certainly languages easier to learn than C, but I would not say C++ or Rust fall into this category. At the same time, I find C compilation extremely fast exactly because of headers. In C you can split interface and implementation cleanly between header and c-file and this enables efficient incremental builds. In C++ most of the implementation is in headers, and all the template processing is order of magnitude more expensive than parsing C headers. Rust also does not seem to have proper separate compilation.
steveklabnik 16 hours ago [-]
> I find C compilation extremely fast exactly because of headers.
The header model is one of the parts that makes compiling C slower than it could be. This doesn't mean that it is slow, but it's fast in spite of headers, not because of them.
> In C you can split interface and implementation cleanly between header and c-file and this enables efficient incremental builds.
That's not what does, it is the ability to produce individual translation units as intermediary files.
> Rust also does not seem to have proper separate compilation.
Rust does separate compilation, and also has efficient incremental builds. Header files are not a hard requirement for this.
uecker 12 hours ago [-]
If you say the header model makes it slower than it could be, you need to compare it to something. I do not see how it causes significant slow downs in C projects (in contrast to C++). And yes, I wrote compilers and (incomplete) preprocessors. I do not understand what you mean by your second point. What separation of interface and implementation allows you to do is updating the implementation without having to recompile other TUs. You can achieve this is also in different ways, but in C this works by in this way.
I am not sure how it works in Rust as you need to monomorphize a lot of things, which come from other crates. It seems this would inevitably entangle the compilations.
steveklabnik 10 hours ago [-]
> I do not see how it causes significant slow downs in C projects
It's that textual inclusion is just a terrible model. You end up reprocessing the same thing over and over again, everywhere it is used. If you #include<foo.h> 100 times, the compiler has to reparse those contents 100 times. Nested headers end up amplifying this effect. It's also at a file-level granularity, if you change a header, every single .c that imports it must be recompiled, even if it didn't use the thing that was changed. etc etc. These issues are widely known.
> I do not understand what you mean by your second point. What separation of interface and implementation allows you to do is updating the implementation without having to recompile other TUs.
Sure, but you don't need to have header files to do this. Due to issues like the above, they cause more things to be recompiled than necessary, not less.
> You can achieve this is also in different ways, but in C this works by in this way.
Right, my point is, those other ways are better.
> I am not sure how it works in Rust as you need to monomorphize a lot of things, which come from other crates. It seems this would inevitably entangle the compilations.
The fact that there are "other crates" is because Rust supports separate compilation: each crate is compiled independently, on its own.
The rlib contains the information that, when you link two crates together, the compiler can use for monomorphization. And it's true that monomorphization can cause a lot of rebuilding.
But to be clear, I am not arguing that Rust compilation is fast. I'm arguing that C could be even faster if it didn't have the preprocessor.
uecker 1 hours ago [-]
> It's that textual inclusion is just a terrible model. You end up reprocessing > the same thing over and over again, everywhere it is used.
One could certainly store the interfaces in some binary format, but is it really worth it? This would also work with headers by using a cache, but nobody does it for C because there is not much to gain. Parsing is fast anyhow, and compilers are smart enough not to look at headers multiple times when protected by include guards. According to some quick measurements, you could save a couple of percent at most.
The advantages of headers is that they are simple, transparent, discoverable, and work with outside tools in a modular way. This goes against the trend of building frameworks that tie everything together in a tightly integrated way. But I prefer the former. I do not think it is a terrible model, quite the opposite. I think it is a much better and nicer model.
pizza234 23 hours ago [-]
Yes, based on a few attempts chronicled in articles from different sources, Rust is a weak choice for game development, because it's too time-consuming to refactor.
Basically all of those problems originate with the tradition of conflating pointers and object identity, which is a problem in Rust as soon as you have ambiguous ownership or incongruent access patterns.
It's also very often not the best way to identify objects, for many reasons, including performance (spatial locality is a big deal).
These problems go away almost completely by simply using `EntityID` and going through `&mut World` for modifications, rather than passing around `EntityPtr`. This pattern gives you a lot of interesting things for free.
bakugo 21 hours ago [-]
The video I linked to is long but goes through all of this.
Pretty much nobody writing games in C++ uses raw pointers in entities to hold references to other related entities, because entities can be destroyed at any time and there's no simple way for a referring entity to know when a referenced entity is destroyed.
Using some sort of entity ID or entity handle is very common in C++, the problem is that when implementing this sort of system in Rust, developers often end up having to effectively "work around" the borrow checker, and they end up not really gaining anything in terms of correctness over C++, ultimately defeating the purpose of using Rust in the first place, at least for that particular system.
simonask 17 hours ago [-]
Can you give an example of what problems need workarounds here?
The benefits seem pretty massive, at least on the surface. For example, you can run any system that only takes `&World` (i.e., immutable access) in parallel without breaking a sweat.
ramon156 23 hours ago [-]
We've only had 6-7 years of hame dev in rust. Bevy is coming along nicely and will hopefully remove these pain points
flohofwoe 22 hours ago [-]
"Mit dem Angriff Steiner's wird das alles in Ordnung kommen" ;)
As shitty as C++ is from today's PoV, the entire gaming industry switched over within around 3 years towards the end of the 90s. 6..7 years is a long time, and a single engine (especially when it's more or less just a runtime without editor and robust asset pipeline) won't change the bigger picture that Rust is a pretty poor choice for gamedev.
eru 21 hours ago [-]
> As shitty as C++ is from today's PoV, the entire gaming industry switched over within around 3 years towards the end of the 90s.
Did they? What's your evidence? Are you including consoles?
Btw, the alternatives in the 1990s were worse than they are now, so the bar to clear for eg C or C++ were lower.
flohofwoe 21 hours ago [-]
I was there Gandalf... ;) Console SDKs offering C or C++ APIs doesn't really matter, because you can call C APIs from C++ just fine. So the language choice was a team and engine developer decision, not a platform owner decision (as it should be).
From what I've seen, around the late mid-90's, C++ usage was still rare, right before 2000 it was already common and most middleware didn't even offer C APIs anymore.
Of course a couple of years later Unity arrived and made the gamedev language choice more complicated again.
pjmlp 20 hours ago [-]
As another Gandalf, Playstation 2 was the very first console to actually offer proper C++ tooling.
That would be 2000, until then Sega, Nintendo and Playstion only had C and Assembly SDKs, even the Playstation Yaroze for hobbists did get released only with C and Assembly support.
PC was naturally another matter, especialy with Watcom C/C++.
eru 21 hours ago [-]
> I was there Gandalf... ;)
You were at most in one place. My question was rather, which corners of the industry are you counting?
However you are right that one of the killer features of C++ was that it provided a pretty simple upgrade path from C to (bad) C++.
It's not just API calls. You can call C APIs from most languages just fine.
flohofwoe 21 hours ago [-]
My corner of the industry back then was mostly PC gamedev with occasional exploration of game consoles (but only starting with the OG Xbox. But that doesn't really matter much since it was obvious that the entire industry was very quickly moving to C++ (we had internet back then after all in my corner of the wood, as well as gamedev conferences to feel the general vibe).
id Software was kinda famous for being the last big C holdout, having only switched to C++ with Doom 3, and development of Doom 3 started in late 2000.
account42 18 hours ago [-]
And there are millions of game engines written in C++. Many of them have also been coming along nicely for years.
Making a nontrivial game with them is a wholly different story.
mgaunard 23 hours ago [-]
Rust forces you to code in the Rust way, while C or C++ let you do whatever you want.
nicoburns 22 hours ago [-]
> C or C++ let you do whatever you want.
C and C++ force you to code in the C and C++ ways. It may that that's what you want, but they certainly dont let me code how I want to code!
mgaunard 19 hours ago [-]
There is no C or C++ ways. It's widely known that every codebase is its own dialect.
nicoburns 19 hours ago [-]
There are lots of C and particularly C++ ways, but you're still restricted. Want to use methods in C: nope, you can't. Want language-level tagged unions and pattern matching in either language: nope. Same for guaranteed tail call optimisation and a bunch of other things.
This is especially true for C which supports almost nothing (it doesn't even have a sensible array type!). But is also true for C++: while it supports a lot, it doesn't support everything.
mgaunard 16 hours ago [-]
The funny part is that all of these things are easy to achieve as libraries/paradigms.
Methods in C, just have function pointers as members. Common in many codebases.
Guaranteed tail calls, all the compilers guarantee that function calls that are a return expression are tail calls.
Tagged union in C++, it's trivial as a library, see std::variant for a bad example of it, and all the various monadic/pattern-matching variants (pun intended) people have written. C is at a disadvantage here due to lack of lambdas, but I'm sure people have built stuff using some GCC extensions.
bigfishrunning 17 hours ago [-]
what changes, in your opinion, would need to be made to the C array type to make it "sensible"? C's array is simplistic, but I don't think it's not "sensible"...
nicoburns 10 hours ago [-]
It would need to store a length and not decay to a pointer when passed to a function.
mgaunard 16 hours ago [-]
consider the C++ std::array, which exists to make arrays behave like normal objects.
You can do the same in C by wrapping your array in a struct.
mckravchyk 15 hours ago [-]
If you wanted to develop a cross-platform native desktop / mobile app in one framework without bundling / using a web browser, only QT comes to mind, which is C++. I think there are some bindings though.
jandrewrogers 15 hours ago [-]
An application domain where C++ is notably better is when the ownership and lifetimes of objects are not knowable at compile-time, only being resolvable at runtime. High-performance database kernels are a canonical example of code where this tends to be common.
Beyond that, recent C++ versions have much more expressive metaprogramming capability. The ability to do extensive codegen and code verification within C++ at compile-time reduces lines of code and increases safety in a significant way.
m-schuetz 21 hours ago [-]
Prototyping in any domain. It's nice to do some quick&dirty way to rapidly evaluate ideas and solutions.
eru 21 hours ago [-]
I don't think C nor C++ were ever great languages for prototyping? (And definitely not better than Rust.)
m-schuetz 16 hours ago [-]
Please try not to be obnoxious and turn this into a language war.
eru 6 hours ago [-]
How is this obnoxious?
C and C++ have their strengths, but rapid prototyping is generally not seen to be amongst them.
This shouldn't be any more controversial than saying that pure Python is generally slow.
m-schuetz 3 hours ago [-]
They are pretty much the best choice for prototyping 3D apps and GPU algorithms. They're fast, powerful, and don't impose restrictions - you can do whatever and however. It also helps that CUDA is C++.
imadr 4 days ago [-]
I haven't used Rust extensively so I can't make any criticism besides that I find compilation times to be slower than C
ost-ing 1 days ago [-]
I find with C/++ I have to compile to find warnings and errors, while with Rust I get more information automatically due to the modern type and linking systems. As a result I compile Rust significantly less times which is a massive speed increase.
Rusts tooling is hands down better than C/++ which aids to a more streamlined and efficient development experience
bch 24 hours ago [-]
> Rusts tooling is hands down better than C/++ which aids to a more streamlined and efficient development experience
Would you expand on this? What was your C tooling/workflow that was inferior to your new Rust experience?
simonask 22 hours ago [-]
Not the GP, but the biggest one is dependency management. Cargo is just extremely good.
As for the language tooling itself, static and runtime analyzers in C and C++ (and these are table stakes at this point) do not come close to the level of accuracy of the Rust compiler. If you care about writing unsafe code, Miri is orders of magnitude better at detecting UB than any runtime analyzer I've seen for C and C++.
uecker 17 hours ago [-]
I do not think package management should be done at the level of programming languages.
johnisgood 21 hours ago [-]
Pacman is extremely good, too, for C. :)
simonask 17 hours ago [-]
Pacman solves a different problem. Cargo manages your project's dependencies, not system packages.
johnisgood 17 hours ago [-]
I know, but often that is all you need for C.
kazinator 23 hours ago [-]
The popular C compilers are seriously slow, too. Orders of magnitude compared to C compilers of yesteryear.
ykonstant 24 hours ago [-]
I also hear that Async Rust is very bad. I have no idea; if anyone knows, how does async in Rust compare to async in C++?
ViewTrick1002 23 hours ago [-]
> I also hear that Async Rust is very bad.
Not sure where this is coming from.
Async rust is amazing as long as you only mix in one more hard concept. Be it traits, generics or whatever. You can confidently write and refactor heavily multithreaded code without being deathly afraid of race conditions etc. and it is extremely empowering.
The problem comes when trying to write async generic traits in a multithreaded environment.
Then just throwing stuff at the wall and hoping something sticks will quickly lead you into despair.
01HNNWZ0MV43FF 24 hours ago [-]
I am yet to use async in c++, but I did work on a multi threaded c++ project for a few years
Rust is nicer for async and MT than c++ in every way. I am pretty sure.
But it's still mid. If you use Rust async aggressively you will struggle with the borrow checker and the architecture results of channel hell.
If you follow the "one control thread that does everything and never blocks" you can get far, but the language does not give you much help in doing that style neatly.
I have never used Go. I love a lot of Go projects like Forgejo and SyncThing. Maybe Go solved async. Rust did not. C++ did not even add good tagged unions yet.
eru 21 hours ago [-]
Go (at least before generics) was really annoying to use.
Doing anything concurrent in Go is also really annoying (be that async or with threads), because everything is mutable. Not just by default but always. So anything shared is very dangerous.
ykonstant 23 hours ago [-]
Thanks for the info!
teunispeters 18 hours ago [-]
embedded hardware, any processor Rust doesn't support (there are many), and any place where code size is critical. Rust has a BIG base size for an application, uselessly so at this time. I'd also love to see if it offered anything that could be any use in those spaces - especially where no memory allocation takes place at all. C (and to a lesser extent C++) are both very good in those spaces.
steveklabnik 17 hours ago [-]
You can absolutely make small rust programs, you just have to actually configure things the right way. Additionally, the Rust language doesn’t have allocation at all, it’s purely a library concern. If you don’t want heap allocations, then don’t include them. It works well.
The smallest binary rustc has produced is like ~145 bytes.
teunispeters 17 hours ago [-]
That is far from my only concern. But it's good to see Rust is finally paying attention to binary sizes. And the overwhelming complexity of rust code is definitely not a gain when one is working in embedded spaces anyway. I am however really REALLY annoyed with the aggressive sales tactics of the rust community.
steveklabnik 16 hours ago [-]
> But it's good to see Rust is finally paying attention to binary sizes.
Just to be clear, this isn't a recent development, it has been this way for many years at this point.
eru 21 hours ago [-]
> Generally, are there specific domains or applications where C/C++ remain preferable?
Well, anything were your people have more experience in the other language or the libraries are a lot better.
mrheosuper 24 hours ago [-]
Rust can do inline ASM, so finding a task Rust "fundamentally cannot handle" is almost impossible.
eru 21 hours ago [-]
That's almost as vacuous as saying that Rust can implement universal Turing machines are that Rust can do FFI?
agent327 16 hours ago [-]
I, once again, disagree with the premise that UB is a necessary precondition for optimisation, or that it exists to allow for optimisation. You do not need UB to unroll a loop, inline a function, lift an object or computation out of a loop, etc. Moreover, _most_ UB does not assist in optimisation at all.
The two instances where UB allows for optimisation are as follows:
1. The 'signed overflow' UB allows for faster array indexing. By ignoring potential overflow, the compiler can generate code that doesn't check for accidental overflow (which would require masking the array index, recomputing the address on each loop iteration). I believe the better solution here would be to introduce a specific type for iterating over arrays that will never overflow; size_t would do fine, and making signed overflow at least implementation defined, if not outright fully defined, after a suitable period during which compilers warn if you use a too-small type for array indexing.
2. The 'aliasing' UB does away with the need to read/write values to/from memory each time they're used, and is extremely important to performance optimisation.
But the rest? Most of it does precisely nothing for performance. At 'best', the compiler uses detected UB to silently eliminate code branches, but that's something to be feared, not celebrated. It isn't an optimisation if it removes vital program logic, because the compiler could 'demonstrate' that it could not possibly take the removed branch, on account of it containing UB.
The claim in the linked article ("what every C programmer should know") that use of uninitialized variables allows for additional optimisation is incorrect. What it does instead is this: if the compiler see you declare a variable, and then reading from it before writing to it, it has detected UB, and since the rule is that "the compiler is allowed to assume UB does not occur", use that as 'evidence' that that code branch will never occur and can be eliminated. It does not make things go faster; it makes them go _wrong_.
Undefined behaviour, ultimately, exists for many reasons: because the standards committee forgot a case, because the underlying platforms differ too wildly, because you cannot predict in advance what the result of a bug may be, to grandfather in broken old compilers, etc. It does not, in any way, shape, or form, exist _in order to_ enable optimisation. It _allows_ it in some cases, but that is, and never was, not the goal.
Moreover, the phrasing of "the compiler is allowed to assume that UB does not occur" was originally only meant to indicate that the compiler was allowed to emit code as if all was well, without introducing additional tests (for example, to see if overflow occurred or if a pointer was valid) - clearly that would be very expensive or downright infeasible. Unfortunately, over time this has enabled a toxic attitude to grow that turns minor bugs into major disasters, all in the name of 'performance'.
The two bullet points towards the end of the article are both true: the compiler SHOULD NOT behave like an adversary, and the compiler DOES NEED license to optimize. The mistake is thinking that UB is a necessary component of such license. If that were true, a language with more UB would automatically be faster than one with less. In reality, C++ and Rust are roughly identical in performance.
IshKebab 20 hours ago [-]
This asserts that UB was deliberately created for optimisation purposes; not to handle implementation differences. It doesn't provide any evidence though and that seems unlikely to me.
The spec even says:
> behavior, upon use of a nonportable or erroneous program construct or of erroneous data, for which this International Standard imposes no requirements
No motivation is given that I could find, so the actual difference between undefined and implementation defined behaviour seems to be based on whether the behaviour needs to be documented.
flohofwoe 20 hours ago [-]
I'd say the original intent of UB was not the sort of "optimizer exploits" we see today, but to allow wiggle room for supporting vastly different CPUs without having to compromise runtime performance or increasing compiler complexity to balance performance versus correctness. Basically an escape hatch for compilers. The difference to IB also has always been quite fuzzy.
Also the C spec has always been a pragmatic afterthought, created and maintained to establish at least a minimal common feature set expected of C compilers.
The really interesting stuff still only exists outside the spec in vendor language extensions.
pizlonator 23 hours ago [-]
I don’t buy the “it’s because of optimization argument”.
And I especially don’t buy that UB is there for register allocation.
First of all, that argument only explains UB of OOB memory accesses at best.
Second, you could define the meaning of OOB by just saying “pointers are integers” and then further state that nonescaping locals don’t get addresses. Many ways you could specify that, if you cared badly enough. My favorite way to do it involves saying that pointers to locals are lazy thunks that create addresses on demand.
OskarS 18 hours ago [-]
No, it's absolutely because of optimization. For instance, C++20 defined signed integer representation as having two's complement, but signed integer overflow is still undefined behaviour. The reason is that if you compile with flags that make it defined, you lose a few percentage points of performance (primarily from preventing loop unrolling and auto-vectorization).
Same thing with e.g. strict aliasing or the various UB that exists in the standard library. For instance, it's UB to pass a null pointer to strlen. Of course, you can make that perfectly defined by adding an `if` to strlen that just returns 0. But then you're adding a branch to every strlen, and C is simply not willing to do that for performance reasons, so they say "this is UB" instead.
Pretty much instance of UB in standard C or C++ is because making it defined would either hamper the optimizer, or it would make standard library functions slower. They don't just make things UB for fun.
pizlonator 17 hours ago [-]
This isn’t the reason why the UB is in the spec in the first place. The spec left stuff undefined to begin with because of lack of consensus over what it should do.
For example the reason why 2s complement took so long is because of some machine that ran C that still existed that was 1s complement.
> The reason is that if you compile with flags that make it defined, you lose a few percentage points of performance (primarily from preventing loop unrolling and auto-vectorization).
I certainly don’t lose any perf on any workload of mine if I set -fwrapv
If your claim is that implementers use optimization as the excuse for wanting UB, then I can agree with that.
I don’t agree that it’s a valid argument though. The performance wins from UB are unconvincing, except maybe on BS benchmarks that C compilers overtune for marketing reasons.
OskarS 17 hours ago [-]
> For example the reason why 2s complement took so long is because of some machine that ran C that still existed that was 1s complement.
You're misunderstanding me: as of C++20, there is no other representation in C++ for signed integers other than two's complement (no signed ones' complement, no signed magnitude, nothing else), but signed overflow is still UB. It's not because of obscure machines or hardware, such hardware is not relevant for C++20 and later. The reason for it is performance. From the accepted paper [1]:
> The following polls were taken, and corresponding modifications made to the paper. The main change between [P0907r0] and the subsequent revision is to maintain undefined behavior when signed integer overflow occurs, instead of defining wrapping behavior. This direction was motivated by:
> * Performance concerns, whereby defining the behavior prevents optimizers from assuming that overflow never occurs
You may disagree, you may think they're wrong, but their motivation is performance, that's why this is UB. It's right there in black and white. This was C++, not C, but it's not at all unthinkable that the C standard will also mandate two's complement at some point, and if they do, they almost certainly keep signed overflow undefined for exactly the same reason.
It's not hard to write code that optimizes much better when you use signed loop variables. One of my favorite examples is this function [2] to turn a 3D mesh inside out by flipping the edges of each triangle in a triangle mesh. The godbolt link has two versions of the same function, one with a signed loop variable, one with an unsigned one. The signed one auto-vectorizes and optimizes much better because it can assume that the loop variable never overflows (this version is C++, it's trivial to rewrite it in C and get the same results).
I agree that the stated motivation for continuing to keep UB is performance.
I know that this is misguided based on my own perf tests and others’ perf tests.
Also, it’s wrong to say flat out that UB on signed ints is somehow necessary for perf when even a simple perf test shows that it just doesn’t matter, and the optimization it enables is quite obscure.
account42 18 hours ago [-]
I wish there was a way to opt into undefined behavior for unsigned overflow. Its rare that wraparound is actually what you want and in many cases overflow is still a bug. Sucks to have to either miss out on potential optimizations or miss out on the guarantee that the value can't be negative.
You do need some way to overflow properly, because sometimes that is what you want. A common example would be PRNGs, which frequently rely on overflow (the classic LCG, for instance). You could argue that should just be a library function or something (e.g. `add_with_overflow`), though that's more C++ than C.
You are absolutely, 100% correct though: I've never seen a case where accidental overflow doesn't start causing bugs anyway. Like, the Pac-Man kill screen is caused by a byte overflowing (it happens on level 256), and the game goes insane. Pac-Man was written in assembly where overflow is defined behavior, but that doesn't matter at all, the game is still broken. If signed overflow is essentially always a bug anyway, why not make it UB and optimize around it? Especially since it is super-valuable in being able to unroll loops.
People always bring up signed integer overflow as an argument for why UB is scary, and it always seemed like such a bad argument to me. Like, I can understand why people think UB has gone too far in C/C++, but signed overflow is such a bad example. It's one of the most sensible bits of UB in the entire standard, IMHO.
pizlonator 17 hours ago [-]
-fwrapv
j16sdiz 21 hours ago [-]
> First of all, that argument only explains UB of OOB memory accesses at best.
It explains many loop-unroll and integer overflow as well.
gpderetta 21 hours ago [-]
> nonescaping locals don’t get addresses
inlining, interprocedural optimizations.
For example, something as an trivial accessor member function would be hard to optimize.
pjmlp 20 hours ago [-]
Safer languages manage similar optimizations without having to rely on UB.
gpderetta 20 hours ago [-]
Well, yes, safer languages prevent pointer forging statically, so provenance is trivially enforced.
And I believe that provenance is an issue in unsafe rust.
tialaramex 16 hours ago [-]
Unlike C++ and (until Martin's work is moved to the actual language ISO document rather than separate) C the Rust language actually has a definition for how provenance is supposed to work.
The definition isn't deemed complete because of aliasing. AIUI The definition we have is adequate if you're OK with treating all edge cases for "Is this an alias?" as "Yes" but eventually Rust will also need to carefully nail down all those edge cases so that you can tread closer without falling off.
pizlonator 17 hours ago [-]
Inlining doesn’t require UB
gpderetta 17 hours ago [-]
I didn't claim that. What I mean is that if a pointer escapes into an inlined function and no further, it will still prevent further optimizations if we apply your rule that only non-escaping locals don't get addresses. The main benefit of inlining is that it is effectively a simple way to do interprocedurally optimizations. I.e.
inline void add(int* to, int what) { *to += what; }
void foo();
void bar() {
int x = 0;
add(&x, 1);
foo();
return x;
}
By your rules, optimizing bar to return the constant 1 would not be allowed.
pizlonator 17 hours ago [-]
I think you’re applying a very strange strawman definition to “nonescaping”. It’s certainly not the definition I would pick.
The right definition is probably something like:
- pointers that come out of the outside world (syscalls) are escaped. They are just integers.
- pointers to locals have provenance. They point to an abstract location. It is up to the implementation to decide when the location gets an integer value (is in an actual address) and what that value is. The implementation must do this no later than when the pointer to the local escapes.
- pointer values passed to the outside world (syscalls) escape.
- pointer values stored in escaped memory also escape, transitively
That’s one possible definition that turns the UB into implementation defined behavior. I’m sure there are others
gpderetta 16 hours ago [-]
I think you have a non-standard definition. An escaping pointer is an address that the compiler cannot fully track (directly or indirectly). It could be to a syscall, it could be a separately compiled function (without LTO), it could even be to a function in the same translation unit if the compiler cannot inline that function nor do sufficient intraprocedural analysis.
Again, I'm not a compiler writer, but my understanding is that non escaping variables can be optimized in SSA form, escaped variables are otherwise treated as memory and the compiler must be significantly more conservative.
In any case, whether a pointer escapes or not depends purely on the compiler capabilities and optimization level, so it would not be sane making a code well defined or UB depending on the compiler or optimization level.
edit: to be more concrete, do you think that in my example the constant folding of the return into return 1 should be allowed? And if so, which variant of this code would prevent the optimization and why?
pizlonator 12 hours ago [-]
> Again, I'm not a compiler write
I am a compiler writer.
The definition I gave in my post is general enough to cover all possible compilers (ones that have LTO, ones that are inside a DBT, etc).
Yes the constant folding should be allowed because the pointer to the local never escaped.
tialaramex 21 hours ago [-]
> Second, you could define the meaning of OOB by just saying “pointers are integers"
This means losing a lot of optimisations, so in fact when you say you "don't buy" this argument you only mean that you don't care about optimisation. Which is fine, but this does mean the "improved" C isn't very useful in a lot of applications, might as well choose Java.
pizlonator 17 hours ago [-]
> This means losing a lot of optimisations
You won’t lose “a lot” of optimizations and you certainly won’t lose enough for it to make a noticeable difference in any workload that isn’t SPEC
roman_soldier 21 hours ago [-]
Just use Zig, it fixes all this
grougnax 20 hours ago [-]
Worse languages ever.
compiler-guy 20 hours ago [-]
Jack Sparrow: “… but you have heard of them.”
The dustbin of programming languages is jam packed with elegant, technically terrific, languages that never went anywhere.
OskarS 20 hours ago [-]
C and C++ are languages that brought us UNIX, the Linux kernel, macOS and Windows, the interpreters of virtually every other language in the world, powering virtually all software in the world as well as the vast majority of embedded devices.
Chill the fuck out.
account42 18 hours ago [-]
Except for all the others.
Rendered at 07:48:12 GMT+0000 (Coordinated Universal Time) with Vercel.
Fixing the document is worthwhile, and certainly a reminder that WG21's equivalent effort needs to make the list before it can even begin that process on its even longer document, but practical C programmers don't read the document and since this UB was a "ghost" they weren't tripped by it. Removing items from the list this way does not translate to the meaningful safety improvement you might imagine.
There's not a whole lot of movement there towards actually fixing the problem. Maybe it will come later?
But the original article also complains about the number of trivial UB.
I would strongly suspect that C compiler implementers very much do read the document, though. Which, as far as I can see, means "ghosts" could easily become actual UB (and worse, sneaky UB that you wouldn't expect.)
It's like one of those tricky self-referential parlor box statements. "The statement on this box is not true"? Thanks I guess. But that's a game, the puzzles are supposed to be like that, whereas the mission of the ISO document was not to confuse people, so it's good that it is being improved.
One huge thing they have on their side is that their implementation is concrete. Whatever it is that, say, GCC does is de facto actually a thing a compiler can do. The standards bodies (and WG21 has been worse by some margin, but they're both guilty) may standardize anything, but concretely the compiler can only implement some things. "Just do X" where X isn't practical works fine on paper but is not implementable. This was the fate of the Consume ordering. Consume/ Release works fine on paper, you "just" need to have whole program analysis to implement it. Well of course that's not practical so it's not implemented.
But then, they also sometimes misread the standard in ways I can't really understand. This often can be seen when the "interpretation" changes over time. Earlier compilers (or even earlier parts of the same compiler) implement the standard as written, some new optimization pass has some creative interpretation.
The compiler bug I'm most closely following, and which I suspect you have your eye on too is: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=119472 aka https://github.com/rust-lang/rust/issues/107975 https://github.com/llvm/llvm-project/issues/45725
But it seems like it's just that everybody fucked this up in similar ways, that's two different major compiler backends! I wouldn't be surprised if Microsoft (whose code we can't see) find that they don't get this quite right either.
https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p14...
i.e., the UB already existed, but it was not explicit had to be inferred from the whole text and the boundaries were fuzzy. Remember that anything not explicitly defined by the standard, is implicitly undefined.
Also remember, just because you can legally construct a pointer it doesn't mean it is safe to dereference.
I have an execution environment, Wasm, where doing this is pretty well defined, in fact. So if I want to read the memory at address 12345, which is within bounds of the linear memory (and there's a builtin to make sure), why should it be undefined behavior?
And regarding pointer provenance, why should going through a pointer-to-integer and integer-to-pointer conversions try to preserve provenance at all, and be undefined behavior in situations where that provenance is ambiguous?
The reason I'm using integer (rather than pointer) arithmetic is precisely so I don't have to be bound by pointer arithmetic rules. What good purpose does it serve for this to be undefined (rather than implementation defined) beyond preventing certain programs to be meaningfully written at all?
I'm genuinely curious.
And I'm taking about both things.
Integer arithmetic that produces pointers that are just out of bounds of an object. Why can't this work? Why can't the compiler assume that, since I explicitly converted a pointer to an integer, the pointed-to object can't be put into a register, or made to go out or scope early?
Second, fabricating pointers. If I have a pointer to mmap/sbrk memory, shouldn't I be allowed to “fabricate” arbitrary pointers from integers that point into that area? If not, why not?
Finally Wasm. The linear memory is addressable from address 0 to __builtin_wasm_memory_size * PAGESIZE. Given this, and except maybe the address at zero, why should it be undefined behavior to dereference any other address?
What's the actual advantage to making these undefined behavior? What to we gain in return?
How would you define it? Especially in a way that is consistent with the rest of the language and allows common optimizations (remember that C supports variables, which may or may not be stored in memory)?
It's the only way to interpret *(uint64_t*)(12345) when the standard says that a integer-to-pointer conversion is "intended to be consistent with the addressing structure of the execution environment".
There exists an instruction to do that load in Wasm, there's a builtin to check that 12345 points to addressable memory, the load is valid at the assembly level, the standard says the implementation should define this to be consistent with the addressing structure of the execution environment, why the heck are we playing games and allowing the compiler to say, "nope, that's not valid, so your entire program is invalid, and we can do what ever we want, no diagnostic required"?
For what it’s worth there’s a reason you’re supposed to do this kind of access through memcpy, not by dereferencing made up pointers.
> There exists an instruction to do that load in Wasm, there's a builtin to check that 12345 points to addressable memory, the load is valid at the assembly level, the standard says the implementation should define this to be consistent with the addressing structure of the execution environment, why the heck are we playing games and allowing the compiler to say, "nope, that's not valid, so your entire program is invalid, and we can do what ever we want, no diagnostic required"?
Because the language standard is defined to target a virtual machine as output, not any given implementation. That virtual machine is then implemented on various platforms, but the capabilities of the underlying system aren’t directly accessible - they are only there to implement the C virtual machine. That’s why C can target so many different target machines.
But converting an integer to a pointer creates a problem if you allow that pointer to point to anything--it breaks all of the optimizations that assumed they could trace all of the uses of an address. So you need something like provenance to say that certain back-conversions are illegal. The most permissive model is a no-address-taken model (you can't forge a pointer to a variable whose address was never taken). But most compilers opt instead for a data-dependency-based model: essentially, even integer-based arithmetic of addresses aren't allowed to violate out-of-bounds at the point of dereference. Or at least, they claim to--the documentation for both gcc and llvm have this claim, but both have miscompilation bugs because they don't actually allow this.
The proposal for pointer provenance in C essentially looks at how compilers generally implement things and suggests a model that's closer to their actual implementation: pointer-to-integer exposes the address such that any integer-to-pointer can point to it. Note this is more permissive than the claimed models of compilers today--you're explicitly able to violate out-of-bounds rules here, so long as both objects have had their addresses exposed. There's some resistance to this because adhering to this model also breaks other optimizations (for example, (void*)(uintptr_t)x is not the same as x).
As a practical matter, pointer provenance isn't that big of a deal. It's not hard to come up with examples that illustrate behaviors that cause miscompilation or are undefined specifically because of pointer provenance. But I'm not aware of any application code that was actually miscompiled because the compiler implemented its provenance model incorrectly. The issue gets trickier as you move into systems code that exists somewhat outside the C object model, but even then, most of the relevant code can ignore their living outside the object model since resulting miscompiles are prevented by inherent optimization barriers anyways (note that to get a miscompile, you generally have to simultaneously forge the object's address, have the object's address be known to the compiler already, and have the compiler think the object's address wasn't exposed by other means).
The performance of compilers is frequently measured using the SPEC series of CPU benchmarks, and one of the main constraints of the series SPEC series of tests is that the source code of the benchmark cannot be changed. It is static.
As a result, compiler authors have to find increasingly convoluted ways to make it possible for various new compiler optimizations to be applied to the legacy code used in SPEC. Take 403.gcc: it's based on gcc version 3.2 which was released on August 14th 2002 -- nearly 23 years ago.
By making certain code patterns undefined behaviour, compiler developers are able to relax the constraints and allow various optimizations to be applied to legacy code in places which would not otherwise be possible. I believe the gcc optimization to eliminate NULL pointer checks when the pointer is dereferenced was motivated by such a scenario.
In the real world code tends to get updated when compilers are updated, or when performance optimizations are made, so there is no need for excessive compiler "heroics" to weasel its way into making optimizations apply via undefined behaviour. So long as SPEC is used to measure compiler performance using static and unchanging legacy code, we will continue to see compiler developers committing undefined behaviour madness.
The only way around this is for non-compiler developer folks to force language standards to prevent compilers from using undefined behaviour to do that which normal software developers considers to be utterly insane code transformations.
https://dl.acm.org/doi/10.5555/11616.11617
https://jacobfilipp.com/DrDobbs/articles/DDJ/1991/9108/9108h...
It's not something that exists in the hardware. It's also not a good idea, though trying to steer people away from it proved beyond my politics.
The essential idea of pointer provenance is that it is somehow possible to enumerate all of the uses of a memory location (in a potentially very limited scope). By the time you need to introduce something like "volatile" to indicate to the compiler that there are unknown uses of a variable, you have to concede the point that the compiler needs to be able to track all the known uses within a compiler--and that process, of figuring out known uses, is pointer provenance.
As for optimizations, the primary optimization impacted by pointer provenance is... moving variables from stack memory to registers. It's basically a prerequisite for doing any optimization.
The thing is that traditionally, the pointer provenance model of compilers is generally a hand-wavey "trace dataflow back to the object address's source", which breaks down in that optimizers haven't maintained source-level data dependency for a few decades now. This hasn't been much of a problem in practice, because breaking data dependencies largely requires you to have pointers that have the same address, and you don't really run into a situation where you have two objects at the same address and you're playing around with pointers to their objects in a way that might cause the compiler to break the dependency, at least outside of contrived examples.
When the hardware behaviour and the pointer provenance model disagree, one should change the model, not change the behavior of the program.
Say you're working with assembly as your medium, on a von neumann machine. Writing to parts of the code section is expected behaviour. What can you optimise in such a world? Whatever cannot be observed. Which might mean replacing instructions with sequences of the same length, or it might mean you can't work out anything at all.
C is much more restricted. The "function code" isn't there, forging pointers to the middle of a function is not a thing, nor is writing to one to change the function. Thus the dataflow is much easier, be a little careful with addresses of starts of functions and you're good.
Likewise the stack pointer is hidden - you can't index into the caller's frame - so the compiler is free to choose where to put things. You can't even index into your own frame so any variable whose address is not taken can go into a register with no further thought.
That's the point of higher level languages, broadly. You rule out forms of introspection, which allows more stuff to change.
C++ has taken this too far with the object model in my opinion but the committee disagrees.
Regarding provenance, consider this:
If the compiler can track the lifetime of the dynamically allocated int, it can remove the allocation and covert this function to simply It can't if arbitrary code (for example inside bar()) can forge pointers to that memory location. The code can seem silly, but you could end up with something similar after inlining.Yes. It absolutely can. What are you even talking about?
C is not the Windows Start Menu. This habit of thinking it needs to do what it thinks I might expect instead of what I told it is deeply psychotic.
If you don't know the extents of the object pointed to by the char*, using an aligned vector load can reach outside the bounds of the object. Keeping provenance makes that undefined behavior.
Using integer arithmetic, and pointer-to-integer/integer-to-pointer conversions would make this implementation defined, and well defined in all of the hardware platforms where an aligned vector load can never possibly fail.
So you can't do some optimizations to functions where this happens? Great. Do it. What else?
As for why you'd want to do this. C makes strings null-terminated, and you can't know their extents without strlen first. So how do you implement strlen? Similarly your example. Seems great until you're the one implementing malloc.
But I'm sure "let's create undefined behavior for a libc implemented in C" is a fine goal.
I think your example would fall foul of reading beyond the end of an object in addition to pointer provenance. In your case the oob read is harmless as you do not expect any meaningful values for the extra bytes, but generally the compiler would not be able to give any guarantees about the content of the additional memory (or that the memory exists in the first place).
This specific use case could be addressed by the standard, but vectors are already out of the standard, so in practice you use whatever extension you have to use and abide to whatever additional rule the compiler requires (of course this is often underspecified). For example, on GCC simd primitives already have carve-outs for TBAA.
FWIW, a libc implementation in practice already must rely on compiler specific, beyond the standard behaviour anyway.
As an off-topic aside here that might help anybody who is wondering: HN deliberately doesn't provide "Reply" for very recent comments to try to dissuade you from having the sort of urgent back-and-forth you might reasonably do in a real time chat system, and less reasonably attempt (and likely regret) on platforms like Twitter.
A brief window to think about the thing you just read might cause you to write something more thoughtful, and even to realise that it wasn't saying what you had thought in the first place.
My favourite example was an example where somebody said a feature means "less typing" and another comment insisted it did not, and I was outraged until I realised all that's happening is that one person thinks "Typing" means "You know, pressing keys on your keyboard" and the other person thinks "Typing" means "You know, why an integer is different from a float in C" and so they're actually not even disagreeing the conflict is purely syntax!
This is sort of on the one hand not a meaningful claim, and then on the other hand not even really true if you squint anyway?
Firstly the hardware does not have pointers. It has addresses, and those really are integers. Rust's addr() method on pointers gets you just an address, for whatever that's worth to you, you could write it to a log maybe if you like ?
But the Morello hardware demonstrates CHERI, an ARM feature in which a pointer has some associated information that's not the address, a sort of hardware provenance.
If it only exists in some hardware, how should the standard deal with that?
Generally seems to me the C standard makes things like that UB. Signed integer overflow, for example. Implemented as wrapping two's-complement on modern architectures, defined as such in many modern languages, but UB in C due to ongoing support for niche architectures.
The issues around pointer provenance are inherent to the C abstract machine. It's a much more immediate show-stopper on architectures that don't have a flat address space, and the C abstract machine doesn't assume a flat address space because it supports architecture where that's not true. My understanding is that reflects some oddball historical architectures that aren't relevant anymore, nowadays that includes CHERI.
Including a header that is not in the program, and not in ISO C, is undefined behavior. So is calling a function that is not in ISO C and not in the program. (If the function is not anywhere, the program won't link. But if it is somewhere, then ISO C has nothing to say about its behavior.)
Correct, portable POSIX C programs have undefined behavior in ISO C; only if we interpret them via IEEE 1003 are they defined by that document.
If you invent a new platform with a C compiler, you can have it such that #include <windows.h> reformats all the attached storage devices. ISO C allows this because it doesn't specify what happens if #include <windows.h> successfully resolves to a file and includes its contents. Those contents could be anything, including some compile-time instruction to do harm.
Even if a compiler's documentationd doesn't grant that a certain instance of undefined behavior is a documented extension, the existence of a de facto extension can be inferred empirically through numerous experiments: compiling test code and reverse engineering the object code.
Moreover, the source code for a compiler may be available; the behavior of something can be inferred from studying the code. The code could change in the next version. But so could the documentation; documentation can take away a documented extension the same way as a compiler code change can take away a de facto extension.
Speaking of object code: if you follow a programming paradigm of verifying the object code, then undefined behavior becomes moot, to an extent. You don't trust the compiler anyway. If the machine code has the behavior which implements the requirements that your project expects of the source code, then the necessary thing has been somehow obtained.
True, most compilers have sane defaults in many cases for things that are technically undefined (like take sizeof(void) or do pointer arithmetic on something other than a char). But not all of these cases can be saved by sane defaults.
Undefined behavior means the compiler can replace the code with whatever. So if you e.g. compile optimizing for size, the compiler will rip out the offending code, as replacing it with nothing yields the greatest size optimization.
See also John Regehr's collection of UB-Canaries: https://github.com/regehr/ub-canaries
Snippets of software exhibiting undefined behavior, executing e.g. both the true and the false branch of an if-statement or none etc. UB should not be taken lightly IMO...
Or replacing all you mp3s with a Rick Roll. Technically legal.
(Some old version of GHC had a hilarious bug where it would delete any source code with a compiler error in it. Something like this would technically legal for most compiler errors a C compiler could spot.)
The code change might come in something as innocent as a bug fix to the compiler.
I for one am glad that compilers can assume that things that can't happen according to the language do in fact not happen and don't bloat my programs with code to handle them.
Yes, unthinkable happenstances like addition on fixed-width integers overflowing! According to the language, signed integers can't overflow, so code like the following:
can be optimized to the much leaner Well, I sure am glad the compiler helpfully reduced the bloat in my program!Again, this isn't undefined behavior to fuck you over and compilers don't use it for optimizations because they hate you. It's because it makes a real difference for performance which is the primary reason low level languages are used.
If you for some reason want less efficient C++ then compilers even provide you flags to make this particular operation defined. There is no reason the rest of us have to suffer worse performance because of you.
Personally I would prefer if unsigned ints had the same undefined behavior by default with explicit functions for wrapping overflow. That would make developer intent much clearer and give tools a chance to diagnose unwanted overflow.
What would the "sky falling" look like to you? If you're expecting dramatic movie scenes like something out of Mr Robot, I'm afraid the reality is more mundane, just a never-ending series of basic programming errors that turn into remote code execution exploits because of language and compiler choices by people who don't pay the costs.
Vulnerabilities to ransomware (and other forms of malware) can be perpetrated without a single bad pointer being dereferenced.
For instance, a memory-safe e-mail program can automatically open an attachment, and the memory-safe application which handles the attachment can blindly run code embedded in the document in a leaky sandbox.
There is an incredible amount of infrastructure out there that depends on C. Embedded devices, mobile devices, desktops, servers. Network stacks, telephony stacks, storage, you name it. Encryption, codecs, ...
Sky is falling would mean all of it would be falling down so badly that, for instance, you would have about a 50% chance of connecting a server that is more than four hops away.
Unfortunely too many folks still pretend C is some kind of magic portable Assembly language that no other language on Earth is able to achieve the same.
Also if I care enough about ultimate performace, like anyone that actually cares about performance, I dust off my Assembly programming skills, alongside algorithms, datastructures and computer organisation.
What is this supposed to mean? I can't think of any interpretation that makes sense.
I think ISO C defines the executable program to be something like the compiled translation units linked together. But header files do not have to have any particular correspondence to translation units. For example, a header might declare functions whose definitions are spread across multiple translation units, or define things that don't need any definitions in particular translation units (e.g. enum or struct definitions). It could even play macro tricks which means it declares or defines different things each time you include it.
Maybe you mean it's undefined behaviour to include a header file that declares functions that are not defined in any translation unit. I'm not sure even that is true, so long as you don't use those functions. It's definitely not true in C++, where it's only a problem (not sure if it's undefined exactly) if you ODR-rule use a function that has been declared but not defined anywhere. (Examples of ODR-rule use are calling or taking the address of the function, but not, for example, using sizeof on an expression that includes it.)
Start with a concrete example. A header that is not in our program, or described in ISO C. How about:
Defined behavior or not? How can an implementation respond to this #include while remaining conforming? What are the limits on that response?> But header files do not have to have any particular correspondence to translation units.
A header inclusion is just a mechanism that brings preprocessor tokens into a translation unit. So, what does the standard tell us about the tokens coming from #include <winkle.h> into whatever translation unit we put it into?
Say we have a single file program and we made that the first line. Without that include, it's a standard-conforming Hello World.
I just checked, and if you attempt to include a file that cannot be found (in the include path, though it doesn't use that exact term) then that's a constraint violation and the compiler is required to stop compilation and issue a diagnostic. Not undefined behaviour.
What are the requirements then?
Correct; but processing doesn't stop there.
> There might be undefined behaviour in the overall translation unit
But what does that mean; how do you infer that there might be undefined behavior?
Does ISO C define the behavior, or does it not?
ISO C has nothing to say about what is in #include <winkle.h> if such a header is found and didn't come from the program.
Without having anything to say about what is in it, if it is found at all, ISO C cannot be giving a definition of behavior of the tokens that are substituted for that #include.
N3220: 5.1.1.1 Program Structure
A C program is not required to be translated in its entirety at the same time. The text of the program is kept in units called source files, (or preprocessing files) in this document. A source file together with all the headers and source files included via the preprocessing directive #include is known as a preprocessing translation unit. After preprocessing, a preprocessing translation unit is called a translation unit. Previously translated translation units may be preserved individually or in libraries. The separate translation units of a program communicate by (for example) calls to functions whose identifiers have external linkage, manipulation of objects whose identifiers have external linkage, or manipulation of data files. Translation units may be separately translated and then later linked to produce an executable program.
> Couldn't it be the set {main.c, winkle.h}
No; in this discussion it is important that <winkle.h> is understood not to be part of the program; no such header is among the files presented for translation, linking and execution. Thus, if the implementation doesn't resolve #include <winkle.h> we get the uninteresting situation that a constraint is violated.
Let's focus on the situation where it so happens that #include <winkle.h> does resolve to something in the implementation.
Your question about an include file that isn't part of the program just doesn't make any sense.
(Technically it says that those files together make up the "program text". As my other comment says, "program" is the binary output.)
So what I mean is that no file matching <winkle.h> has been presented as part of the external file set given to the implementation for processsing.
I agree that if such a file is found by the implementation it becomes part of the program, as makes sese and as that word is defined by ISO C, so it is not right terminology to say that the file is not part of the program, yet may be found.
If the inclusion is successful, though, the content of that portion of that program is not defined by ISO C.
> So what I mean is that no file matching <winkle.h> has been presented as part of the external file set given to the implementation for processsing.
The thing is, there is no "external file set" that includes header files, so this sentence makes no sense.
Note that when the preprocessor is run, the only inputs are the file being preprocessed (i.e., the .c file) and the list of directories to find include files (called the include path). That's not really part of the ISO standard, but it's almost universal in practice. Then the output of the preprocessor is passed to the compiler, and now it's all one flat file so there isn't even a concept of included files at this point. The object files from compilation are then passed to the linker, which again doesn't care about headers (or indeed the top-level source files). There are more details in practice (especially with libraries) but that's the essence.
I wonder if your confusion is based on seeing header files in some sort of project-like structure in an IDE (like Visual Studio). But those are just there for ease of editing - the compiler (/preprocessor) doesn't know or care which header files are in your IDE's project, it only cares about the directories in the include path. The same applies to CMake targets: you can add include files with target_sources(), but that's just to make them show up in any generated IDE projects; it has no effect on compilation.
Or are you just maybe saying that the developer's file system isn't part of the ISO C standard, so this whole textual inclusion process is by some meaning not defined by the standard? If so, I don't think that matches the conventional meaning of undefined behaviour.
If it's neither of those, could you clarify what exactly you mean by "the external file set given to the implementation for processing"?
The "project" is all the files going into a program supplied other than by the implementation.
C programs can contain #include directives. Those #include directives can be satisfied in one of three ways: they can reference a standard header which is specified by ISO C and hence effectively built into the hosted language, such as <stdio.h>.
C programs can #include a file from the project. For instance someone's "stack.c" includes "stack.h". So yes, there is an external file set (the project) which can have header files.
C programs can also #include something which is neither of the above. That something might be not found (constraint violation). Or it might be found (the implementation provides it). For instance <sys/mmap.h>: not in your project, not in ISO C.
My fictitious <winkle.h> falls into this category. (It deliberately doesn't look like a common platform-specific header coming from any well-known implementation---but that doesn't matter to the point).
> Or are you just maybe saying that the developer's file system isn't part of the ISO C standard, so this whole textual inclusion process is by some meaning not defined by the standard?
Of course, it isn't, no I'm not saying that. The C standard gives requirements as to how a program (project part and other) is processed by the implementation, including all the translation phases that include preprocessing.
To understand what the requirements are, we must consider the content of the program. We know what the content is of the project parts: that's in our files. We (usually indirectly) know the content of the standard headers, from the standard; we ensure that we have met the rules regarding their correct use and what we may or may not rely on coming form them.
We don't know the content of successfully included headers that don't come from our project or from ISO C; or, rather, we don't know that content just from knowing ISO C and our project. In ISO C, we can't find any requirements as to what is supposed to be there, and we can't find it in our project either.
If we peek into the implementation to see what #include <winkle.h> is doing (and such a peeking is usually possible), we are effectively looking at a document, and then if we infer from that document what the behavior will be, it is a documented extension --- standing in the same place as what ISO C calls undefined behavior. Alternatively, we could look to actual documentation. E.g. POSIX tells us what is in <fcntl.h> without us having to look for the file and analyze the tokens. When we use it we have "POSIX-defined" behavior.
#include <winkle.h> is in the same category of thing as __asm__ __volatile__ or __int128_t or what have you.
#include <winkle.h> could contain the token __wipe_current_directory_at_compile_time which the accompanying compiler understands and executes as soon as it parses the token. Or __make_demons_fly_out_of_nose. :)
Do you see the point? When you include a nonstandard header that is not coming from your project, and the include succeeds, anything can happen. ISO C no longer dictates the requirements as to what the behavior will be. Something unexpected can happen, still at translation time.
Now headers like <windows.h> or <unistd.h> are exactly like <winkle.h>: same undefined behavior.
Most of my most recent comment is addressing the possibility that you meant this.
As I said, there is no such concept to the compiler. It isn't passed any list of files that could be included with #includr, only the .c files actually being compiled, and the directories containing includable files.
The fact that your IDE shows project files is an illusion. Any header files shown there are not treated differently by the compiler/preprocessor to any others. They can't be, because it's not told about them!
It's even possible to add header files to your IDE's project that are not in the include path, and then they wouldn't be picked up by #include. That's how irrelevant project files are to #include.
It doesn't seem as if you're engaging with the standard-based point I've been making, in spite of detailed elaboration.
> Any header files shown there are not treated differently by the compiler/preprocessor to any others.
This is absolutely false. Headers which are part of the implementation, such as standard-defined headers like <stdlib.h> need not be implemented as files. When the implementation processes #include <stdlib.h>, it just has to flip an internal switch which makes certain identifiers appear in their respective scopes as required.
For that reason, if an implementation provides <winkle.h>, there need not be such a file anywhere in its installation.
i) "Fil-C is a fanatically compatible memory-safe implementation of C and C++. Lots of software compiles and runs with Fil-C with zero or minimal changes. All memory safety errors are caught as Fil-C panics." "Fil-C only works on Linux/X86_64."
ii) "scpptool is a command line tool to help enforce a memory and data race safe subset of C++. It's designed to work with the SaferCPlusPlus library. It analyzes the specified C++ file(s) and reports places in the code that it cannot verify to be safe. By design, the tool and the library should be able to fully ensure "lifetime", bounds and data race safety." "This tool also has some ability to convert C source files to the memory safe subset of C++ it enforces"
The resulting language doesn't make sense for commercial purposes but there's no reason it couldn't be popular with hobbyists.
Run your test suite and some other workloads under Fil-C for a while, fix any problems report, and if it doesn't report any problems after a while, compile the whole thing with GCC afterwards for your release version.
But the point is also to dispel the conventional wisdom that C/C++ is necessarily intrinsically unsafe. It's a tradeoff between safety, performance and flexibility/compatibility. And you don't necessarily need to jump to a completely different language to get a different tradeoff.
Fil-C sacrifices some performance for safety and compatibility. The traditional compilers sacrifice some safety for performance and flexibility/compatibility. And scpptool aims to provide the option of sacrificing some flexibility for safety and performance. (Along with the other two tradeoffs available in the same program). The claim is that C++ turns out to be expressive enough to accommodate the various tradeoffs. (Though I'm not saying it's always gonna be pretty :)
To get an example that's easy to understand: before the introduction of the 'const' keyword, you just couldn't express that some variable should never be changed. And no amount of UB sanitisers would have fixed this for you: you just couldn't express the concept. There's lots of other areas of these languages that are still in a similar state.
Eg there's no way to express that a function should be pure, ie not have side effects (but is allowed to use mutation internally).
I believe gcc actually does support `__attribute__ ((pure))` to indicate function purity. (I assume it doesn't actually enforce it, but presumably it theoretically could at some point.)
We have zig, Hare, Odin, V too.
Because it never achieved mainstream success?
And Zig for example is very much not memory safe. Which a cursory search for ”segfault” in the Bun repo quickly tells you.
https://github.com/oven-sh/bun/issues?q=is%3Aissue%20state%3...
And with this attitude it never will. With Rust's hype, it would.
Ada would rather be a nice choice, but most hackers love their curly brackets.
They at least fixed this in c++26. No longer UB, but "erroneous behavior". Still some random garbage value (so an uninitialized pointer will likely lead to disastrous results still), but the compiler isn't allowed to fuck up your code, it has to generate code as if it had some value.
In effect if you don't opt out your value will always be initialized but not to a useful value you chose. You can think of this as similar to the (current, defanged and deprecated as well as unsafe) Rust std::mem::uninitialized()
There were earlier attempts to make this value zero, or rather, as many 0x00 bytes as needed, because on most platforms that's markedly cheaper to do, but unfortunately some C++ would actually have worse bugs if the "forgot to initialize" case was reliably zero instead.
Bad guys find the corner case and they can now authenticate as user 0x4C6F6769 which doesn't exist and so that's useless. But - when we upgrade to C++ 26 with the hypothetical zero "fix" now they're root instead!
Access to an uninitialized object defined in automatic storage, whose address is not taken, is UB.
Access to any uninitialized object whose bit pattern is a non-value, likewise.
Otherwise, it's good: the value implied by the bit pattern is obtained and computation goes on its merry way.
(The B language was implemented for the PDP-7 before the PDP-11, which are rather different machines. It’s sometimes suggested that the increment and decrement operators in C, which were inherited from B, are due to the instruction set architecture of the PDP-11, but this could not have been the case. Per Dennis Ritchie:¹
> Thompson went a step further by inventing the ++ and -- operators, which increment or decrement; their prefix or postfix position determines whether the alteration occurs before or after noting the value of the operand. They were not in the earliest versions of B, but appeared along the way. People often guess that they were created to use the auto-increment and auto-decrement address modes provided by the DEC PDP-11 on which C and Unix first became popular. This is historically impossible, since there was no PDP-11 when B was developed. The PDP-7, however, did have a few “auto-increment” memory cells, with the property that an indirect memory reference through them incremented the cell. This feature probably suggested such operators to Thompson; the generalization to make them both prefix and postfix was his own.
Another person puts it this way:²
> It's a myth to suggest C’s design is based on the PDP-11. People often quote, for example, the increment and decrement operators because they have an analogue in the PDP-11 instruction set. This is, however, a coincidence. Those operators were invented before the language [i.e. B] was ported to the PDP-11.
In any case, the PDP-11 usually gets all the love, but I want to make sure the other PDPs get some too!)
[1] https://www.bell-labs.com/usr/dmr/www/chist.html
[2] https://retrocomputing.stackexchange.com/questions/8869
- it is an automatic variable whose address has not been taken; or
- the uninitialized object' bits are such that it takes on a non-value representation.
Not only you are faced with creating your own wrappers, if no one else has done it already.
The tooling, for IDEs and graphical debuggers, assumes either C or C++, so it won't be there for Rust.
Ideally the day will come where those ecosystems might also embrace Rust, but that is still decades away maybe.
C is a different kind of animal that encourages terseness and economy of expression. When you know what you are doing with C pointers, the compiler just doesn't get in the way.
> When you know what you are doing with C pointers, the compiler just doesn't get in the way.
Alas, it doesn't get in the way of you shooting your own foot off, too.
Rust allows unsafe and other shenanigans, if you want that.
In the most basic cases, yes. It can be used as a more polished switch statement.
It's the whole paradigm of "define an ad-hoc Enum here and there", encoding rigid semantic assumptions about a function's behaviour with ADTs, and pattern matching for control-flow. This feels like a very academic approach and modifying such code to alter its opinionated assumptions isn't funny.
Tell me you use -fno-strict-aliasing without telling me.
Fwiw, I agree with you and we're in good[citation needed] company: https://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg...
IMHO you can today deal with UB just fine in C if you want to by following best practices, and the reasons given when those are not followed would also rule out use of most other safer languages.
C is portable in the least interesting way, namely that compilers exist for all architectures. But that's where it stops.
I'm guessing you mean that every cross-platform C codebase ends up being plastered in cascading preprocessor code to deal with OS and architecture differences. Sure that's true, you still have to do some porting work regardless of the language you chose.
But honestly, is there any language more portable than C? I struggle to come up with one.
If someone told me "I need a performant language that targets all major architectures and operating systems, but also maybe I want to run it on DOS, S390X, an old Amiga I have in my closet, and any mystery-meat microcontroller I can find." then really wouldn't have a better answer for them than C89.
If C isn't portable then nothing is.
If you are targeting any recent platform, both Rust and Zig do what you want.
A decade earlier I also used Xenix and DG/UX.
That is a nice way to learn how "portable" C happens to be, even between UNIX systems, its birthplace.
You cannot write portable code without platform-specific and even environment-specific adaptations, like handling the presence of certain headers (looking at you, stdint.h and stddef.h), and let's not even start about interacting with the OS in any way.
> IMHO you can today deal with UB just fine in C if you want to by following best practices
In the other words, short compilation time has been traded off with wetware brainwashing... well, adjustment time, which makes the supposed advantage much less desirable. It is still an advantage, I reckon though.
I mean to say that best practices do help much but learning those best practices take much time as well. So short compilation time is easily offseted by learning time, and C was not even designed to optimize compilation time anyway (C headers can take a lot to parse and discard even when unused!). Your other points do make much more sense and it's unfortunate that first points are destructively interfering each other, hence my comment.
The header model is one of the parts that makes compiling C slower than it could be. This doesn't mean that it is slow, but it's fast in spite of headers, not because of them.
> In C you can split interface and implementation cleanly between header and c-file and this enables efficient incremental builds.
That's not what does, it is the ability to produce individual translation units as intermediary files.
> Rust also does not seem to have proper separate compilation.
Rust does separate compilation, and also has efficient incremental builds. Header files are not a hard requirement for this.
I am not sure how it works in Rust as you need to monomorphize a lot of things, which come from other crates. It seems this would inevitably entangle the compilations.
It's that textual inclusion is just a terrible model. You end up reprocessing the same thing over and over again, everywhere it is used. If you #include<foo.h> 100 times, the compiler has to reparse those contents 100 times. Nested headers end up amplifying this effect. It's also at a file-level granularity, if you change a header, every single .c that imports it must be recompiled, even if it didn't use the thing that was changed. etc etc. These issues are widely known.
> I do not understand what you mean by your second point. What separation of interface and implementation allows you to do is updating the implementation without having to recompile other TUs.
Sure, but you don't need to have header files to do this. Due to issues like the above, they cause more things to be recompiled than necessary, not less.
> You can achieve this is also in different ways, but in C this works by in this way.
Right, my point is, those other ways are better.
> I am not sure how it works in Rust as you need to monomorphize a lot of things, which come from other crates. It seems this would inevitably entangle the compilations.
The fact that there are "other crates" is because Rust supports separate compilation: each crate is compiled independently, on its own.
The rlib contains the information that, when you link two crates together, the compiler can use for monomorphization. And it's true that monomorphization can cause a lot of rebuilding.
But to be clear, I am not arguing that Rust compilation is fast. I'm arguing that C could be even faster if it didn't have the preprocessor.
One could certainly store the interfaces in some binary format, but is it really worth it? This would also work with headers by using a cache, but nobody does it for C because there is not much to gain. Parsing is fast anyhow, and compilers are smart enough not to look at headers multiple times when protected by include guards. According to some quick measurements, you could save a couple of percent at most.
The advantages of headers is that they are simple, transparent, discoverable, and work with outside tools in a modular way. This goes against the trend of building frameworks that tie everything together in a tightly integrated way. But I prefer the former. I do not think it is a terrible model, quite the opposite. I think it is a much better and nicer model.
Relevant: https://youtu.be/4t1K66dMhWk?si=dZL2DoVD94WMl4fI
It's also very often not the best way to identify objects, for many reasons, including performance (spatial locality is a big deal).
These problems go away almost completely by simply using `EntityID` and going through `&mut World` for modifications, rather than passing around `EntityPtr`. This pattern gives you a lot of interesting things for free.
Pretty much nobody writing games in C++ uses raw pointers in entities to hold references to other related entities, because entities can be destroyed at any time and there's no simple way for a referring entity to know when a referenced entity is destroyed.
Using some sort of entity ID or entity handle is very common in C++, the problem is that when implementing this sort of system in Rust, developers often end up having to effectively "work around" the borrow checker, and they end up not really gaining anything in terms of correctness over C++, ultimately defeating the purpose of using Rust in the first place, at least for that particular system.
The benefits seem pretty massive, at least on the surface. For example, you can run any system that only takes `&World` (i.e., immutable access) in parallel without breaking a sweat.
As shitty as C++ is from today's PoV, the entire gaming industry switched over within around 3 years towards the end of the 90s. 6..7 years is a long time, and a single engine (especially when it's more or less just a runtime without editor and robust asset pipeline) won't change the bigger picture that Rust is a pretty poor choice for gamedev.
Did they? What's your evidence? Are you including consoles?
Btw, the alternatives in the 1990s were worse than they are now, so the bar to clear for eg C or C++ were lower.
From what I've seen, around the late mid-90's, C++ usage was still rare, right before 2000 it was already common and most middleware didn't even offer C APIs anymore.
Of course a couple of years later Unity arrived and made the gamedev language choice more complicated again.
That would be 2000, until then Sega, Nintendo and Playstion only had C and Assembly SDKs, even the Playstation Yaroze for hobbists did get released only with C and Assembly support.
PC was naturally another matter, especialy with Watcom C/C++.
You were at most in one place. My question was rather, which corners of the industry are you counting?
However you are right that one of the killer features of C++ was that it provided a pretty simple upgrade path from C to (bad) C++.
It's not just API calls. You can call C APIs from most languages just fine.
id Software was kinda famous for being the last big C holdout, having only switched to C++ with Doom 3, and development of Doom 3 started in late 2000.
Making a nontrivial game with them is a wholly different story.
C and C++ force you to code in the C and C++ ways. It may that that's what you want, but they certainly dont let me code how I want to code!
This is especially true for C which supports almost nothing (it doesn't even have a sensible array type!). But is also true for C++: while it supports a lot, it doesn't support everything.
Methods in C, just have function pointers as members. Common in many codebases.
Guaranteed tail calls, all the compilers guarantee that function calls that are a return expression are tail calls.
Tagged union in C++, it's trivial as a library, see std::variant for a bad example of it, and all the various monadic/pattern-matching variants (pun intended) people have written. C is at a disadvantage here due to lack of lambdas, but I'm sure people have built stuff using some GCC extensions.
You can do the same in C by wrapping your array in a struct.
Beyond that, recent C++ versions have much more expressive metaprogramming capability. The ability to do extensive codegen and code verification within C++ at compile-time reduces lines of code and increases safety in a significant way.
C and C++ have their strengths, but rapid prototyping is generally not seen to be amongst them.
This shouldn't be any more controversial than saying that pure Python is generally slow.
Rusts tooling is hands down better than C/++ which aids to a more streamlined and efficient development experience
Would you expand on this? What was your C tooling/workflow that was inferior to your new Rust experience?
As for the language tooling itself, static and runtime analyzers in C and C++ (and these are table stakes at this point) do not come close to the level of accuracy of the Rust compiler. If you care about writing unsafe code, Miri is orders of magnitude better at detecting UB than any runtime analyzer I've seen for C and C++.
Not sure where this is coming from.
Async rust is amazing as long as you only mix in one more hard concept. Be it traits, generics or whatever. You can confidently write and refactor heavily multithreaded code without being deathly afraid of race conditions etc. and it is extremely empowering.
The problem comes when trying to write async generic traits in a multithreaded environment.
Then just throwing stuff at the wall and hoping something sticks will quickly lead you into despair.
Rust is nicer for async and MT than c++ in every way. I am pretty sure.
But it's still mid. If you use Rust async aggressively you will struggle with the borrow checker and the architecture results of channel hell.
If you follow the "one control thread that does everything and never blocks" you can get far, but the language does not give you much help in doing that style neatly.
I have never used Go. I love a lot of Go projects like Forgejo and SyncThing. Maybe Go solved async. Rust did not. C++ did not even add good tagged unions yet.
Doing anything concurrent in Go is also really annoying (be that async or with threads), because everything is mutable. Not just by default but always. So anything shared is very dangerous.
The smallest binary rustc has produced is like ~145 bytes.
Just to be clear, this isn't a recent development, it has been this way for many years at this point.
Well, anything were your people have more experience in the other language or the libraries are a lot better.
The two instances where UB allows for optimisation are as follows:
1. The 'signed overflow' UB allows for faster array indexing. By ignoring potential overflow, the compiler can generate code that doesn't check for accidental overflow (which would require masking the array index, recomputing the address on each loop iteration). I believe the better solution here would be to introduce a specific type for iterating over arrays that will never overflow; size_t would do fine, and making signed overflow at least implementation defined, if not outright fully defined, after a suitable period during which compilers warn if you use a too-small type for array indexing.
2. The 'aliasing' UB does away with the need to read/write values to/from memory each time they're used, and is extremely important to performance optimisation.
But the rest? Most of it does precisely nothing for performance. At 'best', the compiler uses detected UB to silently eliminate code branches, but that's something to be feared, not celebrated. It isn't an optimisation if it removes vital program logic, because the compiler could 'demonstrate' that it could not possibly take the removed branch, on account of it containing UB.
The claim in the linked article ("what every C programmer should know") that use of uninitialized variables allows for additional optimisation is incorrect. What it does instead is this: if the compiler see you declare a variable, and then reading from it before writing to it, it has detected UB, and since the rule is that "the compiler is allowed to assume UB does not occur", use that as 'evidence' that that code branch will never occur and can be eliminated. It does not make things go faster; it makes them go _wrong_.
Undefined behaviour, ultimately, exists for many reasons: because the standards committee forgot a case, because the underlying platforms differ too wildly, because you cannot predict in advance what the result of a bug may be, to grandfather in broken old compilers, etc. It does not, in any way, shape, or form, exist _in order to_ enable optimisation. It _allows_ it in some cases, but that is, and never was, not the goal.
Moreover, the phrasing of "the compiler is allowed to assume that UB does not occur" was originally only meant to indicate that the compiler was allowed to emit code as if all was well, without introducing additional tests (for example, to see if overflow occurred or if a pointer was valid) - clearly that would be very expensive or downright infeasible. Unfortunately, over time this has enabled a toxic attitude to grow that turns minor bugs into major disasters, all in the name of 'performance'.
The two bullet points towards the end of the article are both true: the compiler SHOULD NOT behave like an adversary, and the compiler DOES NEED license to optimize. The mistake is thinking that UB is a necessary component of such license. If that were true, a language with more UB would automatically be faster than one with less. In reality, C++ and Rust are roughly identical in performance.
The spec even says:
> behavior, upon use of a nonportable or erroneous program construct or of erroneous data, for which this International Standard imposes no requirements
No motivation is given that I could find, so the actual difference between undefined and implementation defined behaviour seems to be based on whether the behaviour needs to be documented.
Also the C spec has always been a pragmatic afterthought, created and maintained to establish at least a minimal common feature set expected of C compilers.
The really interesting stuff still only exists outside the spec in vendor language extensions.
And I especially don’t buy that UB is there for register allocation.
First of all, that argument only explains UB of OOB memory accesses at best.
Second, you could define the meaning of OOB by just saying “pointers are integers” and then further state that nonescaping locals don’t get addresses. Many ways you could specify that, if you cared badly enough. My favorite way to do it involves saying that pointers to locals are lazy thunks that create addresses on demand.
Same thing with e.g. strict aliasing or the various UB that exists in the standard library. For instance, it's UB to pass a null pointer to strlen. Of course, you can make that perfectly defined by adding an `if` to strlen that just returns 0. But then you're adding a branch to every strlen, and C is simply not willing to do that for performance reasons, so they say "this is UB" instead.
Pretty much instance of UB in standard C or C++ is because making it defined would either hamper the optimizer, or it would make standard library functions slower. They don't just make things UB for fun.
For example the reason why 2s complement took so long is because of some machine that ran C that still existed that was 1s complement.
> The reason is that if you compile with flags that make it defined, you lose a few percentage points of performance (primarily from preventing loop unrolling and auto-vectorization).
I certainly don’t lose any perf on any workload of mine if I set -fwrapv
If your claim is that implementers use optimization as the excuse for wanting UB, then I can agree with that.
I don’t agree that it’s a valid argument though. The performance wins from UB are unconvincing, except maybe on BS benchmarks that C compilers overtune for marketing reasons.
You're misunderstanding me: as of C++20, there is no other representation in C++ for signed integers other than two's complement (no signed ones' complement, no signed magnitude, nothing else), but signed overflow is still UB. It's not because of obscure machines or hardware, such hardware is not relevant for C++20 and later. The reason for it is performance. From the accepted paper [1]:
> The following polls were taken, and corresponding modifications made to the paper. The main change between [P0907r0] and the subsequent revision is to maintain undefined behavior when signed integer overflow occurs, instead of defining wrapping behavior. This direction was motivated by:
> * Performance concerns, whereby defining the behavior prevents optimizers from assuming that overflow never occurs
You may disagree, you may think they're wrong, but their motivation is performance, that's why this is UB. It's right there in black and white. This was C++, not C, but it's not at all unthinkable that the C standard will also mandate two's complement at some point, and if they do, they almost certainly keep signed overflow undefined for exactly the same reason.
It's not hard to write code that optimizes much better when you use signed loop variables. One of my favorite examples is this function [2] to turn a 3D mesh inside out by flipping the edges of each triangle in a triangle mesh. The godbolt link has two versions of the same function, one with a signed loop variable, one with an unsigned one. The signed one auto-vectorizes and optimizes much better because it can assume that the loop variable never overflows (this version is C++, it's trivial to rewrite it in C and get the same results).
This is why signed overflow is UB.
[1]: https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p09...
[2]: https://godbolt.org/z/a1P5Y17fn
I know that this is misguided based on my own perf tests and others’ perf tests.
Also, it’s wrong to say flat out that UB on signed ints is somehow necessary for perf when even a simple perf test shows that it just doesn’t matter, and the optimization it enables is quite obscure.
You are absolutely, 100% correct though: I've never seen a case where accidental overflow doesn't start causing bugs anyway. Like, the Pac-Man kill screen is caused by a byte overflowing (it happens on level 256), and the game goes insane. Pac-Man was written in assembly where overflow is defined behavior, but that doesn't matter at all, the game is still broken. If signed overflow is essentially always a bug anyway, why not make it UB and optimize around it? Especially since it is super-valuable in being able to unroll loops.
People always bring up signed integer overflow as an argument for why UB is scary, and it always seemed like such a bad argument to me. Like, I can understand why people think UB has gone too far in C/C++, but signed overflow is such a bad example. It's one of the most sensible bits of UB in the entire standard, IMHO.
It explains many loop-unroll and integer overflow as well.
inlining, interprocedural optimizations.
For example, something as an trivial accessor member function would be hard to optimize.
And I believe that provenance is an issue in unsafe rust.
https://doc.rust-lang.org/std/ptr/index.html#provenance
The definition isn't deemed complete because of aliasing. AIUI The definition we have is adequate if you're OK with treating all edge cases for "Is this an alias?" as "Yes" but eventually Rust will also need to carefully nail down all those edge cases so that you can tread closer without falling off.
The right definition is probably something like:
- pointers that come out of the outside world (syscalls) are escaped. They are just integers.
- pointers to locals have provenance. They point to an abstract location. It is up to the implementation to decide when the location gets an integer value (is in an actual address) and what that value is. The implementation must do this no later than when the pointer to the local escapes.
- pointer values passed to the outside world (syscalls) escape.
- pointer values stored in escaped memory also escape, transitively
That’s one possible definition that turns the UB into implementation defined behavior. I’m sure there are others
Again, I'm not a compiler writer, but my understanding is that non escaping variables can be optimized in SSA form, escaped variables are otherwise treated as memory and the compiler must be significantly more conservative.
In any case, whether a pointer escapes or not depends purely on the compiler capabilities and optimization level, so it would not be sane making a code well defined or UB depending on the compiler or optimization level.
edit: to be more concrete, do you think that in my example the constant folding of the return into return 1 should be allowed? And if so, which variant of this code would prevent the optimization and why?
I am a compiler writer.
The definition I gave in my post is general enough to cover all possible compilers (ones that have LTO, ones that are inside a DBT, etc).
Yes the constant folding should be allowed because the pointer to the local never escaped.
This means losing a lot of optimisations, so in fact when you say you "don't buy" this argument you only mean that you don't care about optimisation. Which is fine, but this does mean the "improved" C isn't very useful in a lot of applications, might as well choose Java.
You won’t lose “a lot” of optimizations and you certainly won’t lose enough for it to make a noticeable difference in any workload that isn’t SPEC
The dustbin of programming languages is jam packed with elegant, technically terrific, languages that never went anywhere.
Chill the fuck out.