This is not normal. Modern cryptosystems don't have anything like PGP's insane "packet" format, which has caused other problems before this. There's no principal of design that would lead you to what PGP came up with, and the only reason we still have to deal with it is path dependence. I don't even care if you call the next design "PGP2", just throw this system in the bin and start over.
mananaysiempre 15 hours ago [-]
Unfortunately, modern cryptographers refuse to design systems for confidentiality of store-and-forward communications, motivated by the fact that one can provide better security in instant messaging[1] (which in most cases implies vendor lock-in). Age is one of the few systems that could accomodate email, and its author’s “solution”[2] is for the user do ASCII armor/dearmor like a caveman, which handily loses to PGP/MIME in just about every email client that supports the latter. So, until somebody does better, PGP email it is.
But none of that excuses PGP's clanking and outmoded design. Things that are bad are bad. We can't moralize our way around that.
mananaysiempre 15 hours ago [-]
I don’t really want to make a moral argument about PGP. Yes, the design of PGP is bad even if we hold its problem statement fixed. The thing is, the people I would trust to design something better consider that problem not worth solving. So if I as a user do want a solution to that problem[1], I’m stuck with PGP, which is bad. But at least it makes an attempt.
[1] Because I need to be in areas with bad connectivity often enough that I don’t want my every communication method to require a Internet connection to be available continuously (or even daily). Because Signal’s refusal to issue me an account without a phone number (which I cannot legally obtain without tying it to my government ID) is a real risk to my personal safety, given my particular situation. Because Signal’s stance on alternative implementations and the like is a hair’s breadth away from my refusing to use it. Some of these you could call moral arguments, but none of them are about PGP, as such.
tptacek 14 hours ago [-]
The basic thesis of that post (the original of which I wrote) is that if messages are important enough that they need to be protected from state-level adversaries, then they're important enough that you shouldn't care about all the other reasons you like SMTP. If any of SMTP's affordances trump message security, you're LARPing the security part: you're protecting them in a threat model that doesn't correspond to a real-world attacker.
jcranmer 14 hours ago [-]
With email, you necessarily leak the triple (to, from, date): no matter how good your cryptosystem is, that information can be pulled from the mailserver logs (and if you don't leak that triple, spam deluge is unsolvable). As a practical matter, you generally need to leak the mail headers as well, which contains a decent amount of useful metadata as well. Now I'm not an expert on how spy agencies do their analysis, but my understanding is that the set of information that has to be leaked by email is already the most useful information for them. And that's part of tptacek's point: if your main threat adversary is state-level actors, you've already lost if you're using email simply because you're using email.
What if your threat model isn't state-level actors? Well, the baseline of email these days is that your communication and your recipient's communication to the mail servers are both encrypted with TLS. The mail servers themselves may or may not communicate with each other using TLS, but if you really care about security, you can choose a mail server which will be using TLS. In other words, email is already at a baseline state of the only people aware of the message being the sender, the recipient, and their mail admins (and whomever these people choose to leak the message to, perhaps unwillingly). Encrypting email will only remove the mail admins from the list, and even then, they can still tell anybody whom you talked about.
So the use case of encrypted email boils down to wanting to hide the contents of communication but not hiding the fact of communication itself. Which isn't a broad use case; the best examples I've found in my own life is something like financial statements, but even in those cases, there's a pretty decent workaround: send an email saying "hi, we have a document for you online in our usual secure file repository" (protected by modern, useful standards for secure files). I don't like that only because I can't automate saving those files off to my own storage for my own purposes, but that is such a niche desire that I can understand why the bank doesn't bother.
tptacek 14 hours ago [-]
The original version of this post also mentioned, and was in fact motivated by, a fatal flaw in encrypted email that everyone who has used it at scale has experienced, which is that participants will reply to encrypted messages with unencrypted responses that quote the original (whether or not a reply quotes the original doesn't change how devastating this problem is, but sort of highlights how insane the system is).
So I think the metadata argument is dispositive. I agree with you that it's difficult to compose a coherent threat model that leaves metadata exposed the way SMTP does.
But the core argument of the piece is that encrypted email makes security concessions nobody would make if e.g. the wire for the down payment on their house was at stake, or if they were organizing against an oppressive state-level adversary. If encrypted email is unsuitable for those scenarios, then it seems more important to keep it from being used there than it does to accommodate the interests of people who using it for other reasons.
jcranmer 17 hours ago [-]
I'll admit that my knowledge of signed/encrypted email is mostly from S/MIME (and underlying CMS), so I'm curious if you could enlighten me on what PGP is doing that is so much more insane than that.
woodruffw 13 hours ago [-]
Well, I don't know if I would call PGP so much more insane than CMS or PKCS#7 :-). Definitely worse, but CMS is not high up on the list of honorable cryptographic envelope designs.
On the format level, CMS has some of the same flaws as PGP: dynamic TLV encodings (BER), extension points everywhere, and a disconnect between format and cryptographic versioning. On the cryptographic level, S/MIME benefits somewhat from certificates on the Internet PKI being less of a wild west than PGP certificates, and from having a community group (the S/MIME Cert WG of the CA/B Forum) invested in strengthening S/MIME's certificate profile beyond the baseline stipulated in RFC 5280. Of course, for non-public S/MIME deployments, none of that applies.
All that said, I don't think I would treat S/MIME (or CMS, or PKCS#7) as a guiding star: EFAIL affected S/MIME too[1]. But they have the "advantage" of being bad at just their niche (signing and encryption of email), versus being bad at every niche. The latter is PGP's historic curse.
The way I've previously internalized it is CMS is bad principally because it's generic container of general generalizablity (on multiple levels, even) and the fundamentally wrong notion that signature and encryption are fully orthogonal. But generic generalizability can be ameliorated with a concerted, coordinated effort. As for S/MIME (specifically, working out how to embed CMS in MIME)... well, email and MIME make a good solution impossible from the outset, and S/MIME is probably the least bad you can do.
Encrypted email can never have a good solution simply because email is the poster child for "Why Postel's Law is a bad idea"
woodruffw 10 hours ago [-]
Completely agreed!
tptacek 15 hours ago [-]
I'm hanging back hoping 'woodruffw answers this so I don't have to. :)
woodruffw 17 hours ago [-]
Another year, another critical parsing vulnerability in the PGP ecosystem. Latacora has an excellent post[1] that touches on the excessive complexity of PGP's encoding which, remarkably, probably isn't even in the top 3 things wrong with PGP.
My personal favorite of these is when someone sent a weaponized compression packet to oss-sec in 2022[2].
The complaint about excessive complexity was about the packet length representation. That isn't a really great example. The PGP packet length representation is fairly straightforward.
woodruffw 17 hours ago [-]
You've linked this commentary in just about every PGP thread I've seen on HN, but the vulnerabilities keep coming. I don't think a dynamic TLV encoding was defensible a decade ago, and it certainly isn't defensible in 2025.
(As the Latacora post points out, this is the same essential error that cryptographic applications of BER make. The difference is that serious users of ASN.1 have mostly sobered up and switched to DER; no such sobering has happened in the PGP ecosystem.)
tptacek 17 hours ago [-]
You are writing this comment on a story where that packet format literally created a signature bypass vulnerability. Tell us more about how straightforward it is?
cbarrick 17 hours ago [-]
The issue discovered in TFA isn't about the format of individual packets (which "The PGP Problem" laments) but the grammar above the packets, i.e. the correct ordering of valid packets.
Edit: foot successfully inserted in mouth
tptacek 17 hours ago [-]
I wrote the "PGP Problem" article and definitely was not just talking about "the format of individual packets", which you might be able to tell from the fact that it brings up a quadratic parsing DoS in that design as well.
upofadown 17 hours ago [-]
Since you are the author of TPP could you please explain how your example shows some sort of quadratic parsing DOS? I don't see anything like that. Just a huge number of signatures.
tptacek 17 hours ago [-]
The postmortem on that DoS attack was at pains to point out that they were "officially" documenting only one of the vulnerabilities. Somewhere on an old drive I have the giant mountain of mailing list posts I sorted through to write that post, and I may dig it out at some point, but right now you're just going to have to take my word for the fact that I was not suggesting that the only problem with the packet format was the fact that you can represent lengths 8 different ways.
This is a deeply silly discussion to be having on this particular thread, which, again, is about a vulnerability that owes to the PGP packet format.
woodruffw 17 hours ago [-]
An absence of a canonical order or a definition of a well-formed packet sequence is itself a flaw in the packet format. Other cryptographic serialization and encoding schemes do not make this mistake.
The packet sequence used by this vulnerability was not a valid OpenPGP message, as pointed out by the blog post (under the header "An invalid packet list").
When we evaluate the design of a cryptosystem, we debit implementation vulnerabilities (at least in mainstream implementations) to the design. It is part of the goal of a cryptosystem design to foreclose on the possibility of implementation vulnerabilities.
twiss 11 hours ago [-]
I would usually tend to agree with that, I was mainly just responding to the specific claim that OpenPGP doesn't have a definition of a well-formed packet sequence, which is false.
Also, as a maintainer of OpenPGP.js, I'd say that while the complexity of OpenPGP certainly didn't help, quite a lot of things needed to go wrong to create this vulnerability:
- The message grammar validation was incomplete, as mentioned
- The streaming decryption/validation code affected how the packet sequence was processed
- A later optimization when not streaming affected it further in a way that caused an inconsistency in which packets were being read when
- Finally, the architecture of the code made it possible to return different data than what was verified, which should not have been possible (and we'll address this as well in a future refactor)
All in all, I would place more of the "blame" on OpenPGP.js rather than OpenPGP. That being said, I don't think placing blame is the most important here; both OpenPGP.js and OpenPGP should and will learn from this.
tptacek 10 hours ago [-]
What you're seeing me and 'woodruffw implying is that modern formats are (deliberately) trivial to parse. OpenPGP is not. That is a security design feature that OpenPGP lacks, and it's a big one, because it's caused multiple issues already.
twiss 9 hours ago [-]
Yes, I understand and never said I disagree with that :) I'm just trying to provide some nuance and background about the specific issue discussed in this submission, as IMHO I think it's more interesting than making the same high-level points about OpenPGP over and over in every thread related to it (no offense meant, of course).
deknos 11 hours ago [-]
with that argument TLS would be insecure, because there are insecure TLS implementations.
pvg 11 hours ago [-]
That's not the argument, it's that it's a bad design repeatedly shown to be shown to be prone to serious vulnerabilities and it's silly to argue it's not a bad design at yet another such time.
People have made serious arguments for all sorts of design problems in SSL/TLS.
deknos 4 hours ago [-]
the design is not bad per se. pgp is just a bit outdated and needs an overhaul.
And cryptographic serialization is just difficult.
egberts1 16 hours ago [-]
[flagged]
Rendered at 08:52:21 GMT+0000 (Coordinated Universal Time) with Vercel.
[1] https://blog.cryptographyengineering.com/2014/08/13/whats-ma...
[2] https://github.com/FiloSottile/age/issues/93
https://www.latacora.com/blog/2020/02/19/stop-using-encrypte...
But none of that excuses PGP's clanking and outmoded design. Things that are bad are bad. We can't moralize our way around that.
[1] Because I need to be in areas with bad connectivity often enough that I don’t want my every communication method to require a Internet connection to be available continuously (or even daily). Because Signal’s refusal to issue me an account without a phone number (which I cannot legally obtain without tying it to my government ID) is a real risk to my personal safety, given my particular situation. Because Signal’s stance on alternative implementations and the like is a hair’s breadth away from my refusing to use it. Some of these you could call moral arguments, but none of them are about PGP, as such.
What if your threat model isn't state-level actors? Well, the baseline of email these days is that your communication and your recipient's communication to the mail servers are both encrypted with TLS. The mail servers themselves may or may not communicate with each other using TLS, but if you really care about security, you can choose a mail server which will be using TLS. In other words, email is already at a baseline state of the only people aware of the message being the sender, the recipient, and their mail admins (and whomever these people choose to leak the message to, perhaps unwillingly). Encrypting email will only remove the mail admins from the list, and even then, they can still tell anybody whom you talked about.
So the use case of encrypted email boils down to wanting to hide the contents of communication but not hiding the fact of communication itself. Which isn't a broad use case; the best examples I've found in my own life is something like financial statements, but even in those cases, there's a pretty decent workaround: send an email saying "hi, we have a document for you online in our usual secure file repository" (protected by modern, useful standards for secure files). I don't like that only because I can't automate saving those files off to my own storage for my own purposes, but that is such a niche desire that I can understand why the bank doesn't bother.
So I think the metadata argument is dispositive. I agree with you that it's difficult to compose a coherent threat model that leaves metadata exposed the way SMTP does.
But the core argument of the piece is that encrypted email makes security concessions nobody would make if e.g. the wire for the down payment on their house was at stake, or if they were organizing against an oppressive state-level adversary. If encrypted email is unsuitable for those scenarios, then it seems more important to keep it from being used there than it does to accommodate the interests of people who using it for other reasons.
On the format level, CMS has some of the same flaws as PGP: dynamic TLV encodings (BER), extension points everywhere, and a disconnect between format and cryptographic versioning. On the cryptographic level, S/MIME benefits somewhat from certificates on the Internet PKI being less of a wild west than PGP certificates, and from having a community group (the S/MIME Cert WG of the CA/B Forum) invested in strengthening S/MIME's certificate profile beyond the baseline stipulated in RFC 5280. Of course, for non-public S/MIME deployments, none of that applies.
All that said, I don't think I would treat S/MIME (or CMS, or PKCS#7) as a guiding star: EFAIL affected S/MIME too[1]. But they have the "advantage" of being bad at just their niche (signing and encryption of email), versus being bad at every niche. The latter is PGP's historic curse.
[1]: https://efail.de/
Encrypted email can never have a good solution simply because email is the poster child for "Why Postel's Law is a bad idea"
My personal favorite of these is when someone sent a weaponized compression packet to oss-sec in 2022[2].
[1]: https://www.latacora.com/blog/2019/07/16/the-pgp-problem/
[2]: https://seclists.org/oss-sec/2022/q3/9
* https://articles.59.ca/doku.php?id=pgpfan:tpp
The complaint about excessive complexity was about the packet length representation. That isn't a really great example. The PGP packet length representation is fairly straightforward.
(As the Latacora post points out, this is the same essential error that cryptographic applications of BER make. The difference is that serious users of ASN.1 have mostly sobered up and switched to DER; no such sobering has happened in the PGP ecosystem.)
Edit: foot successfully inserted in mouth
This is a deeply silly discussion to be having on this particular thread, which, again, is about a vulnerability that owes to the PGP packet format.
The packet sequence used by this vulnerability was not a valid OpenPGP message, as pointed out by the blog post (under the header "An invalid packet list").
Part of the issue in OpenPGP.js was that it didn't fully validate the message packet grammar, which has now been fixed: https://github.com/openpgpjs/openpgpjs/pull/1853
Also, as a maintainer of OpenPGP.js, I'd say that while the complexity of OpenPGP certainly didn't help, quite a lot of things needed to go wrong to create this vulnerability:
- The message grammar validation was incomplete, as mentioned
- The streaming decryption/validation code affected how the packet sequence was processed
- A later optimization when not streaming affected it further in a way that caused an inconsistency in which packets were being read when
- Finally, the architecture of the code made it possible to return different data than what was verified, which should not have been possible (and we'll address this as well in a future refactor)
All in all, I would place more of the "blame" on OpenPGP.js rather than OpenPGP. That being said, I don't think placing blame is the most important here; both OpenPGP.js and OpenPGP should and will learn from this.
People have made serious arguments for all sorts of design problems in SSL/TLS.
And cryptographic serialization is just difficult.