I'm quoted in this article. Happy to discuss what we're working on at the Library Innovation Lab if anyone has questions.
There's lots of people making copies of things right now, which is great -- Lots Of Copies Keeps Stuff Safe. It's your data, why not have a copy?
One thing I think we can contribute here as an institution is timestamping and provenance. Our copy of data.gov is made with https://github.com/harvard-lil/bag-nabit , which extends BagIt format to sign archives with email/domain/document certificates. That way (once we have a public endpoint) you can make your own copy with rclone, pass it around, but still verify it hasn't been modified since we made it.
Some open questions we'd love help on --
* One is that it's hard to tell what's disappearing and what's just moving. If you do a raw comparison of snapshots, there's things like 2011-glass-buttes-exploration-and-drilling-535cf being replaced by 2011-glass-buttes-exploration-and-drilling-236cf, but it's still exactly the same data; it's a rename rather than a delete and add. We need some data munging to work out what's actually changing.
* Another is how to find the most valuable things to preserve that aren't directly linked from the catalog. If a data.gov entry links to a csv, we have it. If it links to an html landing page, we have the landing page. It would be great to do some analysis to figure out the most valuable stuff behind the landing pages.
josh-sematic 1 days ago [-]
A common metric for how much actual content has changed is the Jaccard Index. Even for large numbers of datasets that are too large to fit in memory it can be approximated with various forms of MinHash algorithms. Some write up here: https://blog.nelhage.com/post/fuzzy-dedup/
> sign archives with email/domain/document certificates
I do a bit of web archival for fun, and have been thinking about something.
Currently I save both response body and response headers and request headers for the data I save from the net.
But I was thinking that maybe if instead of just saving that, I could go a level deeper and preserve actual TCP packets and TLS key exchange stuff.
And then, I might be able to get a lot of data provenance “for free”. Because if in some decades when we look back at the saved TCP packets and TLS stuff, we would see that these packets were signed with a certificate chain that matches what that website was serving at the time. Assuming of course that they haven’t accidentally leaked their private keys in the meantime and that the CA hasn’t gone rogue since etc.
To me I think that would make sense to build out web archival infra that preserves the CA chain and enough to be able to see later that it was valid. And if many people across the world save the right parts we don’t have to trust each other in order to verify that data that the other saved was also really sent by the website our archives say it was from.
For example maybe I only archived a single page from some domain, and you saved a whole bunch of other pages from that domain around the same time so the same certificate chain was used in the responses to both of us. Then I can know that the data you are saying you archived from them really was served by their server because I have the certificate chain I saved to verify that.
kro 22 hours ago [-]
The idea is good, as far as I understand TLS however, the cert / asymmetric key is only used prove the identity/authenticity of the cert and thus the host for this session.
But the main content is not signed / checksummed with it, but rather a symmetrical session key, so one could probably manipulate this in the packet dump anyway.
I read about a Google project named SXG (Signed HTTP exchanges) that might do related stuff, albeit likely requiring the assistance of the publisher
ethbr1 14 hours ago [-]
To extend this to archival integrity without cooperation from the server/host, you'd need the client to sign the received bytes.
But then you need the client to be trusted, which clashes with distributing.
Hypothetically, what about trusted orgs standing up an endpoint that you could feed a URL, then receive back attestation from them as to the content, then include that in your own archive?
Compute and network traffic are pretty cheap, no?
So if it's just grabbing the same content you are, signing it, then throwing away all the data and returning you the signed hash, that seems pretty scalable?
Then anyone could append that to their archive as a certificate of authenticity.
catlifeonmars 7 hours ago [-]
Reminds me of timestamp protocol and timestamp authorities.
There are some special cases, like I think certain headers for signing e-mails, that do provide non-repudiation.
For that, `tcpdump` with `SSLKEYLOGFILE` will probably get you started on capturing what you need.
tomatocracy 21 hours ago [-]
In terms of tooling there's scoop[0] which does a lot of the capture part of what you're thinking about. The files it creates include request headers and responses, TLS certificates, PDF and screenshots and it has support for signing the whole thing as proof of provenance.
Overall though I think archive.org is probably sufficient proof that a specific page had certain content on a certain day for most purposes today.
Unfortunately, the standard TLS protocol does not provide a non-repudiation mechanism.
It works by using public key cryptography and key agreement to get both parties to agree on a symmetric key, and then uses the symmetric key to encrypt the actual session data.
Any party who knows the symmetric key can forge arbitrary data, and so a transcript of a TLS session, coupled with the symmetric key, is not proof of provenance.
There are interactive protocols that use multi-party computation (see for example https://tlsnotary.org/) where there are two parties on the client side, plus an unmodified server. tlsnotary only works for TLS1.2. One party controls and can see the content, but neither party has direct access to the symmetric key. At the end, the second party can, by virtue of interactively being part of the protocol, provably know a hash of the transaction. If the second party is a trusted third party, they could sign a certificate.
However, there is not a non-interactive version of the same protocol - you either need to have been in the loop when the data was archived, or trust someone who was.
The trusted third party can be a program running in a trusted execution environment (but note pretty much all current TEEs have known fault injection flaws), or in a cloud provider that offers vTPM attestation and a certificate for the state (e.g. Google signs a certificate saying an endorsement key is authentically from Google, and the vTPM signs a certificate saying a particular key is restricted to the vTPM and only available when the compute instance is running particular known binary code, and that key is used to sign a certificate attesting to a TLS transcript).
Another solution is if the server will cooperate with a TLS extension. TLS-N (https://eprint.iacr.org/2017/578.pdf) provides a solution for this. That provides a trivial solution for provenance.
Intralexical 4 hours ago [-]
As important as cryptography is, I also wonder how much of it is trying to find technical solutions for social problems.
People are still going to be suspicious of each other, and service providers are still going to leak their private keys, and whatnot.
3np 20 hours ago [-]
You may be interested in Reclaim Protocol and perhaps zkTLS. They have something very similar going and the sources are free.
It’s an interesting idea for sure. Some drawbacks I can think off:
- bigger resource usage. You will need to maintain a dump of the TLS session AND an easily extractable version
- difficulty of verification. OpenSSL / BoringSSL / etc. will all evolve and say, completely remove support for TLS versions, ciphers, TLS extensions… This might make many dumps unreadable in the future, or requiring the exact same version of a given software to read it. Perhaps adding the decoding binary to the dump would help, but then, you’d get Linux retro-compatibility issues.
- compression issues: new compression algorithms will be discovered and could reduce data usage. You’ll have a hard time doing that since TLS streams will look random to the compression software.
I don’t know. I feel like it’s a bit overkill — what are the incentives for tampering with this kind of data?
Maybe a simpler way of going about it would be to build a separate system that does the « certification » after the data is dumped; combined with multiple orgs actually dumping the data (reproducibility), this should be enough the prove that a dataset is really what it claims to be.
mrshadowgoose 1 days ago [-]
Just commenting to double-down on the need for cryptographic timestamping - especially in the current era of generative AI.
_heimdall 1 days ago [-]
How does that work exactly? Does it all still hinge on trusting a know Time Stamp Authority, or is there some way of time stamping in a trustless manner?
yencabulator 12 hours ago [-]
I'm so sad roughtime never got popular. It can be used to piggyback a "proof of known hash at time" mechanism, without blockchain waste.
You can publish the hash in some durable medium, like the classified section of a newspaper.
This proves you generated it before this time.
You can also include in the hash the close of the stock market and all the sports scores from the previous day. That proves you generated it after that time.
ethbr1 14 hours ago [-]
My mind immediately went to adversarial fixing of all sports games and the stock market in order to create a collision.
Sports sports are an interesting source of entropy.
3eb7988a1663 12 hours ago [-]
If you are looking to prove that something happened after a certain timestamp, you can use a randomness beacon[0]. Every <interval>, the beacon outputs a long random number. Include the timestamped random number into your artifact.
You are relying upon the authority of the beacon to be random, but good practice is to utilize multiple independent beacons.
This is the one thing blockchains are truly good for.
_heimdall 1 days ago [-]
Yeah it definitely could be, though you may similarly find yourself in a spot of trusting a limited number of nodes that guarantee the chain was never tampered with.
Retric 23 hours ago [-]
For something like this there’s ways to minimize how much you need to trust nodes such as regularly publishing hashes to 3rd parties like HN.
Not so useful if something was edited a few minutes after posting, but it makes it more difficult for a new administration to suddenly edit a bunch of old data.
kergonath 21 hours ago [-]
> there’s ways to minimize how much you need to trust nodes such as regularly publishing hashes to 3rd parties like HN.
But you could do the same thing with any hashes, right? There is no need for a blockchain in the middle.
Retric 17 hours ago [-]
What happens as websites disappear? With a blockchain in 2090 you can point to a website post in 2060 as support that your hashes on data posted in 2030 are still valid. That’s useful when preventing people from rewriting history is the goal.
There’s also a size advantage. You can keep a diff on the archive for each hash being posted instead of the full index for every time you post a hash.
mrshadowgoose 1 days ago [-]
You make use of several independent authorities for each timestamped document.
The chance is exceedingly low that the PKI infrastructure of all the authorities becomes compromised.
I'd love to learn more about what is in scope of the Library Innovation Lab projects. Is it targeting data.gov specifically or all government agency websites?
Given the rapid take downs of websites (cdc, usaid) do you have a prioritization framework for which website pages to prioritize or do you have "comprehensive" coverage of pages (in scope of the project)?
As you allude to, I've been having a hard time learn about what sort of duplicate work might be happening given that there isn't a great "archived coverage" source of truth for government websites (between projects such as End of Term archive, Internet archive, research labs, and independent archivists).
Your open questions are interesting. Content hashes for each page/resource would be a way to do quick comparisons, but I assume you might want to set some threshold to determine how much it's changed vs if it changed?
Is the second question about figuring out how to prioritize valuable stuff behind two depth traversals? (ex data.gov links to another website and that website has a csv download)
JackC 1 days ago [-]
As a library, the very high level prioritization framework is "what would patrons find useful." That's how we started with data.gov and federal Github repos as broad but principled collections; there's likely to be something in there that's useful and gets lost. Going forward I think we'll be looking for patron stories along the lines of "if you could get this couple of TB of stuff it would cover the core of what my research field depends on."
In practice it's some mix of, there aren't already lots of copies, it's valuable to people, and it's achievable to preserve.
> Is the second question about figuring out how to prioritize valuable stuff behind two depth traversals?
Right -- how do you look at the 300,000 entries and figure out what's not at depth one, is archivable, and is worth preserving? If we started with everything it would be petabytes of raw datasets that probably shouldn't be at the top of the list.
glitchcrab 7 hours ago [-]
Very tangentially related, but it always makes me smile to see rclone mentioned in the wild - its creator ncw was the CEO of the previous company I worked at.
jszymborski 1 days ago [-]
Hi! Is there any one place that would be easiest for folks to grab these snapshots from? Would love to try my hand at finding documents that moved/documents that were removed.
Unfortunately it's a bit messy because we weren't initially thinking about tracking deletions. data_20241119.jsonl.zip (301k rows) and data_20250130.jsonl.zip (305k rows) are simple captures of the API on those dates. data_db_dump_20250130.jsonl.zip (311k rows) is a sqlite dump of all the entries we saw at some point between those dates. My hunch is there's something like 4,000 false positives and 2,000 deletions between the 311k and 305k set, but that could be way off.
jszymborski 1 days ago [-]
Very cool! I take a look :)
LastTrain 1 days ago [-]
How can people help? Sounds like a global index of sources is needed and the work to validate those sources, over time, parceled out. Without something coordinated I feel like it is futile to even jump in.
JackC 1 days ago [-]
I spent a bunch of time on this project feeling like it was futile to jump in and then just jumped in; messing with data is fun even if it turns out someone else has your data. But the government is huge; if you find an interesting report and then poke around for the .gov data catalog or directory index structure or whatever that contains it, you're likely to find a data gathering approach no one else is working on yet.
There's coordinated efforts starting to come together in a bunch of places -- some on r/datahoarders, some around specific topics like climate data (EDGI) or CDC data, there's datasets being posted on archive.org. I think one way is to find a topic or kind of data that seems important and search around for who's already doing it. Eventually maybe there'll be one answer to rule them all, but maybe not; it's just so big.
smrtinsert 1 days ago [-]
Thank you for this effort.
alexvoda 1 days ago [-]
Trump did this last time too. Is there a difference in the level of preparedness in archiving data compared to last time? If so, in what way is it different? Is there institutional or independent preparedness?
JackC 1 days ago [-]
(Note my lab isn't partisan and this isn't a partisan effort; public data always needs saving. But there's definitely a reason people are paying attention right now.)
I think in some ways the community was less prepared this time, because there was a lot of investment in 2016-2017 and then many of the archives created at that point didn't end up being used; partly because the changes at the federal level turned out to be smaller and slower in 2017 than they're looking like this time. So some people didn't choose to invest that way this time around.
[Edit: this means I think it's really important that data archives are useful. Sorting through data and putting a good interface on it should help people out today as well as being good prep for the future.]
In other ways there's much more preparation; EOT Archive now has a regular practice of crawling .gov websites before and after each change of administration, which is a really great way of giving citizens a sense of how their government evolves. It will just tend to miss data that you can't click to in a generic crawl.
What in the Cold War conspiracy theory was that...
>> The Kremlin’s design necessarily depends on the adoption of a single world belief system or religion. Expect a syncretic, gnostic blend, rooted in hierarchy — the Russian Orthodox Church at the core, and other religious factions accorded favor based on demonstrated fealty.
Good luck with that. The most popular religions today are forked versions of one guys story that they couldn't agree on and have been involved in acts of genocide towards one another because of it--for centuries.
0n0n0m0uz 1 days ago [-]
One of the USA greatest strengths is the almost unprecedented degree of transparency of governments records going back decades. We can actually see the true facts including when our government has lied to us or covered things up. Many other nations do not have this luxury and it has provided the evidentiary basis for both legal cases and "progress" in general. Not surprising that authoritarians would target and destroy data as it makes their objective of a post-truth society that much easier
bamboozled 21 hours ago [-]
It’s also the driver for a great economy imo. Why can’t Russia and China build and innovate like the USA can?
Because they spend a lot of time censoring and covering things up.
RIP USA.
johnnyanmac 9 hours ago [-]
China as of late is indeed running circles around the US. We spend 3 decades depending on them for manufacturing, and surprise. They got really good at manufacturing stuff for themselves when the chips are down.
I guess great economies also give it away. RIP USA.
stackedinserter 12 hours ago [-]
> Why can’t Russia and China build and innovate like the USA can?
> Because they spend a lot of time censoring and covering things up.
What a statement! How did you come to this conclusion?
bamboozled 12 hours ago [-]
Who built all Russia’s oil infrastructure ? Who has the strongest economy? Who builds the airliners ? Has the most advanced space programs ? Landed men on the moon ? Etc ?
It’s not just America either. Europe too. Australia.
Authoritarians copy and steal. Democracies innovate and adapt.
Democracies are more transparent and honest, which leads to better outcomes in all areas.
Time spent hiding information to appease the dictator and protecting “the party” is time wasted.
achierius 11 hours ago [-]
> Who built all Russia's oil infrastructure ?
The Soviet Union? Yeah, the new offshore stuff (mostly? developed after the fall of the USSR) relied on western firms, but I believe the bulk of oil production still comes from / goes through domestically sourced systems. Gas production on the other hand was initially almost entirely built out by western firms, though that's obviously begun to change since the war in Ukraine.
hylaride 10 hours ago [-]
A not insignificant percentage of Soviet-era drilling and pipeline pumping equipment and tech is/was western. There's even many allegations of CIA sabotage before delivery (which is hard to prove).
cess11 11 hours ago [-]
Do you apply the same thinking to the workplace?
chrishoyle 1 days ago [-]
Beyond federal websites (.gov, .mil) there are lot of gov contractor websites that are being taken down (presumably at the demand of agencies) that contain a wealth of information and years of project research.
The pedestrian "right", which I encounter on a day-to-day basis the months I visit client sites a couple hundred miles inland of the Gulf of America, will look at climatelinks.org and say something like: "all I see are foreign countries, why are we spending money on this instead of citizens of the United States?".
bamboozled 12 hours ago [-]
Yeah, what has avoiding another plague ever done for the USA.
johnnyanmac 9 hours ago [-]
"We're America, we wait until it's too late and then react!"
A rough paraphrasing from Boondocks, said by the richest man in that neighborhood.
cscurmudgeon 1 days ago [-]
[flagged]
neaden 15 hours ago [-]
The US provided 14% of the WHO funding but is 25% of global GDP, so proportionately we don't contribute as much as many other countries.
harimau777 24 hours ago [-]
IMHO, we should do it because the person who pays tends to have more power over what happens. Just like how in high school the kid who drives everyone tends to have a higher than normal say in what the friend group does.
honestSysAdmin 17 hours ago [-]
Smart.
sanp 21 hours ago [-]
We wouldn’t know this if the information isn’t shared? So, aren’t you making a case for not removing this information?
kergonath 21 hours ago [-]
> Why should US fund WHO ~5-6 times more than China [0] (and more than EU)
The base contributions are a function of GDP. The extra contributions are voluntary, and the US did it because it was in the US’ interests. It’s a founding error in the US foreign policy budget and was a good investment in terms of goodwill and data for American health research institutions.
WHO must focus where it is needed most. Public health is much better in the EU (and even in Europe, accounting for places like Belarus and Ukraine) than in China, and there are much fewer epidemics that emerge in Europe in general.
The whole idea is that if we limit the emergence of epidemics where they are likely to happen, we end up with fewer pandemics after these epidemics spread worldwide (which includes Europe and North America). The whole world is better without another COVID, Ebola, or Polio.
> only to have the WHO be controlled by China
This is bullshit. The WHO is not controlled by China any more than other UN institutions. What is certain, though, is that the US won’t have any say whatsoever once they are out.
AuryGlenz 12 hours ago [-]
“The whole idea is that if we limit the emergence of epidemics where they are likely to happen, we end up with fewer pandemics after these epidemics spread worldwide”
I realize I’m arguing against a negative but has that actually been accomplished? I don’t argue that they (I assume) probably help with things like Ebola outbreaks but that’s almost certainly never going to become a pandemic.
crusty 12 hours ago [-]
Prior to 2014, it was thought that ebola outbreaks were naturally self limiting to an extent. Woops.
cle 1 days ago [-]
I’ve been archiving data.gov for over a year now and it’s not unusual to see large fluctuations on the order of hundreds or thousands of datasets. I’ve never bothered trying to figure out what exactly is changing, maybe I should build a tool for that…
nemofoo 1 days ago [-]
Do you mirror these data sets anywhere?
cle 1 days ago [-]
It's not in any sort of format to do this kind of analysis unfortunately. I'm also missing some data b/c I throw away certain kinds of datasets that are not useful for me. I can probably write some scripts to diff my archives with the current data.gov and see what's missing, but it won't be "complete". But it might still be useful...
I did however just write a Python script to pull data.gov from archive.org and check the dataset count on the front page for all of 2024, here are the results:
As you can see, there were multiple drops on the order of ~10,000 during 2024. So it's not that unusual. There could be something bad going on right now, but just from the numbers I can't conclude that yet.
(Specifically it takes the first snapshot of every Wednesday of 2024).
If I get around to re-formatting my archives this week, I'll follow up on HN :).
jl6 1 days ago [-]
> The outlet reports that deleted datasets "disproportionately" come from environmental science agencies like the Department of Energy, National Oceanic and Atmospheric Administration (NOAA), and the Environmental Protection Agency (EPA).
Was there an EO targeting these areas?
_DeadFred_ 1 days ago [-]
Looks like the EPA is being targeted (Even though ninety-five percent of the funding going to EPA has not only been appropriated, but is locked in, legally obligated grant funding. The Constitution does not give the president a line item veto over Congress's spending decisions):
The President's ability to affect spending is definitely limited, and hasn't been exercised really since Reagan, but still exists.
Congress rarely makes spending money it's goal, rather it appropriates money to accomplish some goal. Which is to say that if Congress wants a bridge across a river and appropriate 10 billion to build it, the President is not obligated to spend $10 billion if 7 or 8 or 9 will do. In some cases, Congress does appropriate money toward causes and intends all the money to be spent in furtherance of some guiding principle and in these cases all the money must be spent.
rhinoceraptor 1 days ago [-]
It's not limited if he hijacks the entire government from the inside and installs loyalists everywhere
_DeadFred_ 1 days ago [-]
If that money has already been awarded to be given out the executive can not arbritrality withdraw it. The Administrative Procedure Act (APA) of 1946 contains the rule that prevents the U.S. executive branch from acting in an arbitrary and capricious manner. Trump was already slapped for doing this by the Supreme Court in Department of Homeland Security v. Regents of the University of California.
zeagle 1 days ago [-]
I think it has become pretty clear Trump can do whatever the hell he wants because nobody will call him out or prosecute him for it.
NewJazz 1 days ago [-]
Or if he gets convicted of a crime by a jury, he receives no consequences.
chgs 20 hours ago [-]
It’s a travesty that he wasn’t tried for Jan 6 in 4 months let alone 4 years.
Justice delayed is justice denied.
trashtester 15 hours ago [-]
I believe they tries impeaching him in 2021. At the time, though, even people like Tucker thought his career was over after his behaviour.
But I suppose 4 years with Biden/Kamala made it easier to forgive him, for many. Not just the MAGA base, but even swing voters.
Honestly, I think Democrats were contributing to this by the outrage over the January 6 riots. Half the country would consider the BLM riots as equally bad.
If instead, Democrats had focused on Trump's betrayal of Pence and general disregard for the institutions and traditions of the country, a lot more moderates would remain with the Democrats.
But as a foreigner, it seems to me that the prosecution of the Jan 6 rioters, not to mention Trump himself was excessive and overtly politically motivated. And it definitely took attention away from the less spectacular, but far more obviously immoral behavior he definitely, provably WAS guilty of leading up to January 6.
trashtester 20 hours ago [-]
I'll probably be downvoted for this, but don't you think winning the popular vote is similar to being found "not guilty" after appealing to a higher court?
voltaireodactyl 19 hours ago [-]
I for one do not think that they are similar because an election is a popularity contest not a logical examination of facts, and voters are not required to sit through a thorough presentation of the facts in evidence, which means they tend to make less than ideal jurors.
saalweachter 16 hours ago [-]
It's at least a little akin to jury nullification, where the jury finds an obviously guilty person not guilty out of a belief the law is unjust.
(You hear about it on the Internet as a way to Fight The Man on eg arrests stemming from protests or minor drug charges, but historically it was more often used to absolve white supremacists of murder.)
trashtester 15 hours ago [-]
Don't you think this is precisly why there is a jury system in the first place?
It seems that most legal systems from the start was intended to codify what was considered "just and fair" in the eyes of "the people".
Juries seem to have been put in place specifically to ensure that the legal system operates within this mandate, and to preven overreach or abuse.
tsimionescu 12 hours ago [-]
The jury system is not intended as a substitute for the law, it's intended as a safeguard to protect from the subjectivity of judges. Juries, like all of the legal system, are still intended to be subordinate to the legislative branch's decisions in terms of what is and isn't wrong.
Jury nullification is a weird rule that goes against the constitutional framework, but is so rarely used, at least for any important matter, that it has never really received too much scrutiny.
trashtester 15 hours ago [-]
There are also some that feel he was unjustly singled out by politically motivated prosecutors.
But if you you think there is no risk that the justice system can be misused for such political ends, then I suppose you have the same problems with Biden pardoning Hunter and half his family to protect them from politically motivated prosecution?
Or is it only wrong when Republicans do it?
At minimum I would say that the general public in a general election should have the same power to effetively pardon someone as the President has. After all, the electorate is the basis from which the President draws his power and legitimacy.
And not only for the President, by the way. The legitimacy of the entire system, including both Congress and the Legal System draws its legitimacy from the same source (even if the Constitution is designed to provide protection against short term simple majorities).
If the outcome of the vote cannot be accepted, then that basically means the country cannot remain a democracy.
Personally, I don't think I could have voted for Trump after how he behaved in January 2021 (if I were a citizen). Primarily because of how he betrayed Pence.
But Biden and Kamala sure made it a lot easier for him in 2024 than it would have been if the country had been led by someone like Obama.
neaden 15 hours ago [-]
The difference is Trump prosecution wasn't politically motivated, he legit did the crimes. Outside of Hunter Biden, who never actually had a role in the administration, no one did anything illegal. Trump has been produced to an unprecedented degree because he committed crimes to an unprecedented degree.
mschuster91 14 hours ago [-]
> But if you you think there is no risk that the justice system can be misused for such political ends, then I suppose you have the same problems with Biden pardoning Hunter and half his family to protect them from politically motivated prosecution?
Well, Trump made his intentions of prosecuting Fauci, Hunter Biden and many many others more than abundantly clear. He can't whine about Biden protecting people from the threat he himself had announced. (Well, he obviously can and does, and half the population falls for it)
zeagle 12 hours ago [-]
The general population is not able to grasp broader issues like COVID 19 response, climate change, addressing inequality and systemic barriers to health/employment/life that don't directly impact themselves, or understand that you can't just hit spokes with a wrench when it comes to the government without the system falling apart anymore than my cat can plan ahead to get dinner. And I am not saying I am smarter in areas outside my lane and also get manipulated but there is no uncertainty he, were he ANYONE else, would be guilty of at least some of what has been investigates (Jan 6, hush money to say least) which has been pushed away for political reasons by those propping him up. What you say just shows people can be manipulated by media, group think, religion and oligarchs into voting against their own interests yet again and zero, truly zero pity that hurt themselves supporting him. Us Canadians have our own populists too on the right working against my values (Smith, PP, Ford) but at least they aren't criminals out for revenge.
immibis 9 hours ago [-]
I take it that you think there should be no requirements to be president, since winning the popular vote when you are ineligible is similar to the population deciding you are actually eligible?
beretguy 18 hours ago [-]
> Majority rule don't work in mental institutions
NOFX - "The Idiots Are Taking Over"
johnnyanmac 9 hours ago [-]
Judges thankfully did call him out. Prosecute? Who knows.
1 days ago [-]
johnnyanmac 9 hours ago [-]
Clinton did try to pass something but it was vetoed. Pretty much all case law in this area has ruled thst Congress's budget is final and very hard to modify.
Trump's case was a completely constitutional overreach. Thankfully they shot it down fast in court.
ks2048 1 days ago [-]
I really don't think "Trump can't do it, if it's illegal" is going to fly this time. I hope so, but I'm not very confident.
Of course, he can't get away with too much alone, but with the right appointees, judges, etc, who knows.
ks2048 1 days ago [-]
And people saying "Congress controls spending, not the President" - there are already reports of Musk trying to take control of the system that sends out money.
I'm more afraid of Musk getting access to IRS information and using that information against people/intimidate/blackmail. Remember people are compelled to report all kinds of things on the understanding that 1. It won't be used against them and 2. Will be kept confidential. If Musk breaks that not only does he wield a powerful weapon against those he dislikes, but suddenly it could be argued 'the right to self incrimination' comes into play on taxes.
sidewndr46 1 days ago [-]
I wouldn't be afraid of it in any capacity. I'd plan for it to happen
exe34 1 days ago [-]
he'll tweet them in revenge.
sidewndr46 1 days ago [-]
Isn't this whole thing just a bunch of political grandstanding?
Train v. City of New York isn't a constitutional case. So if Congress controls spending, it's a power they gave to themselves. Any other portion of government could likewise determine they control spending. So maybe it goes back to the SCOTUS because they agree to hear another case about it. Maybe they like the idea that the executive branch controls spending? Or maybe they don't after all. The tell the President to stop doing that. But the president does not. Does Congress pass another law reaffirming they control the power for spending?
If the executive branch is in charge of running the government and they listen to the President & the President decide's they aren't spending any money, who exactly is going to change that? Would Congress fund another department to change this? Would it also just report to the President?
From what I understand it's actually pretty common for the executive branch not to actually be able to spend appropriated money. The simplest case would be if you put out a contract and get no bids on it. There are other cases, like Congress agreeing to fund military submarines for Taiwan. But no one is going to build them internationally and we don't build diesel-electric combat submarines in the US as a matter of policy.
neaden 15 hours ago [-]
The legislature is the supreme power in the federal government with the power to impeach the other two branches. So no, it's not political grandstanding. Now if the congressional majority allows for the executive to take an illegal amount of power there's no one to stop that, in the same way that I'd the police stop enforcing laws against their own family members there's not really anyone to enforce those laws in their place.
lowercased 1 days ago [-]
Congress appropriates money to be spend. Executive branch ... 'executes' things with appropriated money. Previous Trump administration tested a lot of norms, and current one is going to be pushing past every boundary of normal/accepted/traditional behaviour we've ever seen. The will of the people, expressed through Congress, is that Dept X gets $Y, but the incoming admin is claiming their own 'will of the people mandate' to completely upend how appropriated money is spent, and indeed, if it's spent at all. When you do not believe government agencies should exist, why would you want to enable them to spend money?
ks2048 1 days ago [-]
Well, we'll see what the courts do - and maybe more importantly - what the people on the street do when/if it starts affecting their lives.
I'm not a legal scholar, but I'm going to guess that saying "congress appropriates money, but the President can decide if they actually get it" is a pretty fringe idea.
__loam 1 days ago [-]
It's called impoundment and it's patently illegal and unconstitutional. There's already been a SCOTUS case. The executive branch absolutely does not have the authority to decide how money is allocated.
BlarfMcFlarf 1 days ago [-]
The executive branch having the power of selective enforcement and pardon generally seems like it might have an impact on the relevance of any supposed authority.
rectang 1 days ago [-]
It's easier to beg forgiveness than to ask permission, and it's easier still to do neither.
lowercased 1 days ago [-]
Most of the MAGA ideas are fringe. And they're pissed they've not been 'listened to' for years. That's part of the point. This is all revenge/retribution, both on Trump's part, but also for a minority class of people who've felt 'left out' and 'ridiculed'.
__loam 1 days ago [-]
We ridiculed them because their ideas, like slapping blanket tariff on our closest allies, are really really stupid.
Viliam1234 14 hours ago [-]
Yeah, but that doesn't make them any less angry.
johnnyanmac 9 hours ago [-]
Shame, maybe they should have focused on subsidizing therapy.
I don't even know what more to say. Paradox of tolerance so I'm no even going to pretend to empathize with ideas like"deport the illegals in chains " and ćmy body your choice".
watwut 7 hours ago [-]
[dead]
vixen99 17 hours ago [-]
There are other sites where you can deliver this kind of homily to your heart's desire.
Nuzzerino 18 hours ago [-]
Hmm, I'm not sure if "is it legal" would be at the top of my priority list if I was working (in good faith) on the team to recover the country from a nose dive and prevent world war 3. But it is ok to disagree on whether those things actually are real imminent risks to fix.
Erem 1 hours ago [-]
How can someone in good faith try to “recover” a nation governed by laws by publicly flouting the law?
softwaredoug 1 days ago [-]
I don't get why there's not legal action then? Maybe its a matter of nobody having standing?
toyg 21 hours ago [-]
There is some action , certain states have already started suing. But it takes some time and effort to file suits that are watertight, it's harder and slower than just writing illegal, unconstitutional, or inapplicable acts
johnnyanmac 9 hours ago [-]
Trump is basically doing a blitzkrieg of EO's. Lawsuits are happening slowly between judges injuctioning Trump's budget freeze, employees suing the government over the OP issues, and states suing over various orders. But so much is happening at once that it's hard to keep track of your full time job isn't politics.
TheBlight 1 days ago [-]
[flagged]
dangrossman 1 days ago [-]
> "While 5 CFR 315 does permit immediate termination, it does not permit arbitrary termination. The termination must be related to unsatisfactory performance or conduct (section 804) or conditions arising before employment, which usually means something from your background investigation (section 805)..."
The Administrative Procedure Act (APA) of 1946 contains the rule that prevents the U.S. executive branch from acting in an arbitrary and capricious manner. Definitely not legit.
eadler 15 hours ago [-]
The Civil Service Reform Act (CSRA) precludes APA review amongst other relief (since it precludes district court jurisdiction entirely). That being said, this "termination" is likely invalid under the CSRA.
Please see Elgin v. Dept. of Treasury 567 U.S. 1 (2012) for details.
I would not take definitions of "arbitrary" or "capricious" for granted. Legal words never mean what most people think they mean, especially once the Supreme Court gets their hands on it.
_DeadFred_ 1 days ago [-]
This term is well defined in settled case law over the last 80 years. Thee Supreme Court already ruled Trump can not do these sort of arbitrary and capricious executive orders his last term.
I'm not sure he really cares if these actions get overturned. At least some of them will be sustained, even if only because somebody's lawyers faffs oral arguments. And I suspect that he'll just ignore the court in some cases. How many legions hath the Supreme Court?
johnneville 1 days ago [-]
Politico reports that USDA landing pages regarding climate change were ordered to be deleted by a directive from the USDA's office of communications.
I think it is likely that orders to these other agencies follows this model. Many other datasets are being targeted via EO 14168 which has quite wide impacts but doesn't appear at first glance to apply to what i would expect to be a part of NOAA and EPA reports.
Don’t worry, it is a matter of great doctrinal import that all scientific datasets be replaced with datasets that have been properly refined in accordance with scripture. /s
Maybe this administration will get better over time?
uni_rule 1 days ago [-]
Nah, the whole executive branch is getting Jack Welch'ed. Hopefully your tap water cleanliness regulations are strong on a state level.
BeefWellington 1 days ago [-]
After the utter bullshit pulled in California, better hope your state is willing to defend its water reservoirs or for some places clean tap water may be the least of their problems.
What's a good way to be an "Archivist" on a low budget these days?
Say you have a few TBs of disk space, and you're willing to capture some public datasets (or parts of them) that interest you, and publish them in a friendly jurisdiction - keyed by their MD5/SHA1 - or make them available upon request. I.e. be part of a large open-source storage network, but only for objects/datasets you're willing to store (so there are no illegal shenanigans).
Is this a use case for Torrents? What's the most suitable architecture available today for this?
josh-sematic 1 days ago [-]
I’m not an expert in such things, but this seems like a good use case for IPFS. Kinda similar to a torrent except that it is natively content-addressed (essentially the key to access is a hash of the data).
Set up a scrape using ArchiveTeam's fork of wget. It can save all the requests and responses into a single WARC file. Then you can use https://replayweb.page/ or some other tool to browse the contents.
honestSysAdmin 1 days ago [-]
In my experience, to archive effectively you need a physical datacenter footprint, or to rent capacity of someone who does. Over a longer timespan (even just 6 months), having your own footprint is a lower total cost of ownership, provided you have the skills or access to someone with the skills to run Kubernetes + Ceph (or something similar).
.
> Is this a use case for Torrents?
Yes, provided you have a good way to dynamically append a distributed index of torrents and users willing to run that software in addition to the torrent software. Should be easy enough to define in container-compose.
crowcroft 1 days ago [-]
Still, even with best efforts this is such a shame. There is always going to be a question around governance over the data, integrity, and potentially chain of custody as well. If the goal is to muddy the waters and create a narrative that whatever might be in this data isn't reliable or accurate then mission accomplished. I don't see how anything can stop that.
Not to say the data isn't incredibly valuable and should be preserved for many other reasons of course. All the best to anyone archiving this, this is important work.
debeloo 2 days ago [-]
Is this normal when there's change in presidency?
meesles 2 days ago [-]
From the article:
> Changes in presidential administrations have led to datasets being deleted in the past, either on purpose or by accident. When Biden took office, 1,000 datasets were deleted according to the Wayback Machine, via 404 Media's reporting.
derbOac 2 days ago [-]
I think the question is the nature of the losses in the two cases, the transparency circumstances about them, and who exactly is making the decisions about specific datasets.
Time will tell but loss of public datasets is probably not usually good in general.
This is not a direct quote, the actual quote from the article is
> But archivists who have been working on analyzing the deletions and archiving the data it held say that while some of the deletions are surely malicious information scrubbing, some are likely routine artifacts of an administration change, and they are working to determine which is which. For example, in the days after Joe Biden was inaugurated, data.gov showed about 1,000 datasets being deleted as compared to a day before his inauguration, according to the Wayback Machine.
akudha 24 hours ago [-]
Why are the datasets deleted though? Biden or Trump, Democrats or Republicans - What do they gain?
I don’t see a list of the datasets that have gone missing. Is there a list?
mistrial9 1 days ago [-]
[flagged]
NortySpock 1 days ago [-]
And you could also run your own archive bot (x86 only). I've got one running in a docker container, it downloads a webpage and auto-uploads it to archive.org
archiveteam:
image: atdr.meo.ws/archiveteam/warrior-dockerfile
ports:
- '8101:8001'
mem_limit: 4G
cpus: 3
dns:
- 9.9.9.10
- 8.8.8.8
labels:
- com.centurylinklabs.watchtower.enable=true
container_name: archiveteam-warrior
environment:
- DOWNLOADER=asdf # Change this to your nickname
- SELECTED_PROJECT=auto # Change this to your project of preference or let the archiveteam decide with 'auto'
- CONCURRENT_ITEMS=6 # Change this to the amount of concurrent download threads you can handle
watchtower:
command: '--label-enable --include-restarting --cleanup --interval 3600'
cpu_shares: 128
mem_limit: 1G
cpus: 1
image: containrrr/watchtower
volumes:
- '/var/run/docker.sock:/var/run/docker.sock'
container_name: watchtower
derektank 1 days ago [-]
Does anyone know if the St Louis Federal Reserve (and I guess the federal reserve banks generally) is subject to presidential executive orders or is it entirely responsible to the Federal Reserve Board and the St. Louis Bank president? FRED is the only dataset I access regularly
choobacker 1 days ago [-]
It's impressive that volunteers are stepping up to archive this. I understand the desire to keep this open data available.
How much of this sort of effort results in that data being used? Are there success stories for these datasets being discoverable enough and useful to others?
liontwist 9 hours ago [-]
I think people are interested in archiving and the political image associated with that but I don’t think anybody cares about the content. Who is going to go back and read Biden era agency publications?
andyjohnson0 17 hours ago [-]
If the intention is to restore these data sets at some future date, when sanity has possibly been restored, then there needs to be a way to demonstrate that the archived data hasn't itself been modified. Without that, malign actors (e.g. oil/gas lobby) could very easily poison the future.
DJBunnies 16 hours ago [-]
If only we had checksums or file hash integrity.
andyjohnson0 7 hours ago [-]
If the originals are gone, and if the original authoritative, trustworthy data custodian no longer exists, then how will we know what the checksum/hash should be?
generalizations 1 days ago [-]
Do we know what datasets these are? Do we actually have a diff here so we know what's been removed? There's a lot of assumptions being thrown around here, but we don't even know if this is some kind of malicious compliance. An actual list of what's been removed would probably clear the air a lot.
As one of the reddit comments (in the thread linked by the article) pointed out,
> During the start of Biden’s term, On 6th feb data.gov had “218,384 DATASETS” but on 7th feb it only had “192,180 DATASETS”
2 days ago [-]
smrtinsert 2 days ago [-]
Are datasets mirrored anywhere where the govt doesn't automatically have a take down authority? If not there should be a mirroring effort.
don't they have to have to have done this /before/ it gets deleted?
downrightmike 10 hours ago [-]
Already seeing: 404 Not Found: Requested route ('ed-public-download.app.cloud.gov') does not exist.
bawolff 1 days ago [-]
Tbh, im kind of surprised these things weren't being archived as they were being published. Trump is an extreme case, but its not the first time a change in administration resulted in removing old websites.
1 days ago [-]
notavalleyman 2 days ago [-]
I read, in past days, that the man who ordered the construction of the nearly infinite Wall of China was that First Emperor, Shih Huang Ti, who likewise ordered the burning of all the books before him. That the two gigantic operations - the five or six hundred leagues of stone to oppose the barbarians, the rigorous abolition of history, that is of the past - issued from one person and were in a certain sense his attributes, inexplicably satisfied me and, at the same time, disturbed me.
Yes; and for further reference, the non-Pinyin transliteration is https://en.wikipedia.org/wiki/Wade%E2%80%93Giles and the "add/drop" is that instead of the title 秦始皇 Qín Shǐ Huáng (lit. "first Qin emperor") Borges is using the title 始皇帝 Shǐ Huángdì (lit. "first emperor").
It's astonishing how little information that phonetic representations of Chinese allow to be retained.
Qin Shi Huang/Shih Huang Ti both means [Qin]Founding Emperor[-King]. In layperson's approximation, He's so ancient that his canonical name and title is debatable, and he's almost always referred to as "the first emperor" or variants thereof.
belter 1 days ago [-]
“Now I will tell you the answer to my question. It is this. The Party seeks power entirely for its own sake. We are not interested in the good of others; we are interested solely in power, pure power. What pure power means you will understand presently. We are different from the oligarchies of the past in that we know what we are doing. All the others, even those who resembled ourselves, were cowards and hypocrites. The German Nazis and the Russian Communists came very close to us in their methods, but they never had the courage to recognize their own motives. They pretended, perhaps they even believed, that they had seized power unwillingly and for a limited time, and that just around the corner there lay a paradise where human beings would be free and equal. We are not like that. We know that no one ever seizes power with the intention of relinquishing it. Power is not a means; it is an end. One does not establish a dictatorship in order to safeguard a revolution; one makes the revolution in order to establish the dictatorship. The object of persecution is persecution. The object of torture is torture. The object of power is power. Now you begin to understand me.”
― George Orwell, 1984
numpad0 1 days ago [-]
Tangential: this seem to be an extremely minor opinion of mine, but does messaging from O'Brien to Winston not boil down to:
- "for god's sake please play by the rules, get into the power, and only then and only by doing so join us the Miniluv equals IRA; do not try to walk through the front door and topple the building by hand"
- "I thought you were smart enough to understand the intricacy of this game, spared precious resources for you, and giving you multiple chances. Your undecided and uncultured reactions to my efforts are irritating"
Or am I absolutely hallucinating? Is O'Brien as a sad, captured, ex-revolutionist government torturer all there is to it?
rightbyte 1 days ago [-]
I think quotes like this should be attributed to the character (the secret police commissaire?) not the author?
matwood 1 days ago [-]
Including 1984 in the cite makes it clear IMO.
I referenced 1984 in comment this morning related to websites disappearing. We’ve (always|never) been at was with Eurasia…
rightbyte 1 days ago [-]
Ye I get that but my main point is that it is not Orwell the real person that is quoted, but a fictional character that is a high level official in a dystopic police agency. If you haven't read the book it might be as Orwell the author is complicit.
But sry might be a nitpick and obvious from context.
matwood 1 days ago [-]
I’m old and have read 1984 many times. But to your point, it doesn’t mean everyone has. Your note provides useful clarification. Cheers!
Terr_ 1 days ago [-]
Alternately, if there were two layers of quoting (here, adding a >) then the outer layer would more-clearly refer to the book, rather than the lines of the character.
gopher_space 1 days ago [-]
It’d be a point against in a formal debate, but in casual conversation it’s safe to assume your audience passed junior high and is familiar with the work.
itronitron 1 days ago [-]
can't be "First Emperor" if it's known that someone else was in charge before you
o11c 1 days ago [-]
Eh, the history doesn't support that meme; only the details and not the existence was allegedly burned (if it even happened at all). It's pretty well established that there were 7 major states (Qin, Qi, Chu, Yan, and the 3 Jin (Han, Wei, Zhao); notably, not including the official Zhou king) at the end of the Warring States period, and the Qin conquered the other 6 within a decade.
(Fun fact: the lesser-known state of Wey actually lasted longer than Wei due to not being worth conquering. In modern Chinese they are pronounced the same, but at the time they weren't.)
zamadatix 1 days ago [-]
Sure, they just used a different title.
exe34 1 days ago [-]
First week we had mass deportation, second week we've heard of the building of concentration camps for undesirables, and now the modern version of book burning. There's something different about this republican government.
matwood 1 days ago [-]
First week they also declared banned books a ‘hoax’. So maybe getting rid of existing information takes longer than a week.
southernplaces7 1 days ago [-]
>second week we've heard of the building of concentration camps for undesirables
Concentration camps have been operated by many regimes, not just the nazis. That equation is in your head.
I am however personally pessimistic about the probity of a camp run on a remote island without any oversight. And I think insisting that everyone wait until after a dictatorship is fully embedded and engaging in retaliation against political enemies (after making open threats to do so) is strategically stupid, much like saying you need to let an asailant get in 4 or 5 punches just to be sure the first two weren't accidental.
southernplaces7 1 days ago [-]
>I think insisting that everyone wait until after a dictatorship is fully embedded and engaging in retaliation against political enemies
You realize this is a democratically elected government whose leader was already president once without a dictatorship?
Just a couple of weeks ago a federal court stopped his plans to end birthright citizenship in its tracks, and what was Trump's response? "We'll appeal it in court".... Oh damn! all those dictatoring plans stopped by the monstrous hurdle of constitutionally mandated judicial oversight.... You'd almost think he's not quite.... a dictator.
This kind of tedious hyperbole not only ran rampant before and during the first four years, it also made it harder to make genuine criticisms of Trump gain attention. With so much hysteria about every bullshit little thing being touted far and wide as an example of a 21st century Hitlerism, it's easy for the critics to be viewed as a laughing stock except in their own little echo chambers of like-thinkers.
RealityVoid 1 days ago [-]
So, what would be the red line in the sand for you where you stop and say... "Aw, shit, this guy _really_ is fascist."
Because all I've been seeing so far were excuses, he won't do that's (that promptly were then done), contorsions to justify his insane dribble and complete disregard for the law.
This guy is going to destroy your country, and it's absolutely mind-boggling people are cheering him.
exe34 20 hours ago [-]
I love how people keep saying he's disavowed project 2025 while he's in the process of implementing it.
pessimizer 23 hours ago [-]
> This guy is going to destroy your country, and it's absolutely mind-boggling people are cheering him.
He was already president for four years, and the next four years were so bad for the world and the US that he was asked to come back overwhelmingly. Or it could just be because the Republicans run democratic primaries that party leadership can't just overrule or turn into a mockery, and the Democrats don't.
> insane dribble
"Drivel." It's a common error.
exe34 1 days ago [-]
he pardoned the January 6 rioters that he sent to the capitol to stop the certification of the election. they can all walk free now including the ones who killed a police officer.
khazhoux 1 days ago [-]
> Concentration camps have been operated by many regimes, not just the nazis. That equation is in your head.
They may been operated by hundreds of regimes, but the term is 100% identified with Nazis now. It's a rather daft statement to say that these are not equated.
BeetleB 1 days ago [-]
> They may been operated by hundreds of regimes, but the term is 100% identified with Nazis now
The reason this is being discussed here is because it isn't.
Terr_ 1 days ago [-]
> It's also way the fuck off base to call this a concentration camp for undesirables with all the insinuations of Nazism that come with that.
Tell me, how many other governments can you list that planned to deport large numbers of people living in their country out to a special island they controlled in order to take advantage of its extraterritorial legal limbo separating it from the source country?
Comparing this to the Madagascar plan and the core aims of the regime behind it is even more the fuck off base than the original insinuation of the current plans for Gitmo being akin to a concentration camp for undesirables.
Truly, at least use some basic bloody comparative reasoning to formulate arguments.
notusmerican 1 days ago [-]
[dead]
Dalewyn 1 days ago [-]
[dead]
Terr_ 1 days ago [-]
> they broke the law crossing the border illegally
The majority here crossed legally and overstayed their visa. That's just basic table-stakes knowledge for this issue, please remember it. Bonus points if you know that overstaying is a civil infraction like a parking ticket, below the level of a misdemeanor.
> these initial rounds of deportations are prioritizing illegal aliens with violent criminal records.
That's what the professional liar says, but why would you trust him on that?
> These people are factually the worst of the worst.
How the hell can you "factually" know that when the Trump administration still won't even explain what criteria they are using!? (Even assuming they'll be making any honest effort at all.)
forgetfreeman 1 days ago [-]
If you find yourself in a situation where you feel compelled to quibble over fine details of the definition of terms like " concentration camp", this is an indication that you are desperately compromised in some way.
timeon 20 hours ago [-]
What would you expect when people are even arguing against their eyes about Fascist salute.
exe34 1 days ago [-]
the parallel with the Nazis for me is the building of it on foreign soil instead of somewhere with judicial oversight - even while they own the supreme court and Congress.
that and the Nazi salute at the inauguration.
misnome 1 days ago [-]
Did the Nazi program start at those levels? Or did it start with exactly this rhetoric and excuses?
Terr_ 1 days ago [-]
That was probably a rhetorical question, but just-in-case, the Nazis absolutely started with deportations many years before the gas chambers.
They even plotted to forcibly deport victims to fenced-in zones on an island which they would seize from another country, putting it outside their laws at home.
The Nazi program of concentration camps and terror started almost immediately, and included an enormous amount of overt, obvious, very violent state repression against anyone at all who criticized the regime. This included sending them to the first camps, formed in 1933, right after Hitler's appointment, and openly torturing and beating people, or executing them extrajudicially.
None of anything from Trump's government is at all comparable so far and making such comparisons, aside from being a genuinely idiotic case of ideological exaggeration, is also an insult to real victims of real historical concentration camps and violent regimes.
For all the self-congratulatory patting on the back by many readers on HN about their "superior" intellectual reasoning, I see a remarkable abundance of largely failing at such reasoning when it comes to many comment threads and subjects on this site.
misnome 17 hours ago [-]
Wikipedia says it was two months after Hitler’s appointment before the first camps were established.
It’s been less than two weeks and you are already doing the groundwork to preemptively excuse it.
Spivak 1 days ago [-]
I think you've associated the entirety of concentration camps with the one that's taught in WWII history. There are extermination camps where the goal is efficient mass-murder, forced labor camps where the goal is slavery, both of which are a kind of concentration camps which is simply a place where groups of people are illegally detained. Don't focus too hard on the illegal part because it's always legal in the state that does it. The Japanese internment camps the US used were concentration camps.
In a weird way the events of WWII saved the US from going down the same path because we were on it. Anything literally Hitler did became politically toxic and it's a bit worrying that this toxicity is wearing off. It kept us far away from
the fence.
The thing that matters for concentration camps is large groups of related civilians detained to advance a political goal— to restore Germany to its former greatness or to make America great again.
1 days ago [-]
1 days ago [-]
exe34 1 days ago [-]
as posted by two others - hotel Gitmo on sands is expanding.
numpad0 1 days ago [-]
There's also the OPM email thing.
(disclaimer: not an American)
tdeck 1 days ago [-]
Didn't you get the memo? Trump won 49.9% of the popular vote. That means we all need to pretend this is normal and OK now.
coolspot 1 days ago [-]
[flagged]
tdeck 1 days ago [-]
You're still upset about the statues?
pjc50 1 days ago [-]
People are weirdly defensive of the traitor statues.
terrabiped 1 days ago [-]
In the Bay Area we had the statues of Ulysses Grant [1], Francis Key [2], and Junipero Serra [3] removed. This isn't even the full list of all the monuments that were labeled "racist" or "oppressive". All was done a couple of years ago.
Are these the traitors you're talking about?
Even if people couldn't dig up anything against the person, the monument could simply be linked to the "wrong part of our history" and be removed, like the Thomas Fallon's statue in San Jose [4]
> On June 19, 2020, the monument was toppled by protestors and defaced with the words "Adios America" in red paint as a response to Grant's brief ownership of a slave.
> Francis Key
> The statue was toppled by vandals on June 19, 2020, in the wake of the murder of George Floyd.
> Juniperro Serra
> Other statues of Junípero Serra were involved as the protests expanded to include monuments of individuals associated with the controversy over the genocide of indigenous peoples in the Americas.
> Thomas Fallon
> Fallon remains a controversial figure in San Jose's history, owing to his role in the American Conquest of California.
Besides Francis Key, which appears to have been vandalized rather than something that people specifically wanted removed, the rest of them do actually have reasons they were taken down? You can disagree with those but they do exist.
terrabiped 8 hours ago [-]
My response was a counterexample to the the parent comment which implied that the only statues being removed are the statues of the traitors. None of the people I listed were traitors.
For the people I listed, I'd encourage you to read more about them and form your own opinion instead of just quoting back from the links I shared earlier.
There is very little reason to believe that Serra treated the indigenous people differently from the rest of his mission or that he didn't believe in what he was preaching. Similarly, almost nothing is known about why Grant had William Jones for a year, but Grant's character had been proven through years of real action - fighting the war and, later, fighting the KKK.
People just clung to some speculative theories, interpreted them in the least charitable way, and get offended by it. While everyone else sees that the topic was labeled as "racist" or "controversial" and tries to distance themselves from it.
saagarjha 3 hours ago [-]
I don't think you want to hear my opinions of those people, though I will agree that they are probably not traitors.
Dalewyn 1 days ago [-]
[flagged]
krapp 1 days ago [-]
They were traitors.
They waged war against the United States, attacking the seams of this very country's unity, which by even the Constitution's purposely narrow definition, is treason.
They were pardoned for their treason because they were traitors.
Dalewyn 1 days ago [-]
Laws and Constitution amendments were written and passed to exclude ex-Confederates from the democratic process, but they were rendered moot once everyone calmed down and realized it was better for the country to reunite as one nation.
These false narratives villifying and dividing the country long after the Civil War concluded are nonsense.
notusmerican 1 days ago [-]
[dead]
pjc50 1 days ago [-]
[flagged]
Dalewyn 1 days ago [-]
>In 1865, Robert E. Lee wrote to a former Confederate soldier concerning his signing the Oath of Allegiance, and I quote: "This war, being at an end, the Southern States having laid down their arms, and the questions at issue between them and the Northern States having been decided, I believe it to be the duty of everyone to unite in the restoration of the country and the reestablishment of peace and harmony."
-President Gerald Ford
>Whereas the authority of the Federal Government having been reestablished in all the States and Territories within the jurisdiction of the United States, it is believed that such prudential reservations and exceptions as at the dates of said several proclamations were deemed necessary and proper may now be wisely and justly relinquished, and that an universal amnesty and pardon for participation in said rebellion extended to all who have borne any part therein will tend to secure permanent peace, order, and prosperity throughout the land, and to renew and fully restore confidence and fraternal feeling among the whole people, and their respect for and attachment to the National Government, designed by its patriotic founders for the general good:
-President Andrew Johnson
The Civil War is over, it ended over a century and a half ago. I again, sincerely, ask you to screw off with that false narrative which serves only to divide and weaken the unity and solidarity of the country.
saagarjha 21 hours ago [-]
Can I ask you to do the same, or is that just something you pull out when you're upset about what other people say and not something you expect to have to do yourself?
Dalewyn 18 hours ago [-]
It is decidedly not factual to call ex-Confederates traitors when they were pardoned, or do you not understand what a pardon is?[1]
Ex-Confederates were traitors, that much is true. Were they never pardoned or otherwise acquitted of treason in a court of law then that would still be true today. Of course, that is not what happened: President Andrew Johnson pardoned them all unconditionally, and in the case of General Robert E. Lee (whose pardon was not processed due to clerical errors) his pardon was made effective retroactive to the date of application by an act of Congress.
Ex-Confederates are all therefore excused of their crime of treason, and to still attack them further citing that crime is baseless and unwarranted. They are Americans, and the men who fought are all American heroes who ultimately wished for a united and healed nation.
Calling Civil War era Americans traitors today is ruthlessly divisive rhetoric that only serves to sow conflict in a time when the country really does not need that kind of incendiary bullshit. Everyone who spews this false narrative should be ashamed of themselves, they are throwing mud upon those who chose to pursue peace instead of generational animosity.
Pardoning someone for a crime they did does not mean they didn't do the crime.
ramabananasumer 1 days ago [-]
[dead]
exe34 1 days ago [-]
I never understood the removal of statues. I think everybody is capable of good and bad - we should still celebrate the good stuff they did and simply add a note that says "they were humans of their time and they screwed up by doing..... - let's not forget that even the great can screw up."
otherwise we keep perpetuating the myth that the great/rich/powerful can do no wrong.
ocschwar 1 days ago [-]
Most of the statues we're talking about were installed during the Civil Rights era as an explicit flex meant to intimidate African Americans.
exe34 22 hours ago [-]
here in the UK we had the same burn the -witch+statue- panic and ours were imperial leaders of some sort - but the people they oppressed were oceans away.
rhinoceraptor 1 days ago [-]
I heard there was this party that had the same weird hand gesture that President Musk did a week or two ago...
honestSysAdmin 1 days ago [-]
Let's make torrents and seed them.
bamboozled 12 hours ago [-]
They might come after you for hosting it ?
bbwbsb 11 hours ago [-]
i2p could be an option
tmshapland 1 days ago [-]
[flagged]
refurb 21 hours ago [-]
Do you feel the same when Biden did it?
“ in the days after Joe Biden was inaugurated, data.gov showed about 1,000 datasets being deleted as compared to a day before his inauguration, according to the Wayback Machine.”
p3rls 1 days ago [-]
really looking forward to four more years of astute political commentary like this
gopher_space 1 days ago [-]
For all of the downsides to a Trump administration, one of the unexpected upsides is that the people who don't have a problem with Nazis will self-identify.
I feel really bad for the kids who've developed an alt-right persona during the pandemic, though. There used to be time to work that out for yourself, but now I think social media will provide the kiss of death for many career ambitions.
pessimizer 23 hours ago [-]
They can't keep it up for another decade, the financing is running out. Anti-Trump hysteria was largely financed by actors who dominated establishment Republican and Democratic corruption networks, but had no connection to Trump (who's a real estate guy, which is like the ghetto of finance.) During that decade, and especially during the four years he was out of office, a most of them have solidly established relationships with Trump and his network, especially tech guys.
One day, upper-middle class people will stop seeing as brave and insightful calling half the country Nazis for not conforming to their suburban aesthetic. They've even eventually stopped progressively describing him as the submissive homosexual partner to Putin, I suppose because we've run out of fighting aged adult males in Ukraine so there's no reason to care about that any more. If Trump starts a war with Iran, they'll start praising him as the new Lincoln. It's not real.
The only weird part is that the people attacking Trump for censorship that hasn't happened are delighted by censorship. They think the main problem with the world is that too many people are allowed to speak about whatever they want.
bagels 19 hours ago [-]
It's merely a coincidence that Trump has said that immigrants are "poisoning the blood" of our country, has invited Nazis and Holocaust deniers to dinner at his home, is rounding up brown people, had Musk deliver Nazi salutes at his inauguration, felt the need to say that Hitler "did some good things", was said to keep Hitler's speeches by his bedside by his wife, ran a campaign that promoted a video referencing a "unified Reich", and more? Just a long series of misunderstandings, I'm sure.
mpalmer 22 hours ago [-]
> calling half the country Nazis for not conforming to their suburban aesthetic
> attacking Trump for censorship that hasn't happened
Federal funds are being withheld - illegally, in most cases - from any entity that supports or even acknowledges what the administration calls "radical gender ideology".
It's totalitarian and regressive. Nazi's a fine word for what the leaders of the party are showing themselves to be. I wouldn't call half the country Nazis - unless they chose to stand with the human trash in the White House. Then I wouldn't really have a problem with it.
palmfacehn 19 hours ago [-]
Can anyone making this comparison cite sources where the collectivist, totalitarian National Socialists reduced the size of the state? It seems like a contradiction in terms. Since when did 19th century liberalism and the founding principles of individual liberty become equated with collectivist totalitarianism?
mpalmer 16 hours ago [-]
No one needs to do that. No one is obliged to you to find the exact policy parallels between Republicans in the White House and the Nazis.
It tends to be enough for most people to notice the fact that both are interested in removing undesirables from public life and generally making society less racially mixed.
palmfacehn 16 hours ago [-]
>No one needs to do that. No one is obliged to you to find the exact policy parallels
Then the claims are unsubstantiated. Political speech is some of the most twisted and manipulated. Unsubstantiated political claims are basically worthless. I'm willing to trust, if you are willing to verify. Engage in the discussion if you are serious about your claims.
"Everyone who disagrees with me is a National Socialist", is a tired line I've been reading on the Interwebs for a long time now. If you are unwilling to substantiate it, it is an odious comparison that trivializes one of the most murderous ideologies of the last century, second only to communism. These ideas demand serious treatment.
If you genuinely care about such things, you would take them more seriously. You might take the time to understand the genesis of these collectivist ideologies. After that, you might choose to champion ideals which stand in contrast. Individualism and individual self-ownership as the basis of private property would be one approach. Those who reason from these first principles are generally in favor of reducing the size of the state.
mpalmer 15 hours ago [-]
Sorry, no. I don't have time for dictionary-waving pedantry like this.
Political speech is perfectly understandable as such. It's the opposite of worthless, if you are trying to make a political point, as I was. I was very clear about the important (not total) parallels between then and now. I trivialize nothing.
Am I going to find you defending mainstream Democrats from accusations of socialism, or is your impassioned defense of the Republican party as it existed up until 10 years ago all you're offering?
palmfacehn 15 hours ago [-]
Words have meaning. We should be willing to agree upon that much. Worry less about who you perceive me to be on the partisan spectrum and try to focus on the words I have written.
I don't see a defense of the Republican Party here. Overall I'd rate them as a big disappointment. Although Trump coming out strong and cutting is a reason to be cautiously optimistic. It certainly isn't a reason to invoke Godwin's Law. As I've explained repeatedly here, cutting government is basically the opposite of the National Socialist economic program. If you are not informed enough to dispute this, then now would be a good time to do some research for yourself.
mpalmer 13 hours ago [-]
I'm well aware of the words you've written. Are you waiting for me to say you're right, or that you make a good point? Do you think people think of the Nazis' economic programs before anything else?
You dodge the question that puts the lie to your high-minded concern about using words according to their precise definition, which is unreasonable and serves to inhibit discourse, as you must realize.
You need only actually read one of his executive orders - again, ILLEGALLY directing appropriated funds not be spent - to see that cutting government is not the only thing these ideologues are attempting to do.
You might, finally, look into what Godwin himself has said regarding his "law" and Trump.
palmfacehn 41 minutes ago [-]
No, I was hoping you might substantiate your claims. Cutting spending stands in direct contrast to totalitarian ideology. If you cannot approach that we shall move on.
>removing undesirables from public life and generally making society less racially mixed.
These are more unsubstantiated claims.
immibis 10 hours ago [-]
When Hitler cancelled all gender studies research and burned their books, he reduced the size of the state, which had been funding that research.
palmfacehn 19 minutes ago [-]
Thank you for providing a tangible claim we may examine together.
>The institute was financed by the Magnus-Hirschfeld-Foundation, a charity which itself was funded by private donations.
Again, the National Socialists increased the purview of the state by interfering with a privately run and privately funded non-profit. They used public funds to attack and regulate private, voluntary behavior.
strictnein 1 days ago [-]
99.9% of commenters here seem to have missed this:
> For example, in the days after Joe Biden was inaugurated, data.gov showed about 1,000 datasets being deleted as compared to a day before his inauguration
It's almost like this stuff happens regularly. If <insert Dem savior> wins in 2028, tons of government websites will also change in the first couple weeks of their presidency. Is it because they're a fascist dictator? Or is it because those websites reflect the administration's viewpoints on issues?
Wish people would take a deep breath and step back and think a little more. I despise Trump, but there's crying wolf and then there's the current state of media and online discourse. Trump thrives in this type of environment. He purposefully fosters it. Playing gotcha with him doesn't work because he doesn't care.
BeetleB 1 days ago [-]
Plenty of comments older than yours are pointing this out. A bit hyperbolic to claim 99.9%.
blindriver 12 hours ago [-]
The depth to which the media can whip people up into a hysteria is truly striking. I think most people that voted for Trump like myself aren’t buying into the propaganda and will rather wait for the dust to settle.
All I’m seeing is a president that is doing exactly what he promised, so I’m very much in the “let the chips fall where they may” category.
Rendered at 05:20:06 GMT+0000 (Coordinated Universal Time) with Vercel.
There's lots of people making copies of things right now, which is great -- Lots Of Copies Keeps Stuff Safe. It's your data, why not have a copy?
One thing I think we can contribute here as an institution is timestamping and provenance. Our copy of data.gov is made with https://github.com/harvard-lil/bag-nabit , which extends BagIt format to sign archives with email/domain/document certificates. That way (once we have a public endpoint) you can make your own copy with rclone, pass it around, but still verify it hasn't been modified since we made it.
Some open questions we'd love help on --
* One is that it's hard to tell what's disappearing and what's just moving. If you do a raw comparison of snapshots, there's things like 2011-glass-buttes-exploration-and-drilling-535cf being replaced by 2011-glass-buttes-exploration-and-drilling-236cf, but it's still exactly the same data; it's a rename rather than a delete and add. We need some data munging to work out what's actually changing.
* Another is how to find the most valuable things to preserve that aren't directly linked from the catalog. If a data.gov entry links to a csv, we have it. If it links to an html landing page, we have the landing page. It would be great to do some analysis to figure out the most valuable stuff behind the landing pages.
https://en.wikipedia.org/wiki/Jaccard_index
I do a bit of web archival for fun, and have been thinking about something.
Currently I save both response body and response headers and request headers for the data I save from the net.
But I was thinking that maybe if instead of just saving that, I could go a level deeper and preserve actual TCP packets and TLS key exchange stuff.
And then, I might be able to get a lot of data provenance “for free”. Because if in some decades when we look back at the saved TCP packets and TLS stuff, we would see that these packets were signed with a certificate chain that matches what that website was serving at the time. Assuming of course that they haven’t accidentally leaked their private keys in the meantime and that the CA hasn’t gone rogue since etc.
To me I think that would make sense to build out web archival infra that preserves the CA chain and enough to be able to see later that it was valid. And if many people across the world save the right parts we don’t have to trust each other in order to verify that data that the other saved was also really sent by the website our archives say it was from.
For example maybe I only archived a single page from some domain, and you saved a whole bunch of other pages from that domain around the same time so the same certificate chain was used in the responses to both of us. Then I can know that the data you are saying you archived from them really was served by their server because I have the certificate chain I saved to verify that.
But the main content is not signed / checksummed with it, but rather a symmetrical session key, so one could probably manipulate this in the packet dump anyway.
I read about a Google project named SXG (Signed HTTP exchanges) that might do related stuff, albeit likely requiring the assistance of the publisher
But then you need the client to be trusted, which clashes with distributing.
Hypothetically, what about trusted orgs standing up an endpoint that you could feed a URL, then receive back attestation from them as to the content, then include that in your own archive?
Compute and network traffic are pretty cheap, no?
So if it's just grabbing the same content you are, signing it, then throwing away all the data and returning you the signed hash, that seems pretty scalable?
Then anyone could append that to their archive as a certificate of authenticity.
Not quite the same problem, but similar enough to have a similar solution. https://www.ietf.org/rfc/rfc3161.txt
But they didn't really go anywhere:
https://security.stackexchange.com/questions/52135/tls-with-...
https://security.stackexchange.com/questions/103645/does-ssl...
There are some special cases, like I think certain headers for signing e-mails, that do provide non-repudiation.
For that, `tcpdump` with `SSLKEYLOGFILE` will probably get you started on capturing what you need.
Overall though I think archive.org is probably sufficient proof that a specific page had certain content on a certain day for most purposes today.
0. https://github.com/harvard-lil/scoop
It works by using public key cryptography and key agreement to get both parties to agree on a symmetric key, and then uses the symmetric key to encrypt the actual session data.
Any party who knows the symmetric key can forge arbitrary data, and so a transcript of a TLS session, coupled with the symmetric key, is not proof of provenance.
There are interactive protocols that use multi-party computation (see for example https://tlsnotary.org/) where there are two parties on the client side, plus an unmodified server. tlsnotary only works for TLS1.2. One party controls and can see the content, but neither party has direct access to the symmetric key. At the end, the second party can, by virtue of interactively being part of the protocol, provably know a hash of the transaction. If the second party is a trusted third party, they could sign a certificate.
However, there is not a non-interactive version of the same protocol - you either need to have been in the loop when the data was archived, or trust someone who was.
The trusted third party can be a program running in a trusted execution environment (but note pretty much all current TEEs have known fault injection flaws), or in a cloud provider that offers vTPM attestation and a certificate for the state (e.g. Google signs a certificate saying an endorsement key is authentically from Google, and the vTPM signs a certificate saying a particular key is restricted to the vTPM and only available when the compute instance is running particular known binary code, and that key is used to sign a certificate attesting to a TLS transcript).
I'm working on a simpler solution that doesn't use multiparty computation, and provides cloud attestation - https://lemmy.amxl.com/c/project_uniquonym https://github.com/uniquonym/tls-attestproxy - but it's not usable yet.
Another solution is if the server will cooperate with a TLS extension. TLS-N (https://eprint.iacr.org/2017/578.pdf) provides a solution for this. That provides a trivial solution for provenance.
People are still going to be suspicious of each other, and service providers are still going to leak their private keys, and whatnot.
https://github.com/reclaimprotocol
https://drive.google.com/file/d/1wmfdtIGPaN9uJBI1DHqN903tP9c...
https://www.reclaimprotocol.org/
https://docs.lighthouse.storage/lighthouse-1/zktls
- bigger resource usage. You will need to maintain a dump of the TLS session AND an easily extractable version
- difficulty of verification. OpenSSL / BoringSSL / etc. will all evolve and say, completely remove support for TLS versions, ciphers, TLS extensions… This might make many dumps unreadable in the future, or requiring the exact same version of a given software to read it. Perhaps adding the decoding binary to the dump would help, but then, you’d get Linux retro-compatibility issues.
- compression issues: new compression algorithms will be discovered and could reduce data usage. You’ll have a hard time doing that since TLS streams will look random to the compression software.
I don’t know. I feel like it’s a bit overkill — what are the incentives for tampering with this kind of data?
Maybe a simpler way of going about it would be to build a separate system that does the « certification » after the data is dumped; combined with multiple orgs actually dumping the data (reproducibility), this should be enough the prove that a dataset is really what it claims to be.
https://www.imperialviolet.org/2016/09/19/roughtime.html
https://int08h.com/post/to-catch-a-lying-timeserver/
https://blog.cloudflare.com/roughtime/
https://news.ycombinator.com/item?id=12599705
This proves you generated it before this time.
You can also include in the hash the close of the stock market and all the sports scores from the previous day. That proves you generated it after that time.
Sports sports are an interesting source of entropy.
You are relying upon the authority of the beacon to be random, but good practice is to utilize multiple independent beacons.
[0] https://csrc.nist.gov/projects/interoperable-randomness-beac...
Not so useful if something was edited a few minutes after posting, but it makes it more difficult for a new administration to suddenly edit a bunch of old data.
But you could do the same thing with any hashes, right? There is no need for a blockchain in the middle.
There’s also a size advantage. You can keep a diff on the archive for each hash being posted instead of the full index for every time you post a hash.
The chance is exceedingly low that the PKI infrastructure of all the authorities becomes compromised.
Given the rapid take downs of websites (cdc, usaid) do you have a prioritization framework for which website pages to prioritize or do you have "comprehensive" coverage of pages (in scope of the project)?
As you allude to, I've been having a hard time learn about what sort of duplicate work might be happening given that there isn't a great "archived coverage" source of truth for government websites (between projects such as End of Term archive, Internet archive, research labs, and independent archivists).
Your open questions are interesting. Content hashes for each page/resource would be a way to do quick comparisons, but I assume you might want to set some threshold to determine how much it's changed vs if it changed?
Is the second question about figuring out how to prioritize valuable stuff behind two depth traversals? (ex data.gov links to another website and that website has a csv download)
In practice it's some mix of, there aren't already lots of copies, it's valuable to people, and it's achievable to preserve.
> Is the second question about figuring out how to prioritize valuable stuff behind two depth traversals?
Right -- how do you look at the 300,000 entries and figure out what's not at depth one, is archivable, and is worth preserving? If we started with everything it would be petabytes of raw datasets that probably shouldn't be at the top of the list.
Unfortunately it's a bit messy because we weren't initially thinking about tracking deletions. data_20241119.jsonl.zip (301k rows) and data_20250130.jsonl.zip (305k rows) are simple captures of the API on those dates. data_db_dump_20250130.jsonl.zip (311k rows) is a sqlite dump of all the entries we saw at some point between those dates. My hunch is there's something like 4,000 false positives and 2,000 deletions between the 311k and 305k set, but that could be way off.
There's coordinated efforts starting to come together in a bunch of places -- some on r/datahoarders, some around specific topics like climate data (EDGI) or CDC data, there's datasets being posted on archive.org. I think one way is to find a topic or kind of data that seems important and search around for who's already doing it. Eventually maybe there'll be one answer to rule them all, but maybe not; it's just so big.
I think in some ways the community was less prepared this time, because there was a lot of investment in 2016-2017 and then many of the archives created at that point didn't end up being used; partly because the changes at the federal level turned out to be smaller and slower in 2017 than they're looking like this time. So some people didn't choose to invest that way this time around.
[Edit: this means I think it's really important that data archives are useful. Sorting through data and putting a good interface on it should help people out today as well as being good prep for the future.]
In other ways there's much more preparation; EOT Archive now has a regular practice of crawling .gov websites before and after each change of administration, which is a really great way of giving citizens a sense of how their government evolves. It will just tend to miss data that you can't click to in a generic crawl.
https://www.reddit.com/r/PrepperIntel/s/scOu1QuhNt
"Project Russia" is spiritual warfare
https://washingtonspectator.org/project-russia-reveals-putin...
Billionaire ransacking the Treasury https://techcrunch.com/2025/02/01/senator-warns-of-national-...
Bernie's statement about this
https://m.youtube.com/watch?v=mL0crkf5Dzw
> https://washingtonspectator.org/project-russia-reveals-putin...
What in the Cold War conspiracy theory was that...
>> The Kremlin’s design necessarily depends on the adoption of a single world belief system or religion. Expect a syncretic, gnostic blend, rooted in hierarchy — the Russian Orthodox Church at the core, and other religious factions accorded favor based on demonstrated fealty.
Good luck with that. The most popular religions today are forked versions of one guys story that they couldn't agree on and have been involved in acts of genocide towards one another because of it--for centuries.
Because they spend a lot of time censoring and covering things up.
RIP USA.
I guess great economies also give it away. RIP USA.
What a statement! How did you come to this conclusion?
It’s not just America either. Europe too. Australia.
Authoritarians copy and steal. Democracies innovate and adapt.
Democracies are more transparent and honest, which leads to better outcomes in all areas.
Time spent hiding information to appease the dictator and protecting “the party” is time wasted.
The Soviet Union? Yeah, the new offshore stuff (mostly? developed after the fall of the USSR) relied on western firms, but I believe the bulk of oil production still comes from / goes through domestically sourced systems. Gas production on the other hand was initially almost entirely built out by western firms, though that's obviously begun to change since the war in Ukraine.
Some below of contractors that work with US AID:
- https://www.edu-links.org/ (taken down)
- https://www.genderlinks.org/ (taken down)
- https://usaidlearninglab.org/ (taken down)
- https://agrilinks.org/ (presumably at risk)
- https://www.climatelinks.org/ (presumably at risk)
- https://biodiversitylinks.org/ (presumably at risk)
A rough paraphrasing from Boondocks, said by the richest man in that neighborhood.
The base contributions are a function of GDP. The extra contributions are voluntary, and the US did it because it was in the US’ interests. It’s a founding error in the US foreign policy budget and was a good investment in terms of goodwill and data for American health research institutions.
WHO must focus where it is needed most. Public health is much better in the EU (and even in Europe, accounting for places like Belarus and Ukraine) than in China, and there are much fewer epidemics that emerge in Europe in general.
The whole idea is that if we limit the emergence of epidemics where they are likely to happen, we end up with fewer pandemics after these epidemics spread worldwide (which includes Europe and North America). The whole world is better without another COVID, Ebola, or Polio.
> only to have the WHO be controlled by China
This is bullshit. The WHO is not controlled by China any more than other UN institutions. What is certain, though, is that the US won’t have any say whatsoever once they are out.
I realize I’m arguing against a negative but has that actually been accomplished? I don’t argue that they (I assume) probably help with things like Ebola outbreaks but that’s almost certainly never going to become a pandemic.
I did however just write a Python script to pull data.gov from archive.org and check the dataset count on the front page for all of 2024, here are the results:
https://postimg.cc/1V0WYtRt
As you can see, there were multiple drops on the order of ~10,000 during 2024. So it's not that unusual. There could be something bad going on right now, but just from the numbers I can't conclude that yet.
(Specifically it takes the first snapshot of every Wednesday of 2024).
If I get around to re-formatting my archives this week, I'll follow up on HN :).
Was there an EO targeting these areas?
https://www.cbsnews.com/news/epa-employees-warned-of-immedia...
Congress rarely makes spending money it's goal, rather it appropriates money to accomplish some goal. Which is to say that if Congress wants a bridge across a river and appropriate 10 billion to build it, the President is not obligated to spend $10 billion if 7 or 8 or 9 will do. In some cases, Congress does appropriate money toward causes and intends all the money to be spent in furtherance of some guiding principle and in these cases all the money must be spent.
Justice delayed is justice denied.
But I suppose 4 years with Biden/Kamala made it easier to forgive him, for many. Not just the MAGA base, but even swing voters.
Honestly, I think Democrats were contributing to this by the outrage over the January 6 riots. Half the country would consider the BLM riots as equally bad.
If instead, Democrats had focused on Trump's betrayal of Pence and general disregard for the institutions and traditions of the country, a lot more moderates would remain with the Democrats.
But as a foreigner, it seems to me that the prosecution of the Jan 6 rioters, not to mention Trump himself was excessive and overtly politically motivated. And it definitely took attention away from the less spectacular, but far more obviously immoral behavior he definitely, provably WAS guilty of leading up to January 6.
(You hear about it on the Internet as a way to Fight The Man on eg arrests stemming from protests or minor drug charges, but historically it was more often used to absolve white supremacists of murder.)
It seems that most legal systems from the start was intended to codify what was considered "just and fair" in the eyes of "the people".
Juries seem to have been put in place specifically to ensure that the legal system operates within this mandate, and to preven overreach or abuse.
Jury nullification is a weird rule that goes against the constitutional framework, but is so rarely used, at least for any important matter, that it has never really received too much scrutiny.
But if you you think there is no risk that the justice system can be misused for such political ends, then I suppose you have the same problems with Biden pardoning Hunter and half his family to protect them from politically motivated prosecution?
Or is it only wrong when Republicans do it?
At minimum I would say that the general public in a general election should have the same power to effetively pardon someone as the President has. After all, the electorate is the basis from which the President draws his power and legitimacy.
And not only for the President, by the way. The legitimacy of the entire system, including both Congress and the Legal System draws its legitimacy from the same source (even if the Constitution is designed to provide protection against short term simple majorities).
If the outcome of the vote cannot be accepted, then that basically means the country cannot remain a democracy.
Personally, I don't think I could have voted for Trump after how he behaved in January 2021 (if I were a citizen). Primarily because of how he betrayed Pence.
But Biden and Kamala sure made it a lot easier for him in 2024 than it would have been if the country had been led by someone like Obama.
Well, Trump made his intentions of prosecuting Fauci, Hunter Biden and many many others more than abundantly clear. He can't whine about Biden protecting people from the threat he himself had announced. (Well, he obviously can and does, and half the population falls for it)
NOFX - "The Idiots Are Taking Over"
Trump's case was a completely constitutional overreach. Thankfully they shot it down fast in court.
Of course, he can't get away with too much alone, but with the right appointees, judges, etc, who knows.
https://newrepublic.com/post/190983/top-treasury-official-qu...
EDIT: apparently Musk succeeded: https://bsky.app/profile/wyden.senate.gov/post/3lh5ejpwncc23
Train v. City of New York isn't a constitutional case. So if Congress controls spending, it's a power they gave to themselves. Any other portion of government could likewise determine they control spending. So maybe it goes back to the SCOTUS because they agree to hear another case about it. Maybe they like the idea that the executive branch controls spending? Or maybe they don't after all. The tell the President to stop doing that. But the president does not. Does Congress pass another law reaffirming they control the power for spending?
If the executive branch is in charge of running the government and they listen to the President & the President decide's they aren't spending any money, who exactly is going to change that? Would Congress fund another department to change this? Would it also just report to the President?
From what I understand it's actually pretty common for the executive branch not to actually be able to spend appropriated money. The simplest case would be if you put out a contract and get no bids on it. There are other cases, like Congress agreeing to fund military submarines for Taiwan. But no one is going to build them internationally and we don't build diesel-electric combat submarines in the US as a matter of policy.
I'm not a legal scholar, but I'm going to guess that saying "congress appropriates money, but the President can decide if they actually get it" is a pretty fringe idea.
I don't even know what more to say. Paradox of tolerance so I'm no even going to pretend to empathize with ideas like"deport the illegals in chains " and ćmy body your choice".
https://www.reddit.com/r/fednews/comments/1id7ud2/comment/m9...
Please see Elgin v. Dept. of Treasury 567 U.S. 1 (2012) for details.
https://www.oyez.org/cases/2011/11-45
https://tile.loc.gov/storage-services/service/ll/usrep/usrep...
https://en.wikipedia.org/wiki/Department_of_Homeland_Securit...
I think it is likely that orders to these other agencies follows this model. Many other datasets are being targeted via EO 14168 which has quite wide impacts but doesn't appear at first glance to apply to what i would expect to be a part of NOAA and EPA reports.
https://www.politico.com/news/2025/01/31/usda-climate-change...
Maybe this administration will get better over time?
CDC data are disappearing - https://news.ycombinator.com/item?id=42897696 - Feb 2025 (216 comments)
Say you have a few TBs of disk space, and you're willing to capture some public datasets (or parts of them) that interest you, and publish them in a friendly jurisdiction - keyed by their MD5/SHA1 - or make them available upon request. I.e. be part of a large open-source storage network, but only for objects/datasets you're willing to store (so there are no illegal shenanigans).
Is this a use case for Torrents? What's the most suitable architecture available today for this?
Set up a scrape using ArchiveTeam's fork of wget. It can save all the requests and responses into a single WARC file. Then you can use https://replayweb.page/ or some other tool to browse the contents.
.
> Is this a use case for Torrents?
Yes, provided you have a good way to dynamically append a distributed index of torrents and users willing to run that software in addition to the torrent software. Should be easy enough to define in container-compose.
Not to say the data isn't incredibly valuable and should be preserved for many other reasons of course. All the best to anyone archiving this, this is important work.
> Changes in presidential administrations have led to datasets being deleted in the past, either on purpose or by accident. When Biden took office, 1,000 datasets were deleted according to the Wayback Machine, via 404 Media's reporting.
Time will tell but loss of public datasets is probably not usually good in general.
> But archivists who have been working on analyzing the deletions and archiving the data it held say that while some of the deletions are surely malicious information scrubbing, some are likely routine artifacts of an administration change, and they are working to determine which is which. For example, in the days after Joe Biden was inaugurated, data.gov showed about 1,000 datasets being deleted as compared to a day before his inauguration, according to the Wayback Machine.
The government information crisis is bigger than you think it is - https://news.ycombinator.com/item?id=42895331
https://tracker.archiveteam.org/
Edit to add:
docker_compose.yml example:
services:
How much of this sort of effort results in that data being used? Are there success stories for these datasets being discoverable enough and useful to others?
As one of the reddit comments (in the thread linked by the article) pointed out,
> During the start of Biden’s term, On 6th feb data.gov had “218,384 DATASETS” but on 7th feb it only had “192,180 DATASETS”
Here's documentation on independent backup efforts of various government websites: https://www.reddit.com/r/DataHoarder/comments/1ifalwe/us_gov...
Also here: https://www.reddit.com/r/DataHoarder/comments/1idj6dm/all_us...
Apparently, much of the data has been back up here: https://eotarchive.org/
Here's also a discussion on whether the Internet Archive is sufficiently backed up/decentralized (it is not): https://www.reddit.com/r/DataHoarder/comments/1if32iq/does_i...
- Borges
https://en.wikipedia.org/wiki/Shi%20Huangdi (redirects to Qin Shi Huang)
Qin Shi Huang/Shih Huang Ti both means [Qin]Founding Emperor[-King]. In layperson's approximation, He's so ancient that his canonical name and title is debatable, and he's almost always referred to as "the first emperor" or variants thereof.
― George Orwell, 1984
I referenced 1984 in comment this morning related to websites disappearing. We’ve (always|never) been at was with Eurasia…
But sry might be a nitpick and obvious from context.
(Fun fact: the lesser-known state of Wey actually lasted longer than Wei due to not being worth conquering. In modern Chinese they are pronounced the same, but at the time they weren't.)
Mind posting a reference for such a declaration?
I am however personally pessimistic about the probity of a camp run on a remote island without any oversight. And I think insisting that everyone wait until after a dictatorship is fully embedded and engaging in retaliation against political enemies (after making open threats to do so) is strategically stupid, much like saying you need to let an asailant get in 4 or 5 punches just to be sure the first two weren't accidental.
You realize this is a democratically elected government whose leader was already president once without a dictatorship?
Just a couple of weeks ago a federal court stopped his plans to end birthright citizenship in its tracks, and what was Trump's response? "We'll appeal it in court".... Oh damn! all those dictatoring plans stopped by the monstrous hurdle of constitutionally mandated judicial oversight.... You'd almost think he's not quite.... a dictator.
This kind of tedious hyperbole not only ran rampant before and during the first four years, it also made it harder to make genuine criticisms of Trump gain attention. With so much hysteria about every bullshit little thing being touted far and wide as an example of a 21st century Hitlerism, it's easy for the critics to be viewed as a laughing stock except in their own little echo chambers of like-thinkers.
Because all I've been seeing so far were excuses, he won't do that's (that promptly were then done), contorsions to justify his insane dribble and complete disregard for the law.
This guy is going to destroy your country, and it's absolutely mind-boggling people are cheering him.
He was already president for four years, and the next four years were so bad for the world and the US that he was asked to come back overwhelmingly. Or it could just be because the Republicans run democratic primaries that party leadership can't just overrule or turn into a mockery, and the Democrats don't.
> insane dribble
"Drivel." It's a common error.
They may been operated by hundreds of regimes, but the term is 100% identified with Nazis now. It's a rather daft statement to say that these are not equated.
The reason this is being discussed here is because it isn't.
Tell me, how many other governments can you list that planned to deport large numbers of people living in their country out to a special island they controlled in order to take advantage of its extraterritorial legal limbo separating it from the source country?
https://en.wikipedia.org/wiki/Madagascar_Plan
If I had a nickel every time that happened, I'd have two nickels, but it's weird that it's happened twice.
https://en.m.wikipedia.org/wiki/Pacific_Solution
Most famously on Nauru https://en.m.wikipedia.org/wiki/Nauru_Regional_Processing_Ce...
Truly, at least use some basic bloody comparative reasoning to formulate arguments.
The majority here crossed legally and overstayed their visa. That's just basic table-stakes knowledge for this issue, please remember it. Bonus points if you know that overstaying is a civil infraction like a parking ticket, below the level of a misdemeanor.
> these initial rounds of deportations are prioritizing illegal aliens with violent criminal records.
That's what the professional liar says, but why would you trust him on that?
https://www.the-independent.com/news/world/americas/us-polit...
> These people are factually the worst of the worst.
How the hell can you "factually" know that when the Trump administration still won't even explain what criteria they are using!? (Even assuming they'll be making any honest effort at all.)
that and the Nazi salute at the inauguration.
https://en.wikipedia.org/wiki/World_War_II_evacuation_and_ex...
They even plotted to forcibly deport victims to fenced-in zones on an island which they would seize from another country, putting it outside their laws at home.
Except it wasn't Guantanamo Bay in Cuba, but:
https://en.wikipedia.org/wiki/Madagascar_Plan
None of anything from Trump's government is at all comparable so far and making such comparisons, aside from being a genuinely idiotic case of ideological exaggeration, is also an insult to real victims of real historical concentration camps and violent regimes.
For all the self-congratulatory patting on the back by many readers on HN about their "superior" intellectual reasoning, I see a remarkable abundance of largely failing at such reasoning when it comes to many comment threads and subjects on this site.
It’s been less than two weeks and you are already doing the groundwork to preemptively excuse it.
In a weird way the events of WWII saved the US from going down the same path because we were on it. Anything literally Hitler did became politically toxic and it's a bit worrying that this toxicity is wearing off. It kept us far away from the fence.
The thing that matters for concentration camps is large groups of related civilians detained to advance a political goal— to restore Germany to its former greatness or to make America great again.
(disclaimer: not an American)
Are these the traitors you're talking about?
Even if people couldn't dig up anything against the person, the monument could simply be linked to the "wrong part of our history" and be removed, like the Thomas Fallon's statue in San Jose [4]
[1] https://en.wikipedia.org/wiki/Bust_of_Ulysses_S._Grant_(San_...
[2] https://en.wikipedia.org/wiki/Statue_of_Francis_Scott_Key_(S...
[3] https://en.wikipedia.org/wiki/Statue_of_Junípero_Serra_(San_...
[4] https://en.wikipedia.org/wiki/Thomas_Fallon#Legacy
> On June 19, 2020, the monument was toppled by protestors and defaced with the words "Adios America" in red paint as a response to Grant's brief ownership of a slave.
> Francis Key
> The statue was toppled by vandals on June 19, 2020, in the wake of the murder of George Floyd.
> Juniperro Serra
> Other statues of Junípero Serra were involved as the protests expanded to include monuments of individuals associated with the controversy over the genocide of indigenous peoples in the Americas.
> Thomas Fallon
> Fallon remains a controversial figure in San Jose's history, owing to his role in the American Conquest of California.
Besides Francis Key, which appears to have been vandalized rather than something that people specifically wanted removed, the rest of them do actually have reasons they were taken down? You can disagree with those but they do exist.
For the people I listed, I'd encourage you to read more about them and form your own opinion instead of just quoting back from the links I shared earlier.
There is very little reason to believe that Serra treated the indigenous people differently from the rest of his mission or that he didn't believe in what he was preaching. Similarly, almost nothing is known about why Grant had William Jones for a year, but Grant's character had been proven through years of real action - fighting the war and, later, fighting the KKK.
People just clung to some speculative theories, interpreted them in the least charitable way, and get offended by it. While everyone else sees that the topic was labeled as "racist" or "controversial" and tries to distance themselves from it.
They waged war against the United States, attacking the seams of this very country's unity, which by even the Constitution's purposely narrow definition, is treason.
They were pardoned for their treason because they were traitors.
These false narratives villifying and dividing the country long after the Civil War concluded are nonsense.
-President Gerald Ford
>Whereas the authority of the Federal Government having been reestablished in all the States and Territories within the jurisdiction of the United States, it is believed that such prudential reservations and exceptions as at the dates of said several proclamations were deemed necessary and proper may now be wisely and justly relinquished, and that an universal amnesty and pardon for participation in said rebellion extended to all who have borne any part therein will tend to secure permanent peace, order, and prosperity throughout the land, and to renew and fully restore confidence and fraternal feeling among the whole people, and their respect for and attachment to the National Government, designed by its patriotic founders for the general good:
-President Andrew Johnson
The Civil War is over, it ended over a century and a half ago. I again, sincerely, ask you to screw off with that false narrative which serves only to divide and weaken the unity and solidarity of the country.
Ex-Confederates were traitors, that much is true. Were they never pardoned or otherwise acquitted of treason in a court of law then that would still be true today. Of course, that is not what happened: President Andrew Johnson pardoned them all unconditionally, and in the case of General Robert E. Lee (whose pardon was not processed due to clerical errors) his pardon was made effective retroactive to the date of application by an act of Congress.
Ex-Confederates are all therefore excused of their crime of treason, and to still attack them further citing that crime is baseless and unwarranted. They are Americans, and the men who fought are all American heroes who ultimately wished for a united and healed nation.
Calling Civil War era Americans traitors today is ruthlessly divisive rhetoric that only serves to sow conflict in a time when the country really does not need that kind of incendiary bullshit. Everyone who spews this false narrative should be ashamed of themselves, they are throwing mud upon those who chose to pursue peace instead of generational animosity.
[1]: https://constitution.congress.gov/browse/essay/artII-S2-C1-3...
otherwise we keep perpetuating the myth that the great/rich/powerful can do no wrong.
“ in the days after Joe Biden was inaugurated, data.gov showed about 1,000 datasets being deleted as compared to a day before his inauguration, according to the Wayback Machine.”
I feel really bad for the kids who've developed an alt-right persona during the pandemic, though. There used to be time to work that out for yourself, but now I think social media will provide the kiss of death for many career ambitions.
One day, upper-middle class people will stop seeing as brave and insightful calling half the country Nazis for not conforming to their suburban aesthetic. They've even eventually stopped progressively describing him as the submissive homosexual partner to Putin, I suppose because we've run out of fighting aged adult males in Ukraine so there's no reason to care about that any more. If Trump starts a war with Iran, they'll start praising him as the new Lincoln. It's not real.
The only weird part is that the people attacking Trump for censorship that hasn't happened are delighted by censorship. They think the main problem with the world is that too many people are allowed to speak about whatever they want.
> attacking Trump for censorship that hasn't happened
Federal funds are being withheld - illegally, in most cases - from any entity that supports or even acknowledges what the administration calls "radical gender ideology".
It's totalitarian and regressive. Nazi's a fine word for what the leaders of the party are showing themselves to be. I wouldn't call half the country Nazis - unless they chose to stand with the human trash in the White House. Then I wouldn't really have a problem with it.
It tends to be enough for most people to notice the fact that both are interested in removing undesirables from public life and generally making society less racially mixed.
Then the claims are unsubstantiated. Political speech is some of the most twisted and manipulated. Unsubstantiated political claims are basically worthless. I'm willing to trust, if you are willing to verify. Engage in the discussion if you are serious about your claims.
"Everyone who disagrees with me is a National Socialist", is a tired line I've been reading on the Interwebs for a long time now. If you are unwilling to substantiate it, it is an odious comparison that trivializes one of the most murderous ideologies of the last century, second only to communism. These ideas demand serious treatment.
If you genuinely care about such things, you would take them more seriously. You might take the time to understand the genesis of these collectivist ideologies. After that, you might choose to champion ideals which stand in contrast. Individualism and individual self-ownership as the basis of private property would be one approach. Those who reason from these first principles are generally in favor of reducing the size of the state.
Political speech is perfectly understandable as such. It's the opposite of worthless, if you are trying to make a political point, as I was. I was very clear about the important (not total) parallels between then and now. I trivialize nothing.
Am I going to find you defending mainstream Democrats from accusations of socialism, or is your impassioned defense of the Republican party as it existed up until 10 years ago all you're offering?
I don't see a defense of the Republican Party here. Overall I'd rate them as a big disappointment. Although Trump coming out strong and cutting is a reason to be cautiously optimistic. It certainly isn't a reason to invoke Godwin's Law. As I've explained repeatedly here, cutting government is basically the opposite of the National Socialist economic program. If you are not informed enough to dispute this, then now would be a good time to do some research for yourself.
You dodge the question that puts the lie to your high-minded concern about using words according to their precise definition, which is unreasonable and serves to inhibit discourse, as you must realize.
You need only actually read one of his executive orders - again, ILLEGALLY directing appropriated funds not be spent - to see that cutting government is not the only thing these ideologues are attempting to do.
You might, finally, look into what Godwin himself has said regarding his "law" and Trump.
>removing undesirables from public life and generally making society less racially mixed.
These are more unsubstantiated claims.
https://en.wikipedia.org/wiki/Institut_f%C3%BCr_Sexualwissen...
>The institute was financed by the Magnus-Hirschfeld-Foundation, a charity which itself was funded by private donations.
Again, the National Socialists increased the purview of the state by interfering with a privately run and privately funded non-profit. They used public funds to attack and regulate private, voluntary behavior.
> For example, in the days after Joe Biden was inaugurated, data.gov showed about 1,000 datasets being deleted as compared to a day before his inauguration
It's almost like this stuff happens regularly. If <insert Dem savior> wins in 2028, tons of government websites will also change in the first couple weeks of their presidency. Is it because they're a fascist dictator? Or is it because those websites reflect the administration's viewpoints on issues?
Wish people would take a deep breath and step back and think a little more. I despise Trump, but there's crying wolf and then there's the current state of media and online discourse. Trump thrives in this type of environment. He purposefully fosters it. Playing gotcha with him doesn't work because he doesn't care.
All I’m seeing is a president that is doing exactly what he promised, so I’m very much in the “let the chips fall where they may” category.