Before the bubble does pop, which I think is inevitable, there will be many stories like this one, and a lot of people will be scammed, manipulated, and harmed. It might take years until the general consensus is negative about the effects of these tools. All while the wealthy and powerful continue to reap the benefits, while those on slightly lower rungs fight to take their place. And even if the public perception shifts, the power might be so concentrated that it could be impossible to dislodge it without violent means.

What a glorious future we've built.

> at which people lose trust in these systems

Most of people do not lose trust in system as long as it confirms their biases (which they could've created in the first place).

It's mostly bad actors, and a smattering of optimists who believe that despite its current problems, AI will eventually and inevitably get better. I also wish the whole thing would calm down and come back to reality, but I don't think it's a bubble that will pop. It will continue to get artificially puffed up for a while because too many businesses and people have invested too much for them to just quit (sunk cost falacy) and there's a big enough market in a certain class of writer/developer/etc... for which the short term benefits will justify the continued existence of the AI products for a while. My prediction is that as the long term benefits for honest users peter out, the bubble won't pop, but deflate into a wrinkled 10 day old helium balloon. There will still be a big enough market driven by cons, ad tech and people trying to suck up as many ad dollars as possible, and other bad actors, that the tech will persist, and continue to infest the web/world for quite a while.

AI is the new crypto. Lots of promise and big ideas, lots of people with blind faith about what it will one day become, a lot of people gaming the system for quick gains at the expense of others. But it never actually becomes what it pretends/promises to be and is filled with people continuing the grift trying to make a buck off the next guy. AI just has better marketing and more corporate buy in than crypto. But neither are going anywhere.

If that outcome were likely, then Fox News and The Daily Mail would have died a death a decade ago and Trump wouldn’t be serving a 2nd term.

Yet here we are, in a world where it doesn’t matter if “facts” are truth or lies, just as long as your target audience agrees with the sentiment.

Thats naive. Look at all the tabloids thriving. The kind of people that bad actors target will continue to believe everything it says. They won't lose trust, or magazines like New York Post, the Sun or BILD would already have crossed to exist with their lies and deception. And Russia would not have so many cult members believing the lies they spread.

This whole attitude against AI reminds me of my parents being upset that the internet changed the way they live. They refused to take part in the internet revolution, and now they're surprised that they don't know how to navigate the web. I think that a part of them is still waiting for computers in general to magically disappear, and everything return to the times of their youth.

AIs can be trained to rely more on critical thinking rather than just regurgitating what it reads. The problem is just like with people, critical thinking takes more power and time. So we avoid it as much as possible.

In fact, optimizing for the wrong things like that, is basically the entire world's problem right now.

The answer isn't a technical advancement but a cultural shift. We need to develop a discipline of skepticism and mistrust. No amount of authority, understanding, reasoning, etc. can be delegated to something that comes from a screen. This will take generations.

Exactly. People say "we have invented X (the LLMs), now if we just invent Y (reasoning AGI) all of X's problems will be solved". Problem is, there's no indication Y is close or even remotely related to X!

> including systems that can evaluate news sources, understand satire, and so forth.

Lets take something that has been in the news recently: https://abcnews.go.com/Business/wireStory/investors-snap-gro...

"Nearly 27% of all homes sold in the first three months of the year were bought by investors -- the highest share in at least five years, according to a report by real estate data provider BatchData."

That sounds like a lot... and people are rage baited into yelling about housing and how it's unaffordable. They point their fingers at corporations.

If you go look at the real report it paints a different picture: https://investorpulse1h25.batchdata.io/?mf_ct_campaign=grayt... -- and one that is woefully incomplete because of how the data is aggregated.

Ultimately all that information is pointless because the real underlying trend has been unmovable for 40 something years: https://fred.stlouisfed.org/series/RSAHORUSQ156S

> every time they unthinkingly repeat propaganda

How do you separate propaganda from perspective, facts from feelings? People are already bad at this, the machines were already well soiled by the data from humans. Truth, in an objective form, is rare and often even it can change.

This is in fact the goal of Russian style propaganda. You have successfully been targeted. The idea is to spread so much confusion that you just throw up your hands and say, I'm not going to try and figure out what's going on any more.

That saps your will to be political, to morally judge actions and support efforts to punish wrongdoers.

https://www.rand.org/pubs/perspectives/PE198.html

https://en.wikipedia.org/wiki/Firehose_of_falsehood

https://jordanrussiacenter.org/blog/propaganda-political-apa...

https://www.newyorker.com/news/annals-of-communications/insi...

There's a more basic problem: it's two very different questions to ask "can the machine reason about the plausibility of things/sources?", and "how does it score on an evaluation on a list of authoritative truths and proven lies?" A machine that thinks critically will perform poorly on the latter, since, if you're able to doubt a bad-actor's falsehood, you're just as capable of doubting an authoritative source (often wrongly/overeagerly; maybe sometimes not). Because you're always reasoning with incomplete information: many wrong things are plausible given limited knowledge, and many true things aren't easy to support.

The system that would score best tested against a list of known-truths and known-lies, isn't the perceptive one that excels at critical thinking: it's the ideological sycophant. It's the one that begins its research by doing a from:elonmusk search, or whomever it's supposed to agree with—whatever "obvious truths" it's "expected to understand".

Yes, it's difficult to detect whether something is enemy propaganda if you only look at the content. During WWII, sometimes propagandists would take an official statement (e.g. the government claiming that food production was sufficient and there were no shortages) and redirect it unchanged to a different audience (e.g. soldiers on a part of the front with strained logistics). Then the official statement and enemy propaganda would be exactly the same! The propaganda effect coming from the selection of content, not its truth or falsity.

But it's very easy to detect whether something is enemy propaganda without looking at the content: if it comes from an enemy source, it's enemy propaganda. If it also comes from a friendly source, at least the enemy isn't lying, though.

A company that doesn't wish to pick a side can still sidestep the issue of one source publishing a completely made-up story by filtering for information covered by a wide spectrum of sources at least one of which most of their users trust. That wouldn't completely eliminate falsehoods, but make deliberate manipulation more difficult. It might be playing the game, but better than letting the game play you.

Of course such a process would in practice be a bit more involved to implement than just feeding the top search results into an LLM and having it generate a summary.

"different and dynamic truths" = fictions

We can not play the game.

I remember when people gave up on digital navigation because the traveling salesman issue makes it too expensive.

Not everything needs to result in a single perfect answer to be useful. Aiming for ~90%, even 70% of a right answer still gets you something very reasonable in a lot of open ended tasks.

I would actually be very interested in a system where there's nothing stored just as a "fact", but rather every piece of information is connected to its sources and the evidence provided.

The real problem is that most people just want answers, they're unwilling to follow the logical chain of thought. When I talk to LLMs I keep asking "but why are you telling me this" until I have a cohesive, logical picture in my mind. Quite often the picture fundamentally disagrees with the LLM. But most people don't want that, they just ask "tell me what to do".

This is a reflection of how social dynamics often work. People tend to follow the leader and social norms without questioning them, so why not apply the same attitude to LLMs. BTW, the phenomenon isn't new, I think one of the first moments when we realized that people are stupid and just do whatever the computer tells them to do was the wave of people crashing their cars because the GPS system lied to them.

There are personalized social media feeds, so why not have personalized LLMs that align with how people want their LLM to act.

> What we consider enemy propaganda today might be an official statement tomorrow.

Remember when worrying about COVID was sinophobia? Or when the lab leak was a far-right conspiracy theory? When masks were deemed unnecessary except for healthcare professionals, but then mandated for everyone?

There is no objective truth because humans are inherently ideological beings and what we consider objective is just a reflection of our ideology.

Consider markets - a capitalist's "objective truth" might be that they are the most efficient mechanism of allocating resources, a marxists "objective truth" might be that they are a mechanism for exploiting the working class and making the capitalist class even richer.

Here's Zizek, famous ideology expert, describing this mechanism via film analysis: https://www.youtube.com/watch?v=TVwKjGbz60k

> What is propaganda for one is truth for another, how could LLM tell the difference?

"How do you discern truth from falsehood" is not a new question, and there are centuries of literature on the answer. Epistemology didn't suddenly stop existing because we have Data(TM) and Machine Learning(TM), because the use of data depends fundamentally on modeling assumptions. I don't mean that in a hard-postmodernist "but can you ever really know anything bro" sense, I mean it in a "out-of-model error is a practical problem" way.

And yeah, sometimes you should just say "nope, this source is doing more harm than good". Most reasonable people do this already - or do you find yourself seriously considering the arguments of every "the end is nigh" sign holder you come across?

> What is propaganda for one is truth for another, how could LLM tell the difference ?

The article isn't even asking for it to tell the difference, just for it to follow its own information about credibility.

[flagged]

The example in this article is particularly funny. Pravda was founded in 1912, predating the internet, and had been Soviet's propaganda machine for its whole existence.

One needs a PhD in mental gymnastics to frame Pravda spreading misinformation as an attempt to specifically groom LLMs.

Yea, not entirely sure what's any different than all of the rest of history?????

Bad actors have been trying to poison facts for-fucking-ever.

But for whatever reason, since it's an LLM, it now means something more than it did before.

This actor?

https://techcrunch.com/2025/07/10/grok-4-seems-to-consult-el...

Assuming this isn’t happening now…

[dead]

The newspapers name originates from the times of USSR and before. It was about as factual then as it is now. But this kinds of ironies are not very rare in these kinds of organizations (Truth Social, Democratic People's Republic of Korea...).

What happened to all the turtles?

When AI can generate and pass formal proof, there is no truth anymore - we are the brain in the vat only left connect ourselves into the vat.

https://dmf-archive.github.io/docs/posts/cognitive-debt-as-a...

1) repeat that people also lie so it is okay for LLM to lie

2) ingest as much VC money and stolen training data as we can

3) profit

Yeah don’t use an established platform for your personal agenda.

I agree that it's a much bigger problem for LLMs, but to be fair it's also not how humans work. A long lasting, high volume stream of propaganda will have considerable effect on a human even if he is aware that it is false.

At every point, during a knowledge/data search for reaching a particular goal, the onus is _always_ on the person searching to do their best to ensure that the sources they use are accurate, and they do the effort required to ensure that they translate that properly to fit that goal.

The education system I grew up in was not perfect. Teachers were not experts in their field, but would state factual inaccuracies - as you say LLMs do - with authority. Libraries didn't have good books; the ones they had were too old, or too propaganda-driven, or too basic. The students were not too interested in learning, so they rote-learned, copied answers off each other and focussed on results than the learning process. If I had today's LLMs then, I'd have been a lot better off, and would've been able to learn a lot more (assuming that I went through the effort to go through all the sources the LLM cited).

The older you grow, you know that there is no arbiter of T-Truth; you can make someone/something that for yourself, but times change, "actual, factual history" could get proven incorrect, and you will need to update your knowledge stores and beliefs along with it, all the while being ready to be proved incorrect again. This has always been the case, and will continue to be, even with LLMs.

Maybe, just maybe people will learn they can’t trust everything that’s written online wether it’s done by a bot or even human.

Hell, they might learn that even real life authorities may lies, cheat and not have everyone’s interest in their mind.

Hope for the best, prepare for the worst.

Leaving aside the Israeli propaganda (certainly the US government shows strong alliance with that), you can't make such a statement without taking into account the nature of what America traditionally is.

A liberal multicultural postmodern democracy continually acting as if immigration (both legal and illegal) and diversity are its strengths, particularly when that turns out to be factual (see: large American cities becoming influential cultural exporters and hotbeds of innovation, like New York and Silicon Valley etc) means American propaganda is only more effective when it's backed by economic might.

It also means the American propaganda is WILDLY contradictory. There's a million sources and it's a noisy burst of neon glamour. It is simply not as controlled by authority, however they may try.

You cannot liken authoritarian propaganda to postmodern multicultural propaganda. The whole reason it's postmodern is that it eschews direct control of the message, and it's a giant scrum of information. Turns out this is fertile ground, and this is also why attacks by alien propaganda have been so effective. If you can grab big chunks of the American propaganda and turn it to your enemy weapon of war and destruction of America quite directly, well then the American propaganda is not on the same destructive level as your rigidly state-controlled propaganda.