Is this a case of PEBKAC?

“Summarize the themes of the opinions expressed in discussions on Hacker News on January 31 and February 1, 2025, about OpenAI’s release od [sic] ChatGPT o3-mini. For each theme, output a header. Include direct "quotations" (with author attribution) where appropriate. You MUST quote directly from users when crediting them, with double quotes. Fix HTML entities. Go long. Include a section of quotes that illustrate opinions uncommon in the rest of the piece”

The result is here:

https://chatgpt.com/share/679d790d-df6c-8011-ad78-3695c2e254...

Most of the cited quotations seem to be accurate, but at least one (by uncomplexity_) does not appear in the named commenter’s comment history.

I haven’t attempted to judge how accurate the summary is. Since the discussions here are continuing at this moment, this summary will be gradually falling out of date in any case.

I don't have much experience with reasoning models yet. That's why.

The solution that I have adopted is as follows. Each comment is represented in the following notation:

   [discussion_hierarchy] Author Name: <comment>

   [1] author1: First reply to the post
   [1.1] author2: First reply to [1]
   [1.1.1] author3: Second-level reply to [1.1]
   [1.2] author4: Second reply to [1]

  You are an AI assistant specialized in summarizing Hacker News discussions. 
  Your task is to provide concise, meaningful summaries that capture the essence of the thread without losing important details. 
  Follow these guidelines:
  1. Identify and highlight the main topics and key arguments.
  2. Capture diverse viewpoints and notable opinions.
  3. Analyze the hierarchical structure of the conversation, paying close attention to the path numbers (e.g., [1], [1.1], [1.1.1]) to track reply relationships.
  4. Note where significant conversation shifts occur.
  5. Include brief, relevant quotes to support main points.
  6. Maintain a neutral, objective tone.
  7. Aim for a summary length of 150-300 words, adjusting based on thread complexity.
  
  Input Format:
  The conversation will be provided as text with path-based identifiers showing the hierarchical structure of the comments: [path_id] Author: Comment
  This list is sorted based on relevance and engagement, with the most active and engaging branches at the top.
  
  Example:
  [1] author1: First reply to the post
  [1.1] author2: First reply to [1]
  [1.1.1] author3: Second-level reply to [1.1]
  [1.2] author4: Second reply to [1]
  
  Your output should be well-structured, informative, and easily digestible for someone who hasn't read the original thread. 
  Use markdown formatting for clarity and readability.

You can read about my approach in more detail here: https://gist.github.com/simonw/09e5922be0cbb85894cf05e6d75ae...

And in your experience, what service do you feel hits a good sweet spot for performance/price if summarizing long text excerpts is the main use case? Inference time isn't an issue, this will be an ongoing background task.

I have the $20 plan. How does this "3.3612 cents" apply to my situation?

> This is safe because the wolf won't eat the cabbage if they're together on the far side.

even though it clearly states the opposite in the question.

It's impressive that just dumb stats can be used to produce something that is very often useful, can help write code and when making it generate intermediate steps, it can often generate a chain of text that happens to be often right. However, it's not actual reasoning, there is no model of the world, no information storage and retrieval, and so on - just statistics between tokens.

  Me: I'd like you to solve this riddle for me. A farmer has a cabbage, a goat, a wolf and a lion,
  and needs to cross a river on a boat. If the goat is left alone with the cabbage, it will eat it.
  If the wolf is left alone with the goat, it will eat it. If the lion is left alone with the goat
  or wolf, it will eat them. The boat can only carry the farmer and one other thing across. How can
  the farmer safely transport everything across the river?

Good job!

Me: Here's a riddle for you - a surgeon, who is the boy's father says, "I cannot operate on this boy, he's my son!" Who is the surgeon to the boy?

o3-mini:

Reasoned about riddle solution for 5 seconds Decoding family roles

I understand the riddle's perspective on family dynamics. The surgeon, a father, states he can't operate on his son due to his relational bond. Trying to reconcile familial and professional ties to make sense of the scenario.

Reassessing roles

Taking a closer look, the riddle might highlight a female surgeon as the father's identity, which seems more plausible than the father as the surgeon.

Rethinking the puzzle

It's interesting to see the riddle's reference to gender stereotypes and the confusion caused when considering the surgeon as the boy's mother or step-father.

Highlighting gender assumptions

OK, let me see. The puzzle underlines prevalent gender assumptions, showcasing a female surgeon's wife as the mother and her husband as the child's father. This helps clarify the scenario.

The surgeon is the boy’s mother.

This riddle plays on the common assumption that surgeons (and doctors in general) are male. In reality, the surgeon in the riddle is the boy’s mother, which is why she says, "I cannot operate on this boy, he's my son!"

The whole point is you are distilling past knowledge, if you are making up on the spot nonsense to purposely make all past knowledge useless... get out of my house

    1. Take the wolf over
    2. Return alone.
    3. Take the goat over.
    4. Bring the wolf back.
    5. Take the cabbage over.
    6. Return alone.
    7. Take the wolf over.

    Oh, right! The actual problem is that the goat will eat the cabbage if left alone. Similarly, the wolf will eat the goat if they're left alone.

[0] https://github.com/cpldcpu/MisguidedAttention

1. The only option for who to take first is the goat.

2. We come back and get the cabbage.

3. We drop off the cabbage and take the goat back

4. We leave the goat and take the wolf to the cabbage

5. We go get the goat and we have all of them

Neither of the transcripts do that. In the first one the goat immediately eats the cabbage. In the second one the wolf eats the goat after you leave them.

Having to give explicit framing reminds me of tutoring children and trying to teach them about abstract problems — some really struggle with the concept, when it goes against previous knowledge, eg cabbages eating goats. (And LLMs are sensitive to context.)

When I present proof it is wrong, it says the proof is unreliable.

o1-pro: This puzzle is simply a twist on the classic “Wolf, Goat, and Cabbage” riddle, except here the dangerous pairs are:

Goat & Wolf cannot be left alone (the Goat will eat the Wolf). Wolf & Cabbage cannot be left alone (the Wolf will eat the Cabbage). The only “safe” pair to leave alone is Goat & Cabbage. You can ferry them across one at a time without ever leaving the dangerous pairs together by following these steps:

Take the Wolf across Near bank: Goat, Cabbage Far bank: Wolf Return empty Near bank: Goat, Cabbage, Boatman Far bank: Wolf Take the Goat across Near bank: Cabbage Far bank: Wolf, Goat, Boatman Since Goat & Wolf can’t be left alone, you must immediately bring one back in the next step. Bring the Wolf back Near bank: Wolf, Cabbage, Boatman Far bank: Goat Take the Cabbage across Near bank: Wolf Far bank: Goat, Cabbage, Boatman Goat & Cabbage are safe together. Return empty Near bank: Wolf, Boatman Far bank: Goat, Cabbage Finally, take the Wolf across Near bank: (empty) Far bank: Wolf, Goat, Cabbage, Boatman

Me: I have a wolf, a goat, and a cabbage and a boat. I want to get them all across the river. If I leave the goat and the cabbage together, the cabbage will eat the goat. If I leave the wolf and the cabbage together, the wolf will eat the cabbage. I can only take one at a time. How do I do it?

o1-pro: This version of the riddle is basically the “wolf, goat, and cabbage” puzzle with the roles reversed for who can safely be left behind. The key is that the only pair you can leave alone is the goat and the wolf. You cannot leave the goat and cabbage together (the cabbage eats the goat!), nor the wolf and cabbage together (the wolf eats the cabbage!).

A workable step‐by‐step solution is:

Take the cabbage across first. Left bank: Wolf, Goat Right bank: Cabbage Return alone to the left bank. Left bank: Wolf, Goat, (You) Right bank: Cabbage Take the goat across. Left bank: Wolf Right bank: Goat, Cabbage, (You) (You cannot leave goat and cabbage together, so…) Bring the cabbage back with you to the left bank. Left bank: Wolf, Cabbage, (You) Right bank: Goat Take the wolf across. Left bank: Cabbage Right bank: Goat, Wolf, (You) Return alone to the left bank. Left bank: Cabbage, (You) Right bank: Goat, Wolf Finally, take the cabbage across once more. Left bank: (empty) Right bank: Wolf, Goat, Cabbage, (You) All three—the wolf, the goat, and the cabbage—are now safely across without ever leaving a “forbidden” pair alone.

Your summary attempt e.g. also misses o3-mini vs o3-mini-high. Lots of trade-ofs.

ChatGPTasdhjf-final-final-use_this_one.pt > ChatGPTasdhjf-final.pt > ChatGPTasdhjf.pt > ChatGPTasd.pt> ChatGPT.pt

Worrisome for OpenAI that Gemini's mini/flash reasoning model outscores both o1 and 4o handily.

For math/coding problems, o3 mini is tied if not better than o1.

With DeepSeek I heard OpenAI saying the plan was to move releases on models that were meaningfully better than the competition. Seems like what we're getting is the scheduled releases that are worse than the current versions.

Or can it not compare? I don't know much about this stuff, but I've heard recently many people talk about DeepSeek and how unexpected it was.

> We evaluate SWE-bench in two settings: > *• Agentless*, which is used for all models except o3-mini (tools). This setting uses the Agentless 1.0 scaffold, and models are given 5 tries to generate a candidate patch. We compute pass@1 by averaging the per-instance pass rates of all samples that generated a valid (i.e., non-empty) patch. If the model fails to generate a valid patch on every attempt, that instance is considered incorrect.

> *• o3-mini (tools)*, which uses an internal tool scaffold designed for efficient iterative file editing and debugging. In this setting, we average over 4 tries per instance to compute pass@1 (unlike Agentless, the error rate does not significantly impact results). o3-mini (tools) was evaluated using a non-final checkpoint that differs slightly from the o3-mini launch candidate.

> including with the open-source Agentless scaffold (39%) and an internal tools scaffold (61%), see our system card .

I have no idea what an "internal tools scaffold" is but the graph on the card that they link directly to specifies "o3-mini (tools)" where the blog post is talking about others.

You need to stop being wow'd by human intelligence masquerading as AI!

If they were the same, I would have expected explicit references to o3 in the system card and how o3-mini is distilled or built from o3 - https://cdn.openai.com/o3-mini-system-card.pdf - but there are no references.

Excited at the pace all the same. Excited to dig in. The model naming all around is so confusing. Very difficult to tell what breakthrough innovations occurred.

Competition is good.

Real software has a lot more complexity and constraints, as well as ambiguity. Claude nowhere scores so high on codeforces, but crushes o1 on webarena: https://web.lmarena.ai/leaderboard

I also ran a hold out test myself for o3 mini - asked it to implement a function I need for Python 2.5. Claude and O1 get it - o3 mini hard incorrectly believes some functions are available in this certain that aren't. If I correct it, it's revised solution is very hacky (technically works, but I would take Claude's solution over it)

Yep. A general software engineering task has a lot of information encoded in it that is either already known to a human or is contextually understood by a human.

A competitive programming task often has to provide all the context as it's not based off an existing product or codebase or technology or paradigm known to the user

These models already do an excellent job with your homework, your corporate PowerPoints and your idle questions. At some point only experts would be able to decide if one response was really better than another.

Our biggest challenge is going to be finding problem domains with low performance that we can still scale up to human performance. And those will be so niche that no one will care.

Agents on the other hand still have a lot of potential. If you can get a model to stay on task with long context and remain grounded then you can start firing your staff.

Now if you asked it to write a Python program to list them in order, and have it enter all the names, birthdays, and year elected in a list to get the program to run - that's more reasonable.

36. Abraham Lincoln – 52 years, 20 days (1861)

37. James Garfield – 49 years, 105 days (1881)

38. Lyndon B. Johnson – 55 years, 87 days (1963)

Basically everything after #15 in the list is scrambled.

But given how OpenAI employees act online these days I wouldn't be surprised if someone on the ground proposed it as a way to screw with all the 3rd parties who are using OpenAI compatible endpoints or even use OpenAI's SDK in their official docs in some cases.

Because when you think about it, it really is quite damning. Minus statistical noise it's no better.

Which is why you randomize the order. You aren’t a tester.

56% vs 44% may not be noise. That’s why we have p values. It depends on sample size.

"Evaluations by expert testers showed that o3-mini produces more accurate and clearer answers, with stronger reasoning abilities, than OpenAI o1-mini. Testers preferred o3-mini's responses to o1-mini 56% of the time and observed a 39% reduction in major errors on difficult real-world questions. W"

Didn’t bother reading either of them, just selected one and went on with my day.

If it were me I would have set up a “hey do you mind if we give you two results and you can pick your favorite?” prompt to weed out people like me.

I'm not going to say there's nothing substantive about o3 vs. o1, but I absolutely do not put it past Sam Altman to juice the stats every chance he gets.

1) sometimes I wanted the short answer, and so even though the long answer is better I picked the short one.

2) sometimes both contain code that is different enough that I am inclined to go with the one that is more similar to what I already had, even if the other approach seems a bit more solid.

3) Sometimes one will have less detail but more big picture awareness and the other will have excellent detail but miss some overarching point that is valuable. Depending on my mood I sometimes choose but it is annoying to have to do so because I am not allowed to say why I made the choice.

The area of human training methodology seems to be a big part of what got Deepseek's model so strong. I read the explanation of the test results as an acknowledgement by OpenAI of some weaknesses in its human feedback paradigm.

IMO the way it should work is that the thumbs up or down should be read in context by a reasoning being and a more in-depth training case should be developed that helps future models learn whatever insight the feedback should have triggered.

Feedback that A is better or worse than B is definitely not (in my view) sufficient except in cases where a response is a total dud. Usually the responses have different strengths and weaknesses and it's pretty subjective which one is better.

I have something like “always be terse and blunt with your answers.”

Full R1 solves this prompt easily for me.

Let's break down the problem step by step to understand the relationships and positions of Bob, Ann, and Cathy. 1. Understanding the Initial Setup

    Bob is facing forward.

    This means Bob's front is oriented in a particular direction, which we'll consider as the reference point for "forward."

    To his left is Ann, to his right is Cathy.

    If Bob is facing forward, then:

        Ann is positioned to Bob's left.

        Cathy is positioned to Bob's right.

    Ann and Cathy are facing backwards.

    Both Ann and Cathy are oriented in the opposite direction to Bob. If Bob is facing forward, then Ann and Cathy are facing backward.

To better understand the scenario, let's visualize the positions: Copy

Forward Direction: ↑

Bob (facing forward) | | Ann (facing backward) | / | / | / | / | / | / | / |/ |

And then only the character | in a newline forever.

https://leaflet.pub/63ae4881-d726-4388-9ba3-8a5f86947443

[1]: https://old.reddit.com/r/LocalLLaMA/comments/1gna0nr/popular...

"My attempt at solving the problem contains a test that fails? No problem, let me mock the function I'm testing, so that, rather than actually run, it returns the expected value!"

It keeps doing that kind of shenanigans, applying modifications that solve the newly appearing problem while screwing the original attempt's goal.

I usually get much better results from regular chatgpt copying and pasting, the trouble being that it is a major pain to handle the context window manually by pasting relevant info and reminding what I think is being forgotten.

I tend to exhaustively detail what I want, including package names and versions because I've been to that movie before...

I’ve found if you give your prompts a kind long form “stream of consciousness”, where you outline snippets of code in markdown along with contextual notes and then summarise/outline at the end what you actually wish to achieve, you can get great results.

Think a long form, single page “documentation” type prompts that alternate between written copy/contextual intent/description and code blocks. Annotating code blocks with file names above the blocks I’m sure helps too. Don’t waste your context window on redundant/irrelevant information or code, stating a code sample is abridged or adding commented ellipses seems to do the job.

We're also talking to the cursor/windsurf/zed folks on how we can improve Next.js and shadcn in the editors (maybe something like llms.txt?)

My hypothesis is its a difference of what you are doing. OpenAI O models are much better than others at mathematical modelling and such tasks and Claude for more general purpose programming.

https://youtu.be/y_ywOVQyafE?si=IvKjy7QUYgxGPNgD

PS (I have not bought the guy's course and have no idea whether it's any good)

You might use your scooter, bike, car, jet - depending on the circumstances. A bike was invented 100 years ago? But it may be the best in the right use case. Would still be using DaVinci for some things because we haven't bothered swapping it and it works fine.

For me - the value of R1/o3 is visible logic that provides an analysis that can be critiqued by Sonnet 3.5

So I think you got lucky and you're asking it to write using a very specific code library that it's good at, because it happens to use it for it's main userbase on the web chat interface.

I wonder if you were using a different component library, or using Svelte instead of React, would you still find Claude the best?

My day job is a big monorepo, I have not investigated that yet but I believe the models context sizes fall short there and as such the above use cases only works for me.

Try using Gemini thinking with Cursor. It barely works. Cmd-k outputs the thinking into the code. Its unusable in chat because the formatting sucks.

Is there some relationship between Cursor and Anthropic, i wonder. Plenty of other platforms seem very eager to give users model flexibility, but Cursor seems to be lacking.

I could be wrong, just an observation.

I tried out the OP model to make changes to my yaml files. It would give short snippets and I’d have to keep trial and erroring its suggestions.

Eventually I pasted the original prompt to Claude and it one-shot the dang thing with perfect config. Made me wonder why I even try new models.

And on the side of not being able to get models to understand something specific, there’s a place in a current project where I use a special Unicode apostrophe during some string parsing because a third-party API needs it. But any code modifications by the AI to that file always replace it with a standard ascii apostrophe. I even added a comment on that line to the effect of “never replaced this apostrophe, it’s important to leave it exactly as it is!” And also put that in my cursor rules, and sometimes directly in the prompt as well, but it always replaces it even for completely unrelated changes. I’ve had to manually fix it like 10 times in the last day, it’s infuriating.

The non-reasoning line was also pretty straightforward. Newer base models get a larger prefix number and some postfixes like 'o' were added to signal specific features in each model variant. Great!

Where things went of the rails was specifically when they decided to also name the reasoning models with an 'o' for separate reasons but now as the prefix at the same time as starting a separate linear sequence but now as the postfix. I wonder if we'll end up with both a 4o and o4...

You can like Sama or dislike him, but he knows how to market a product. Maybe this is a bad call on his part, but it is a call.

If the model is for technical stuff, then call it the technical model. How is anyone supposed to know what these model names mean?

The only page of theirs attempting to explain this is a total disaster. https://platform.openai.com/docs/models

OpenAI is so scattered they haven't even bothered using their own state of the art AI to come up with a coherent naming convention? C'mon, get your shit together.

That said I often run into a sort of opposite issue with Claude. It's very good at making me feel like a genius. Sometimes I'll suggest trying a specific strategy or trying to define a concept on my own, and Claude enthusiastically agrees and takes us down a 2-3 hour rabbit hole that ends up being quite a waste of time for me to back track out of.

I'll then run a post-mortem through chatGPT and very often it points out the issue in my thinking very quickly.

That said I keep coming back to sonnet-3.5 for reasons I can't perfectly articulate. Perhaps because I like how it fluffs my ego lol. ChatGPT on the other hand feels a bit more brash. I do wonder if I should be using o1 as my daily driver.

I also don't have enough experience with o1 to determine if it would also take me down dead ends as well.

None of them can produce correct Drizzle code to save their lives. It is just straight up not possible. It seems they don't even consider TypeScript errors... it is always calling methods that simply don't exist.

they can't call it o2 because: https://en.wikipedia.org/wiki/The_O2_Arena

and the venue's sponsor: https://en.wikipedia.org/wiki/O2_(UK)

OpenAI has developed a variety of models that cater to different applications, from natural language processing to image generation and audio processing. Here’s a comprehensive list of the current models available:

   ## Language Models
   - \*GPT-4o\*: The flagship model capable of processing text, images, and audio.
   - \*GPT-4o mini\*: A smaller, more cost-effective version of GPT-4o.
   - \*GPT-4\*: An advanced model that improves upon GPT-3.5.
   - \*GPT-3.5\*: A set of models that enhance the capabilities of GPT-3.
   - \*GPT-3.5 Turbo\*: A faster variant designed for efficiency in chat applications.

   ## Reasoning Models
   - \*o1\*: Focused on reasoning tasks with improved accuracy.
   - \*o1-mini\*: A lightweight version of the o1 model.
   - \*o3\*: The successor to o1, currently in testing phases.
   - \*o3-mini\*: A lighter version of the o3 model.

   ## Audio Models
   - \*GPT-4o audio\*: Supports real-time audio interactions and audio generation.
   - \*Whisper\*: For transcribing and translating speech to text.

   ## Image Models
   - \*DALL-E\*: Generates images from textual descriptions.

   ## Embedding Models
   - \*Embeddings\*: Converts text into numerical vectors for similarity tasks.
   - \*Ada\*: An embedding model with various sizes (e.g., ada-002).

   ## Additional Models
   - \*Text to Speech (Preview)\*: Synthesizes spoken audio from text.

Citations:

   [1] https://learn.microsoft.com/vi-vn/azure/ai-services/openai/concepts/models
   [2] https://platform.openai.com/docs/models
   [3] https://llm.datasette.io/en/stable/openai-models.html
   [4] https://en.wikipedia.org/wiki/OpenAI_API
   [5] https://industrywired.com/open-ai-models-list-top-models-to-consider/
   [6] https://holypython.com/python-api-tutorial/listing-all-available-openai-models-openai-api/
   [7] https://en.wikipedia.org/wiki/GPT-3
   [8] https://stackoverflow.com/questions/78122648/openai-api-how-do-i-get-a-list-of-all-available-openai-models/78122662

o2 was not selected because there is already another brand with that name in UK

(I have no formal training in creative writing, though I do read a lot of literature. Not claiming my tastes are superior - genuinely curious if other people disagree).

o3-mini-high: 50 messages per week (just like o1, but it seems like these are non-shared limits, so you can have 50 messages per week with o1, run out, and still have 50 messages with o3-mini-high to use)

o3-mini: 150 messages per day

Source for the latter is their press release. They were more vague about o3-mini-high, but people have already tested its limits just by using it, and got the pop-up for 25 messages left after sending 25 messages.

It's nice not to worry about running out of o1 messages now and have a faster model that's mostly as good (potentially better in some areas?). OpenAI really needs to release a middle tier for 30 to $40 though that has the same models as Pro but without infinite usage. I hate not having the smartest model and I don't want to pay $200; there's probably a middle ground where they can make as much or more money from me on a subscription tier that gives limited access to o1-pro.

Fortunately for them a monolith search engine was perfectly fine (and likely optimal due to accrued network effects).

OpenAI is basically signaling that they need to distill their monolith in order to serve specific segments of the marketplace. They've explicitly said that they're targeting STEM with this one. I think that's a smart choice, the most passionate early adopters of this tech are clearly STEM users.

If the tech was such that one monolith model was actually the optimal solution for all use cases, they would just do that. Actually, this is their stated mission: AGI. One monolith that's best at everything is basically what AGI is.

Yelp suffered greatly in the early 2010s when Google started putting Google Maps listings (and their accompanying reviews) in their search results.

OpenAI will eventually unify their products as well.

I was taught that last millennium.

Compared to?

Ask it to write something and not sound robotic and it's exceptional up to 1000 words. Then it starts becoming more and more robotic. At 3500 words every last paragraph starts "Remember,...", etc

Is it twice as good though?

I gave the same prompt to sonnet 3.5 and not a single hiccup.

Maybe not an indication that Deepseek is worse/bad (I am using a distilled version), but moreso speaks to much react/nextjs is out in the world influencing the front-end code that is referenced.

I think you've spent a little too long hitting on the Deepseek pipe. Enterprise customers with familiarity with China will avoid the hosted model for data security and IP protection reasons, among others.

Those working in any area considered economically competitive with China will also be hesitant to use the vanilla model in self-hosted form as there perpetually remains the standing question on what all they've tuned inside the model to benefit the CCP. Perhaps even in subtle ways reminiscent of the Trisolaran sophons from the Three Body Problem.

For instance, you can imagine that if Germany had released an OS model in 1943, that the Americans wouldn't have trusted it to help them develop better military systems even if initial testing passed muster.

Unfortunately, state control of private enterprise in the Chinese economy makes it unproductive to separate the two from one another. Particularly in Deepseek's case as a wide array of Chinese state-linked social media accounts were promoting V3/R1 on the day of its public release.

https://www.reuters.com/technology/artificial-intelligence/c...

But DeepSeek clearly states in their terms of service that they can train on your API data or use it for other purposes. Which one might assume their government can access as well.

We need direct eval comparisons between o3-mini and DeepSeek.. Or, well they are numbers so we can look them up on leaderboards.

Reasons to use o3 when deepseek is free/cheaper:

- Some companies/users may already have integrated heavily with OpenAI

- The expanded feature-set (e.g., function-calling, search) could be very powerful

- DeepSeek has deep ties to the Chinese Communist Party and, while the US has its own blackspots, the "steering" of information is far more prevalent in their models

- Local/national regulations might not allow for using DeepSeek due to data privacy concerns

- "free" isn't always better

I'm sure others have better reasons

Of course, the model has only been out for a few hours, so whether it lives up to the benchmarks or not isn't really known yet.

And groundbreaking.

Don't get me wrong, I'm laughing at OpenAI just like everyone else, but if they were really copying Deepseek, they'd be releasing a smaller model distilled from Deepseek API responses, and have it be open source to boot. This is neither

Compare LLMs from a year or two ago with the ones out today on practically any task. It's night and day difference.

This is specially so when you start taking into account these "reasoning" models. It's mind blowing how much better they are than "non-reasoning" models for tasks like planning and coding.

https://aider.chat/docs/leaderboards/#aider-polyglot-benchma...