With that being said, the web demo is not particularly impressive. It really doesn't like anything I throw at it. I'm fine with accepting that fine-tuning is the solution to this, but I wonder if there's anything to gain from a bigger model? I know it's completely counter to the whole point of this, but a 14MB binary using 28MB of RAM seems unnecessarily small and pretty arbitrary.

Like, what does a 28MB binary get you? Or a 140MB binary? Or a 1.4MB binary? I'm guessing the choice of 14MB came from minimizing the size as much as possible while meeting certain requirements/performance expectations, but even a Pi 5 has plenty more room to spare. Curious if there's a good explanation for this (which I may have missed in my skim of the post).

> Make it a little warmer in here.

The reply:

> "name": "set_thermostat", > "arguments": { > "temperature": 65, > "mode": "cool", > ... > "reasoning": "'warmer' implies need for cooling; set_thermostat with temperature 65 (typical warmth) and mode 'cool'.",

Maybe I'm doing it wrong?

“ 5° warmer”

And it said:

“ setting the temperature to 5°F”

Query: "Make the living room dark" Agent: "User wants lights on in living room. 'dark' implies dim. Room 'living room', action 'on'." (And on every test I did, it just completely ignored the "brightness" parameter)

It also appears to have no concept of what a door or light actually is, whenever the query diverges from "Lock door X" or "Turn on light X", it tries to shoehorn whatever additional context is given into the device name:

Query: "Lock out the vacuum salesman at the front door" Agent tries to lock "front door vacuum salesman"

"The way you talk really makes me appreciate silence" is classified as "positive" with 82% confidence.

Query: HN

Result:

{ "function_calls": [ { "name": "lock_door", "arguments": { "door": "front door" } } ], "reasoning": "User wants to lock the door. No specific door mentioned, so use 'front door' as default.", "confidence": 0 }

I'd expect it to at least ignore (call no tools) for the queries that it doesn't understand. And it seems like it does do that, just not consistently.

I'm wondering what is the overall thesis/plan here and where exactly the innovation lies? Would love if you can throw light on below,

  - If I understand, this is complete stack of a custom architecture (attention only transformers), custom quantisation format and a runtime engine all packaged together?
  - How do you differentiate / compete against LiteRT (former TensorFlowLite) and Lite RT LM? Google is heavily investing in this ecosystem because Android is where they have distribution moat. Wouldn't it be easier for me as a developer to build on top of LiteRT since it is relatively open ecosystem and I can pack large number of open models from HF directly?
  - What exact challenges you saw with TFLite, TVM etc that prompted this effort ?
  - What will be the pricing model like?

Do the creators take something like DeepSeek, and then delete most of the neurons to whittle down the size?

Sets lights to 30% but also off

> Turn the lights low in the bedroom

Sets lights to on

This is a cool idea but I think humans assume more than 14MB of intelligence. This is like the unhelpful guard in the swamp castle of Monty Python's Holy Grail

  import needle

  @needle.tool
  def add(a: int, b: int):
      "Add two numbers."
      return a + b

  agent = needle.Needle(tools=[add])
  print(agent.run("calculate 1 + 1?")["reasoning"])

conclusion: completly useless

While most of the industry focuses on the frontier of “intelligence” (function), a release like this represents the frontier of the other end of the spectrum (form).

Both are important if we ever want to see “Opus-level” capability running locally on commodity machines in the future.

Re: robotics: I'm unsure how this could be helpful.

It fails a pretty simple navigation prompt.

X0: (0.0, 0.0). Object bounding box: [1.0, 1.0, 2.0, 2.0]. navigate to (3.0,3.0)

I changed it to "call path planner to navigate: a_star(x0, xf, obs)"

Another fail.

My intuition tells me micro llms will/are important for robotics. I just can't grok it. Can someone without control theory experience give me a good example?

Probably at the planning level of the navigation stack. That's where I see reasoning being helpful. Lower than that...idk

Give me an example of a robotics prompt that seems useful and I'll give you an example why we don't need LLMs to be a tracking controller, etc.

Edit: I have a pile of d1 minis, but not much time.

Please, though, take a pass at humanizing the text on the page. It's Clauded up all over and makes it hard to read.

{ "function_calls": [ { "name": "lock_door", "arguments": { "door": "tv" } } ], "confidence": 0.0158 }

Very interesting, seems confidence is 0 when tool calls are right?

Given its basically an O(1) lookup with disk space being the main constraint, I was curious if you've tried ablating engram layers and sizes across your setup?

Also, why mHC over attention residuals?

The demo is super — I'm just having trouble seeing the whole picture for e.g. a screenless device.

ETA: pun not intended

so many interesting lowfi hardware use cases

  {
    "function_calls": [
      {
        "name": "set_thermostat",
        "arguments": {
          "temperature": 72,
          "mode": "cool",
          "room": "living room"
        }
      }
    ],
    "reasoning": "'as dark as possible' -> set_thermostat to warm; 'dark' implies higher temperature; 'cool' mode for darkness.",
    "confidence": 0
  }

Since it seems limited to matching a few templates and otherwise falling flat on its face, I wonder how 14MB of regexes would fare in its stead. Normally you wouldn't want to parse arbitrary natural language input with regex because of how tedious and brittle it would be, but for the tedium we have LLMs and this alternative isn't exactly robust either.

For the longest time, I conceptualized LLM's as Text Input -> Text Output transformers, then later as Text Input -> Video Output transformers. Later still I conceptualized them (if they were general purpose) as Any Format Input -> Any Format Output transformers...

The idea of a smaller parameter model runable on smaller/slower/less complex hardware (computers with no GPU, slower CPU's, less memory, aka "Edge Devices") trained for Text Input -> JSON Output (used for tool calls, etc.) I could honestly not conceptualize before seeing the demo on the web page...

But now that I've seen it and conceptualized it -- I'd have to say: "Yes, there's definitely a huge niche, a huge market for this, directly between the non-LLM driven tools and software and SaaS's of yesteryear, and the latest, cutting edge Frontier AI models of today!"

So, I like Needle a lot!

I like Needle a lot, and I love the idea of any tiny resource-thrifty LLM that can run on older hardware, that outputs only JSON!

I can see a huge market for it!