NHacker Next
  • new
  • past
  • show
  • ask
  • show
  • jobs
  • submit
Show HN: Run an 80B Qwen in 4.3 GB of RAM on a Mac, and a 35B on an iPhone (github.com)
jbird99 5 minutes ago [-]
At what, 10 tokens per hour? These disk swapping methods all have the same drawbacks - kill your drive early, and slow as hell.
brrrrrm 15 minutes ago [-]
this is cool but like, are we just vibe coding NAND burners at this point? these decode times don't really tell the whole story, because prefill becomes the bottleneck.

half an hour to process 10k tokens on an M5 seems... not great

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact
Rendered at 04:00:31 GMT+0000 (Coordinated Universal Time) with Vercel.