At what, 10 tokens per hour? These disk swapping methods all have the same drawbacks - kill your drive early, and slow as hell.
brrrrrm 15 minutes ago [-]
this is cool but like, are we just vibe coding NAND burners at this point? these decode times don't really tell the whole story, because prefill becomes the bottleneck.
half an hour to process 10k tokens on an M5 seems... not great
Rendered at 04:00:31 GMT+0000 (Coordinated Universal Time) with Vercel.
half an hour to process 10k tokens on an M5 seems... not great