I like the idea of having automatic code creation from papers, but I’m scared of it.
Suppose you get a paper, you automatically implement the code, and then modify it a bit with a novel idea, and publish your paper. Then somebody else does that with your paper, and does the same.. at some point, we will have a huge quantity of vibe coded code on github, and two similar papers will have very different underlying implementations, so hard to reason about and hard to change.
From a learning perspective, you try to understand the code, and it's all spaghetti, and you loose more time understanding the code than it would take to just reimplement it. You also learn a lot by not only reading the paper but reading the authors code where most of the small details reside.
And I'm not even talking about the reliability of the code, test to know that it's the correct implementation. Authors try to make papers as close as possible to the implementation but sometimes subtle steps are removed, sometimes from inadvertance, sometimes because the number of pages is lionmited.
A paper and an implementation are not one-to-one mappings
tomrod 5 hours ago [-]
> we will have a huge quantity of vibe coded code on github
That may actually be an improvement over much of the code that is generated for papers.
colkassad 10 hours ago [-]
It would be neat to run their pdf through their implementation[1] and compare results.
Damn, i was hoping the link was your result of that. Please do that. I can't start another project currently. But i'd love the short result as an anecdote. But if you don't do it, i might have to. Please let me know. Great idea, really.
omneity 8 hours ago [-]
If I was the paper author I would have done it and include the results as an appendix or a repo.
JackYoustra 8 hours ago [-]
haha would that itself be a product of the paper then?
omneity 8 hours ago [-]
Maybe by doing it enough times o3-mini will end up reimplementing itself?
endofreach 7 hours ago [-]
Imagine, this was actually the consequence— against all odds. The true power of AI is about to be discovered... through this silly experiment... and you are the one... all you gotta do— is do it. And imagine you don't do it, because you think it can't lead to such a serious result... and if we miss this great leap forward... go, throw your life away and do this. now. the universe is waiting on you, my friend.
ks2048 3 hours ago [-]
So who has a code2paper model that we can hook up in a loop?
sitkack 8 hours ago [-]
I have had good results doing bidirectional programming in Tex <=> Python.
bjourne 8 hours ago [-]
It relies on OpenAI's o3-mini model which (I think) you have to pay for.
Rendered at 05:27:33 GMT+0000 (Coordinated Universal Time) with Vercel.
Suppose you get a paper, you automatically implement the code, and then modify it a bit with a novel idea, and publish your paper. Then somebody else does that with your paper, and does the same.. at some point, we will have a huge quantity of vibe coded code on github, and two similar papers will have very different underlying implementations, so hard to reason about and hard to change.
From a learning perspective, you try to understand the code, and it's all spaghetti, and you loose more time understanding the code than it would take to just reimplement it. You also learn a lot by not only reading the paper but reading the authors code where most of the small details reside.
And I'm not even talking about the reliability of the code, test to know that it's the correct implementation. Authors try to make papers as close as possible to the implementation but sometimes subtle steps are removed, sometimes from inadvertance, sometimes because the number of pages is lionmited.
A paper and an implementation are not one-to-one mappings
That may actually be an improvement over much of the code that is generated for papers.
https://github.com/going-doer/Paper2Code