Author here, if you don't want to read all that, I'll post one excerpt that I think sums it up nicely:
> My point is, the spec must live somewhere, even if you don’t write it down. The spec is what you want the software to be. It often exists only in your head or in conversations. You and your team and your business will always care what the spec says, and that’s never going to change. So you’re better off writing it down now! And I think that a plain old list of acceptance criteria is a good place to start. (That’s really all that `feature.yaml` is.)
gnat 22 minutes ago [-]
Nice! Your spec-maxxing is very resonant. I've been doing working with explicit requirements: elicit them from conversation with me or introspecting another piece of software; one-shot from them; and keep them up-to-date as I do the "old man shouts at Claude" iterations after whatever one-shotting came up with.
Unlike you, I wish for the LLM to do as much of the work as possible -- but "as possible" is doing a lot of work in that sentence. I'm still trying to get clear on exactly where I am needed and where Opus and iterations will get there eventually.
It has really challenged me to get clearer on what a requirement is vs a constraint (e.g., "you don't get to reinvent the database schema, we're building part of a larger system"). And I still battle with when and how to specify UI behaviours: so much UI is implicit, and it seems quite daunting to have to specify so much to get it working. I have new respect for whoever wrote the undoubtedly bajillion tests for Flutter and other UI toolkits.
gnat 12 minutes ago [-]
Forgot to add: I get several benefits from doing this.
1. Specifications that live outside the code. We have a lot of code for which "what should this do?" is a subjective answer, because "what was this written to do?" is either oral legend or lost in time. As future Claude sessions add new features, this is how Claude can remember what was intentional in the existing code and what were accidents of implementation. And they're useful for documenters, support, etc.
2. Specifications that stay up to date as code is written. No spec survives first contact with the enemy (implementation in the real world). "Huh, there are TWO statuses for Missing orders, but we wrote this assuming just one. How do we display them? Which are we setting or is it configurable?" etc. Implementer finds things the specifier got wrong about reality, things the specifier missed that need to be specified/decided, and testing finds what they both missed.
I have a colleague working on saving architecture decisions, and his description of it feels like a higher-abstraction version of my saving and maintaining requirements.
energy123 1 minutes ago [-]
I do (1) the same but (2) differently. In my workflow, (2) are AI generated specs using human written (1) as the input. It's an intermediate stage between (1) and the codebase, allowing for a gradual token expansion from 30k to 250k to the final code which is 2-3M. The benefit I've found with this approach is it gives the AI a way to iterate on the details of whole system in one context window, whereas fitting the whole codebase into one prompt is impossible. The code is then nothing more than a style transfer from (2).
nalpha 3 minutes ago [-]
What's the difference between this and Jira. Your specs already live somewhere, it's where you defined them. That's why it's nice to put the Jira ticket number in your code / commit, so you can refer back to the spec when something breaks
slopinthebag 12 minutes ago [-]
I actually read it all since it did not contain any hints of being AI generated (although I wouldn't be surprised to learn you did use AI to write it), so thank you for that. It's kind of crazy how I now have the default expectation that posts posted here are AI slop with little thought or care put in.
I am also stealing the idea of talking to LLMs as if it's an email. So funny, we need to be joymaxxing a bit more I think :)
colinmarc 24 minutes ago [-]
Wow - I love programming in YAML! You know what would make this really fun? Sprinkle in some Jinja. Then we'll be cooking with gas.
photios 19 minutes ago [-]
Dreaming about ` | nindent 12` in my specs! :D
jwpapi 10 minutes ago [-]
And once you’ve written all these specs you realize it became so slow that it’s faster to do it yourself in editor
arikrahman 34 minutes ago [-]
I use OpenSpec for my spec management, and I scrolled down to the comparison. The gripe seems to be with a semantic difference. Specs describing a current system is the basis for AS/IS Gap Analysis.
Also, I mainly pursue these tools so that I can have AI accelerate this process and broker an agreement after negotiating specs with the agent.
energy123 31 minutes ago [-]
I can see one benefit to a structured yaml for specs like the OP is doing: it gives you more control over what you include in the context window. But coming up with a good schema that doesn't handicap you or add cognitive burden, compared to the freeform flexibility of md/txt, is a challenge.
arikrahman 20 minutes ago [-]
If the selling point is a new file format for spec management, it would be more interesting to provide an offering with org-mode. The author admits they were unaware of other pre-existing solutions before this project so I am providing context to their critique of OpenSpec.
photios 17 minutes ago [-]
Yesterday I heard about lat.md [1] which seems to have similar ideas about annotating code with spec refs. I now need to try them both.
That was my initial thought when reading the headline but the author states they didn't know it existed before doing this project and critiques it.
wesselbindt 36 minutes ago [-]
I'm still confused as to why folks don't just write executable specs.
cenamus 16 minutes ago [-]
So basically tests?
carlbarrdahl 24 minutes ago [-]
Could you expand on this?
booi 23 minutes ago [-]
code
arikrahman 17 minutes ago [-]
Literate programming would provide specs and code instead of working backwards from hard coded functions to figure out specs.
fudgeonastick 18 minutes ago [-]
If you're confused, and have tried Opus for coding, I'm keen to hear what problems or workflows it's not good at.
If you're genuinely confused, and haven't tried Opus for coding, then it's not surprising you're confused!
It is also okay for you to just not like the idea of LLMs for coding (but say that!).
oytis 5 minutes ago [-]
That's what the article is about - overcoming problems with AI cooding tools using specs in Yaml. If we've got that far, it might be better to write specs in a proper programming language instead and skip the AI layer altogether
augment_me 23 minutes ago [-]
Completely subjective take, but I feel like 95% of these "tools" that are prompt-engineering inventions created by the authors with their bias and to suit their needs don't have anything supporting them besides the authors' subjective experience.
I have seen the same idea with processes, pipelines, lists, bullet points, jsons, yamls, trees, prioritization queues all for LLM context and instruction alignment. It's like the authors take the structure they are familiar with, and go 100% in on it until it provides value for them and then they think it's the best thing since sliced bread.
I would like, for once, to see some kind of exploration/abalation against other methods. Or even better, a tool that uses your data to figure out your personal bias and structure preference for writing specs, so that you can have a way of providing yourself value.
up-n-atom 1 hours ago [-]
the token usage isn’t sustainable. formal english is a barrier but requirement for specification. brevity is the language of money and that’s the premise of management using ai.
fyi language alone can’t define/describe requirements which is why UML existed.
jstanley 38 minutes ago [-]
Natural language is a fully general system and can define and describe everything.
You could deterministically process any UML diagram into a prose equivalent.
And in fact you couldn't do the other way around (any prose -> UML) because UML is less powerful than natural language and actually can't express everything that natural language can.
rzzzt 4 minutes ago [-]
There are also diagram notation languages and LLMs are happy to both consume and produce e.g. Mermaid.
ako 49 minutes ago [-]
I think uml exists to help humans understand and communicate specifications, not because language alone is insufficient.
null_ptr1 12 minutes ago [-]
I mean, if you can't agree on what UML is, then what hope do you have to agree on what the spec says?
Rendered at 08:14:21 GMT+0000 (Coordinated Universal Time) with Vercel.
> My point is, the spec must live somewhere, even if you don’t write it down. The spec is what you want the software to be. It often exists only in your head or in conversations. You and your team and your business will always care what the spec says, and that’s never going to change. So you’re better off writing it down now! And I think that a plain old list of acceptance criteria is a good place to start. (That’s really all that `feature.yaml` is.)
Unlike you, I wish for the LLM to do as much of the work as possible -- but "as possible" is doing a lot of work in that sentence. I'm still trying to get clear on exactly where I am needed and where Opus and iterations will get there eventually.
It has really challenged me to get clearer on what a requirement is vs a constraint (e.g., "you don't get to reinvent the database schema, we're building part of a larger system"). And I still battle with when and how to specify UI behaviours: so much UI is implicit, and it seems quite daunting to have to specify so much to get it working. I have new respect for whoever wrote the undoubtedly bajillion tests for Flutter and other UI toolkits.
1. Specifications that live outside the code. We have a lot of code for which "what should this do?" is a subjective answer, because "what was this written to do?" is either oral legend or lost in time. As future Claude sessions add new features, this is how Claude can remember what was intentional in the existing code and what were accidents of implementation. And they're useful for documenters, support, etc.
2. Specifications that stay up to date as code is written. No spec survives first contact with the enemy (implementation in the real world). "Huh, there are TWO statuses for Missing orders, but we wrote this assuming just one. How do we display them? Which are we setting or is it configurable?" etc. Implementer finds things the specifier got wrong about reality, things the specifier missed that need to be specified/decided, and testing finds what they both missed.
I have a colleague working on saving architecture decisions, and his description of it feels like a higher-abstraction version of my saving and maintaining requirements.
I am also stealing the idea of talking to LLMs as if it's an email. So funny, we need to be joymaxxing a bit more I think :)
Also, I mainly pursue these tools so that I can have AI accelerate this process and broker an agreement after negotiating specs with the agent.
[1] https://www.lat.md/
If you're genuinely confused, and haven't tried Opus for coding, then it's not surprising you're confused!
It is also okay for you to just not like the idea of LLMs for coding (but say that!).
I have seen the same idea with processes, pipelines, lists, bullet points, jsons, yamls, trees, prioritization queues all for LLM context and instruction alignment. It's like the authors take the structure they are familiar with, and go 100% in on it until it provides value for them and then they think it's the best thing since sliced bread.
I would like, for once, to see some kind of exploration/abalation against other methods. Or even better, a tool that uses your data to figure out your personal bias and structure preference for writing specs, so that you can have a way of providing yourself value.
fyi language alone can’t define/describe requirements which is why UML existed.
You could deterministically process any UML diagram into a prose equivalent.
And in fact you couldn't do the other way around (any prose -> UML) because UML is less powerful than natural language and actually can't express everything that natural language can.