>"If AI makes every engineer 50% more productive, you don't get 50% more output. You get 50% more pull requests. 50% more documentation. 50% more design proposals. And someone, somewhere, still has to review all of it.
When two or three early adopters start generating more PRs than before, the team absorbs it. No big deal.
When everyone does it, review becomes the constraint.
The bottleneck doesn't vanish. It moves upstream, to the parts of the job that are irreducibly human: deciding what to build, defining "done," understanding the domain, making judgment calls about risk.
I've written about this pattern before:
the work didn't disappear, it moved.
What's new here is that it moved specifically into verification - and most teams haven't consciously staffed or structured for that yet.
[...]
The question isn't "how do we produce more code?" anymore. The question is "how do we verify more code?" And I don't think most teams have a real answer to that yet."
Excellent article!
It's a great question... how do we verify AI produced code? We could use AI to do that too, but then:
Who verifies the verifier?
Related:
Quis custodiet ipsos custodes? (Alternatively known as: "Who watches the watchmen?" / "Who oversees the overseers?" / "Who manages the managers?" / "Who guards the guardians?" / "Who reviews the reviewers?", etc., etc.):
Next headache? Any team that is actually reviewing code will notice this immediately.
rramadass 2 days ago [-]
Headache? More like a morass from which we will never get out of, unless we learn to use Formal Methods techniques to have AI generate code using a correct-by-construction approach. Classic ideas from Floyd/Hoare/Dijkstra/Meyer are the key techniques to use before moving on to heavyweight techniques like model-checking/theorem-proving etc.
When two or three early adopters start generating more PRs than before, the team absorbs it. No big deal.
When everyone does it, review becomes the constraint.
The bottleneck doesn't vanish. It moves upstream, to the parts of the job that are irreducibly human: deciding what to build, defining "done," understanding the domain, making judgment calls about risk.
I've written about this pattern before:
the work didn't disappear, it moved.
What's new here is that it moved specifically into verification - and most teams haven't consciously staffed or structured for that yet.
[...]
The question isn't "how do we produce more code?" anymore. The question is "how do we verify more code?" And I don't think most teams have a real answer to that yet."
Excellent article!
It's a great question... how do we verify AI produced code? We could use AI to do that too, but then:
Who verifies the verifier?
Related:
Quis custodiet ipsos custodes? (Alternatively known as: "Who watches the watchmen?" / "Who oversees the overseers?" / "Who manages the managers?" / "Who guards the guardians?" / "Who reviews the reviewers?", etc., etc.):
https://en.wikipedia.org/wiki/Quis_custodiet_ipsos_custodes%...
We need to build on this; Correctness-by-Construction: An Overview of the CorC Ecosystem - https://dl.acm.org/doi/10.1145/3591335.3591343
AI slop is AI slopping.