When I'm working on my own, I scrutinize every line of backend code and frequently send the LLM back for rework. Frontend code I keep an eye on just to make sure the LLM isn't getting stupid (which it occasionally does, even at the frontier), but mostly defer. The CSS I don't even look at.
This isn't too different from having employees. How much do I review other people's work? Depends on the employee, depends on how critical the work is.
xyzzy123 3 hours ago [-]
I think there's a paradox. Is it only "cognitive surrender" if you already know how to code? Not knowing all the details seems more like the default human condition, in the sense that most people don't know exactly where their water comes from or where their waste goes exactly. "Water comes from the tap, food comes from the store".
Most of the non-technical people I see using LLMs to create software are not engaging in "cognitive surrender" as far as I can tell.
They didn't read the code before and they don't now. However, asking questions and improving understanding of the system and architecture is actually faster and easier for them now than it used to be.
They're evaluating what they can see, i.e, judging results. They build things for their own use. They test them, iterate on them to meet their needs and try to achieve their vision. This is roughly the same process they followed when they used to outsource to an agency except the loop is much tighter.
It seems to work well, mainly hurts when the problems are invisible, so security, backups, breaking migrations, perf, maintenance over time (language / runtime updates, dep updates, server updates etc). All the stuff that hurts if you have never been responsible for the uptime of something before. Not knowing that software is never "done". But junior devs trip on those exact same things.
I agree with the tail of the article; tooling and processes that make that stuff more visible would close a lot of the gaps.
jon-wood 1 hours ago [-]
> It seems to work well, mainly hurts when the problems are invisible, so security, backups, breaking migrations, perf, maintenance over time (language / runtime updates, dep updates, server updates etc). All the stuff that hurts if you have never been responsible for the uptime of something before. Not knowing that software is never "done". But junior devs trip on those exact same things.
If we continue from your "water comes from the tap" example this situation feels like someone who has no experience of plumbing deciding they're going to replace their bathroom without bothering to find out how any of it works. They'll possibly end up with a bathroom that superficially works but they're also liable to find problems down the line with leaks, things not draining properly, or water pressure issues.
Some of the people writing software purely via LLMs are even turning this into a business, which I guess would be analogous to deciding you're going to run sewage treatment for a small city without understanding anything about how that works. It'll mainly hurt when the problems are invisible, so food poisoning, or inlets backing up and overflowing into a nearby river.
hannasanarion 58 minutes ago [-]
That analogy breaks down immediately because it's missing the critical novel component: the AI.
In the bathroom remodel analogy, the AI is the plumber that you hire to do the work for you. You tell it what you want, it figures out the details necessary to get it to you. If what you want isn't possible, or requires large changes, a good plumber will raise the issue to you and explain the constraints they are working under, ask for guidance, and offer recommendations.
Sometimes the AI does things in a weird or inefficient way, or neglects details that become important later, or misunderstands something important about the task resulting in an outcome other than what you wanted, but that's also the same as the plumber. This is why people usually want to be at home and keep an eye on the people working on their improvement projects or repairs, because they want to make sure that the guy they hired is replacing the broken faucet, not the functional but odd-looking one.
There is still value in knowing the underlying systems well enough to present a good design vision to the plumber up-front. It's still possible to have a competent executor produce a bad idea, which is how we get stuff like Groverhaus, the Doom Bathroom, and McMansion Hell, which are examples of "slop architecture" in exactly the same way as we are seeing "slop software".
xyzzy123 1 hours ago [-]
There were terrible software developers before LLMs. Honestly for a lot of people AI generated slop is a massive improvement over what they used to ship.
I agree it's pretty bloody cheeky to run an agency like that though. The only ways out I can see are licensing or real liability.
But let he who has never done dodgy home improvements cast the first stone.
Gormo 2 hours ago [-]
I think one of the potentially advantageous things about AI is that it acts as a force multiplier for people who already have expertise in a given discipline, while simultaneously raising the floor for everyone else.
People looking for self-contained, spot solutions to localized problems will have much more capacity to solve them locally, while people with in-depth expertise will be able to build systems that are even more reliable, performant, and scalable than what they were building before.
So we'll get the benefits of disintermediation for the basics, and much greater capacity to deal with effectively with complexity for the not-so-basics.
fatnoah 3 hours ago [-]
> However, asking questions and improving understanding of the system and architecture is actually faster and easier for them now than it used to be.
One of the things I've built in my current role is tooling to help understand current architecture, and to resolve the differences between things as-documented, as-implemented, and as-actually-behaving. It's used for a few things, including updating documentation, but it's been most impactful when onboarding new engineers. They can simply ask questions about how things work and why, and get explanations and deep dives wherever they need it.
gspr 3 hours ago [-]
> Is it only "cognitive surrender" if you already know how to code?
Is the article claiming that?
> Not knowing all the details seems more like the default human condition, in the sense that most people don't know exactly where their water comes from or where their waste goes exactly. "Water comes from the tap, food comes from the store".
Sure. The ability to specialize, and on the flip-side not have to care about a lot of things, remains one of the biggest driving forces of human civilization. But in this article, I don't think the person worried about cognitive surrender is the "most people" in relation to food or water. They're the farmer or the water infrastructure engineer! They absolutely for sure do need to know those things. And all around us we see a lure of systems that invite them to surrender.
> Most of the non-technical people I see using LLMs to create software are not engaging in "cognitive surrender" as far as I can tell.
Maybe not. But the might be cognitively surrendering when they use the LLM to summarize the day's news or write a message to a friend, or even plan dinner for the week. I'd say they are. I agree that it's not clear-cut.
> They're evaluating what they can see, i.e, judging results. They build things for their own use. They test them, iterate on them to meet their needs and try to achieve their vision.
No. They have things built, then they test them. This is different. It might be OK, but it's not the same as building.
> This is roughly the same process they followed when they used to outsource to an agency except the loop is much tighter.
Yes. But they wouldn't usually outsource summarizing a letter from grandma, or the political agenda of a candidate/party in an upcoming election. That's surrender.
> It seems to work well, mainly hurts when the problems are invisible, so security, backups, breaking migrations, perf, maintenance over time (language / runtime updates, dep updates, lib updates, server updates etc).
… or ones own abilities deteriorate.
> Tooling and processes that make that stuff more visible would close a lot of the gaps.
Not the problem of one's own skills though.
xyzzy123 3 hours ago [-]
> I don't think the person worried about cognitive surrender is the "most people" in relation to food or water. They're the farmer or the water infrastructure engineer!
Fair. Maybe I "cognitively surrendered" a long time ago just as a consequence of working in large teams :(
I hit the breaking point of not being able to read all the code well before LLMs came along. Also people would leave and the documentation for their subsystems would be wrong or incomplete, so there were always mysteries to chase down (during an incident, naturally) even in code written by hand.
Gets to the point, you don't exactly read the MRs, you're mainly checking their approach against what you talked about earlier and making sure it doesn't look way more gnarly than it's supposed to.
Working with agents doesn't feel that different.
> No. They have things built, then they test them. This is different. It might be OK, but it's not the same as building.
It does bug me sometimes when people say "yeah we built the deck..." and they got a contractor to do it. But then we do say things like "Nolan made the film" even though he didn't hold a camera or build the sets. To me, one-shotting doesn't feel like building, but engaging in a loop and applying control to get the outcome you want, does. Regardless of the exact toolset.
I don't personally see super hard lines between using a compiler, a 4GL, a sql query, stitching together 10 libraries and a framework I didn't write, using an LLM or even directing a team.
> Not the problem of one's own skills though.
I read somewhere once we can't weld Saturn V engines anymore, or at least not economically. We would also probably have a lot of trouble finding people who could do all the engine calcs the way they did. It doesn't necessarily mean that rocket progress is stalled.
> Yes. But they wouldn't usually outsource summarizing a letter from grandma, or the political agenda of a candidate/party in an upcoming election. That's surrender.
Agree 100% on the letter from Grandma.
gspr 2 hours ago [-]
> But then we do say things like "Nolan made the film" even though he didn't hold a camera or build the sets.
I do say that, too. But I'm not in the film industry. My understanding is that those who are, don't use words as crude as "made" to describe a role in the making of a film. I believe they say that he wrote, directed or produced a film.
We are not talking about some other field of expertise here, where we are outsiders – the production of food, the supply of water, or the making of movies – we're talking about our own field of expertise.
jon-wood 1 hours ago [-]
Somewhat off topic now, but Nolan is fairly famous for being intricately involved with every aspect of the films he directs. He's constantly behind a camera checking out what shots are going to look like, pushing the limits of where you can put an IMAX camera, and coordinating with set builders to do ridiculous stuff like building a reproduction harbour jetty.
customguy 1 hours ago [-]
> They didn't read the code before and they don't now.
They didn't read code before, they just wrote it? That makes no sense.
zmmmmm 7 hours ago [-]
The tendency to over-engineer and over-build is one of the main challenges I'm seeing. When it comes for free, why not build a class hierarchy of 8 classes each with a 200 line unit test? Something I might have previously thought really hard to minimalise down to 30 lines is now bloated out to hundreds or even thousands.
Previously, pretty much everyone all agreed on YAGNI and KISS because code was so expensive to create and more expensive to maintain. Now the code generation is "free" - but at a design level, YAGNI and KISS do still matter. But they are enforced now only by sheer will power. You have to actively restrain the code from being created if you don't want it. Perfectly good code - I can't even fault it. I just know, if it doesn't need to exist yet, it's a mistake to create it earlier than needed.
lioeters 7 hours ago [-]
Like that quote in Le Petit Prince, real intelligence is knowing what not to build, what can be eliminated until only what's necessary is left.
> La perfection est atteinte non pas lorsqu'il n'y a plus rien à ajouter, mais lorsqu'il n'y a plus rien à retirer.
> Perfection is attained not when there is nothing more to add, but when there is nothing more to remove.
n4r9 6 hours ago [-]
That quotation is in fact from "Wind, Sand, and Stars", the memoir of the same author.
On the same theme:
> the supreme goal of all theory is to make the irreducible basic elements as simple and as few as possible without having to surrender the adequate representation of a single datum of experience
- Einstein
> I would have written a shorter letter, but did not have the time
- Pascal
> Simplicity is a great virtue but requires hard work to achieve and education to appreciate, and, to make matters worse, complexity sells better.
- Dijkstra.
6 hours ago [-]
MSFT_Edging 2 hours ago [-]
This is so true. I've been seeing LLM generated code that does way too much, yet still cant accomplish the stated goals. "maximally flexible" plugin systems, that are too complicated to reasonably write a plugin for quickly, when one could have just built in the needed functionality in a few hundred lines.
When endless bootstrapping and framing is free, you end up generating the most beautiful scaffold while ignoring the building behind the scaffold.
GoblinSlayer 6 hours ago [-]
I don't think anybody tries into KISS, most software is bloated and LLMs copy it, they fundamentally can't think out of the box.
AdamN 4 hours ago [-]
That's always been my concern - there's just so much garbage code out there and very few signals about which of it is well written and which is not ... so LLMs will just regurgitate the good enough code that it's been trained on.
Now would be the ideal time for everybody to give it negative feedback when bad code is returned but I'm afraid not enough people care and we'll just settle with what works just like we always have.
pessimizer 27 minutes ago [-]
I don't think that's it. The context that LLMs work with is simply too small, and can't get bigger. Harnesses and the models themselves are bolting structure over it manually; real people are inventing design machines that are a lot more deterministic than they look, and LLMs are just filling them in.
The design machines aren't good enough yet; and their progress has little relationship to LLMs at all. Designing computer programs and the proper level of abstraction for the easiest maintenance, flexibility and debugging is simply an open problem. There's nothing to train LLMs on that is smaller than the size of an entire program, and its history of code changes.
Get a few of those, figure out a way to analyze and reduce the patterns and lessons in them, and train on that maybe. Until then, there's no way for an LLM to figure out what lacking a particular abstraction is going to end up costing.
sfn42 5 hours ago [-]
At the end of the day your work is your work. If you tell Claude or whatever to do a thing it will do that thing however you tell it to do the thing.
You can choose to ignore the fine details and just tell it what to build, or you can spend more time and effort and tell it how to do things, you can ask it to create a plan, review the plan and ask for changes etc, you can review the implementation and ask for changes there. You can simply tell it you don't like how a feature is implemented and ask it for suggestions on how to improve it, include what you don't like about it so it has direction.
It's all about the effort you put in. LLMs are able to write clean beautiful and efficient code, they just can't necessarily do exactly what you want on the first try without any direction. That's why you should figure out what you want, ask for what you want and make sure it's actually building what you want.
Fr0styMatt88 4 hours ago [-]
I think this sorta cuts both ways too. Sometimes AI code has shown me that no, I in fact don't need that big class hierarchy and speculative plumbing that I thought I might need to get the job done.
Keeping the volume of code down is definitely a challenge though.
7 hours ago [-]
orbis-tertius 5 hours ago [-]
[dead]
haukebri 5 hours ago [-]
[dead]
quibono 4 hours ago [-]
I have been thinking about this for a while. I'm not a Luddite, I can see the amazing value that modern AI brings, and how useful it can be.
At the same time I can't help but wonder that this is almost too much for us to cope, the LLMs are _too_ easy to use. I think we're very good at pretending there isn't a problem here but there is definitely going to be some cognitive decline happening.
We want to pretend like reading over the output of an LLM is nearly the same as us making those changes ourselves but I think there's a lot of value in those boring and manual steps. And most people (including me) will be too lazy to not eventually offload most things to the LLMs.
In a way this is similar to how watching a lecture tends to give one an "impression" of understanding that doesn't correlate with reality. We need hands-on work and some course work to get our monkey brains engaged.
I can only imagine what effects this is going to have in education.
glimshe 3 hours ago [-]
We've surrendered many uses of our cognitive power throughout the centuries while replacing by others.
Our cognition didn't evolve to produce arguments for online discussions, but for making crude tools and hunting. As we stopped applying our intelligence towards these uses as a result of our tools, we simply shifted it to new problems that were unthinkable previously.
A caveman probably wouldn't care too much about algebra, a topic most children learn today.
We should stop thinking about the problems AI trivialized and, under the assumption that AI won't fully replace human capability (a key assumption), start thinking about the problems that were previously out of reach. Education would then start focusing on that.
lkey 1 hours ago [-]
This is true, but the nature of declining cognition is to not even notice what used to be possible.
Taking clothing for example, there's more available now than ever. It's disposable at great cost to the environment and the workers that produce it, but very little to you.
Yet, I bet you cannot evaluate garments, construction, or fabric quality. You may not know how to wash woolens and linens. You may have only worn smelly plastic workout clothing and plastic suits for formal events.
You may never have worn a piece of clothing that actually fit your body.
You may not realize that you can alter any clothing, that most clothing was intended to be altered once upon a time. Or that older garments were both more expensive and higher quality, because the expectation was the most important items would need to last for a decade.
The Luddite's central objections about automation was about declining working conditions (objecting to child labour, for instance), bad pay, and declining quality. They used the 'obnoxious' machines they broke. They saw what was being lost and demanded better (without demanding no machines at all).
Later, the rise of unionization brought about many of the changes they'd asked for decades before.
I hope this current technologic movement doesn't end where fast fashion has, with influencers buying a new outfit [repo] every day for a year. A vortex of undifferentiated slop for its own sake.
nbaksalyar 39 minutes ago [-]
> The Luddite's central objections about automation was about declining working conditions
I can also recommend the book "Blood in the Machine" by Brian Merchant [0][1] which is precisely about this.
Superhuman intelligence is much easier for an LLM to achieve if humans are made dumber first.
dspillett 4 hours ago [-]
I've been working on that for years.
It'll take a particularly dumb and lazy AI to replace me.
atzmzt 5 hours ago [-]
Quote of the day award goes to you
tra3 11 hours ago [-]
Great post. I particularly like the 4 column pr view idea. I’ve built something similar to review speckit specs. The scaffolding to ensure that code actually works makes sense too.
There’s one thing that’s bothering me though — all of these have us consuming what LLMs produce. I learned and internalized how things worked by building systems. At least within the constraints of the abstractions I was using. This makes me think of TikTok and shorts. Everybody is just watching. Maybe that’s not charitable though. Maybe the abstraction has now changed from an “os/gui/event loop” to “llm” and the rest is just accidental complexity.
gus_massa 1 days ago [-]
>> A more serious irony is that the automatic control system has been put in place because it can do the job better than the operator, yet the operator is being asked to monitor that it is working effectively.
Automatic control is good for standard case, and the expected errors. For weird cases, for now humans are better to detect them and press the big red button.
esikich 10 hours ago [-]
It's funny that I'm reading this while rewatching Chernobyl. Never underestimate the ability of bureaucracy to completely fuck things up.
mitthrowaway2 10 hours ago [-]
The computer doesn't know we're running a test!
_jackdk_ 7 hours ago [-]
Ironically, these models have demonstrated awareness that they are in tests and chosen to act in ways that maximise their chances of deployment.
oliculipolicula 10 hours ago [-]
Jumping into article, I assumed that the author was going to describe how he enjoys, by drawing it out as far as he can,
the process of surrendering to the machine god..
(Not the machina diabol, as it sadly turned out to be, though some BDSM zealots might disagree.. the things that bureaucracy does to us are mild in comparison? )
But.. we do monitor whether our prayers are effective .. right? Is that a smitable offense?
After that disappointment, I too was "curious": was the author going to talk about "curiosity", that magic virtue that we hackers believe, lifts design into the divine (but more often than not kills the pussy)
internet_points 7 hours ago [-]
The better an automatic control is, the fewer mistakes it makes, the fewer times the human needs to react, the more lax and less vigilant the human becomes -- and less skilled when they need to step in. Source: https://imgur.com/a/Vj2TaDz
cowlevel 13 hours ago [-]
Except experience shows they aren't. Experience shows that when humans are just pressing "ok" 99% of the time they'll press it 100% of the time and completely miss the other cases.
mixedbit 8 hours ago [-]
Maybe things will evolve into projects having a well-defined, hard boundary between agents generated code and human generated code. Where agents generated code is strictly separated from human engineered code and and can call only a strictly defined set of APIs designed by humans.
Think of platforms like Shopify. Shopify allows huge number of external users to write code customizing their web shops. But Shopify engineers do not need to review and understand the users' code, because there is a boundary which limits what the external code can do. The external code can be A Big Ball of Mud without making the whole platform a A Big Ball of Mud.
smodo 7 hours ago [-]
I think so too. And it is how codebases are managed implicitly anyhow. You don’t let anyone PR against core functionality, you keep that API under lock and key.
But UI on top of a good API should be unable to cause problems; the API must prevent invalid state etc. So letting LLM’s write UI code for CRUD-apps (which is most of them…) seems fine to me.
Gareth321 5 hours ago [-]
We are leaning into a kind of federated model. The core applications and vibe-coded satellites. The core apps have stricter requirements and carefully considered API guardrails. Go ham on the AI apps. If you break something, the scope of destruction is much more limited.
That said, the core apps are still full of AI code. I don't see a reality where hand-written code can or will be curated and sequestered. It's unenforceable, too slow, and too expensive.
virgilp 6 hours ago [-]
Well. One of the things that's critical in all that analogy is that the BigBallOfMud that one user creates on Shopify cannot impact the other users - that's why Shopify can keep the platform from becoming ABigBallofMud.
In order to be able to enforce this hard boundary, it means that one "agent generated code" in one place, not only "can call a strictly defined set of APIs" - but, more importantly, it implements an isolated piece of functionality. Think something like: I let the AI write the dashboards, so the dashboards may be slow, crappy, incorrect, whatever - but it doesn't impact the rest of the product. However, I can't let AI write "metrics collection" because that not only impacts dashboard, but potentially impacts core product efficiency too (memory consumption, cpu, IO, etc.)
zaphirplane 6 hours ago [-]
100% can guarantee the balls of mud be connected making giant balls of mud
krapp 4 hours ago [-]
>Maybe things will evolve into projects having a well-defined, hard boundary between agents generated code and human generated code.
The only way things will evolve is towards maximizing AI generated code and minimizing human involvement because that is the most efficient mode for getting product to market. Any trend to the contrary is temporary, paperclip maximizer will maximize. Any code that humans touch directly will be considered implicitly too risky to run in any production environment.
We'll just have to hope things don't break too much and when they do, we'll just have to accept it because no human will be able to (or allowed to, or even trusted to) fix it.
ashishmax31 10 hours ago [-]
i sometimes wonder how all carpenters or music producers in the world would react if tomorrow a magic box arrives where they type-in/dictate what they want and voila... out comes a table/composition.. Will the worlds music producers and carpenters just "accept" this new magic genie? I would hope they wont.
Yet here we are as software engineers...ready to surrender our craft to mere "prompts" with reckless abandon.
rcxdude 4 hours ago [-]
I see a bit of a split in people in creative roles of all kinds (and this is something I had noted before LLMs made the contrast more stark). some people get into it because they like the process of making the thing that they are making (in whole or in part). Others get into it because they have a thing that they want to exist in the world but no-one else has made it (I would also split this into people who just want the thing, or people who want to make something that crosses many domains and so want custom things that will fit into the whole). The former group tend to react with revulsion to automation, because it removes the part they like (though it may depend on what part is automated: most people will find some part of their work necessary but tedious). The latter are usually much more willing to embrace it, because it's the end result that they care about moreso than the process.
smodo 7 hours ago [-]
Have you seen how people use Fruity Loops nowadays? We’re already halfway there if you compare that to using an AKAI S9000 or MPC2000. And then go back to actually playing an instrument. It’s a gradient and we’re already pretty far along on it :)
thefoyer 9 hours ago [-]
Of course they wouldn't, most people who consider themselves engineers aren't engineers. Just look at the job market right now.
Betelbuddy 8 hours ago [-]
There is no such thing as a Software Engineer.
Its just as much as an ego trip...as the Principal Engineers that roam around here, after they survived three layoffs and outputted front end code for six years straight at a FAANG...
Prove me wrong...
gauntr 6 hours ago [-]
"Margaret Hamilton popularized and coined the term 'software engineering' to give the discipline the same legitimacy as hardware engineering".
But oh well, who was she, just the engineer who wrote the software for the Apollo 11. Probably she had no idea what she was talking about.
/s
lioeters 5 hours ago [-]
That's new to me, I didn't know she coined the term. She's the first true "software engineer" then, I suppose. I remember her from that historic photo where she's standing next to a pile of code printed out, a stack talled than her.
Not to be confused with Margaret Hamilton the actress who played the Wicked Witch of the West in the film The Wizard of Oz.
TonyStr 7 hours ago [-]
It turns out that most people are fine with most of their things being mass-produced cheap stuff from IKEA, Amazon, Temu, etc. - if that means that the cost is significantly lower. The functional benefit of having a door is huge, while the aesthetic benefit of having a beautiful, hand-crafted door is much less important to most people.
Code is not like craftsmanship anyway. The end-user doesn't see any difference whether I use microservices or hexagonal design or react or htmx or how I structure the code or which language I use. All they see is the design and the stability. Since the average developer was terrible at design anyway, the average design output from LLMs is actually an improvement in most cases, even if it always has that generic AI generated look. Stability... was not good before AI either. We have become accustomed to SAP taking 10 seconds to load each component on the page, or Atlassian spewing error messages in your face when you click submit on a ticket. It seems that the aspects that are most valued by our end users are QA and UX, not the craftsmanship of our code.
usrme 10 hours ago [-]
For musicians there already exists something called Suno: https://suno.com/
tuvix 9 hours ago [-]
That’s not music
imp0cat 8 hours ago [-]
And computer-generated code is not code? ;)
optimalsolver 6 hours ago [-]
That's up to consumers to decide.
canelonesdeverd 3 hours ago [-]
Consumers have, overall, rejected AI "art".
But of course, every time consumers voice their preferences is actually "a loud minority" or an "echo chamber" :)
MonkeyClub 5 hours ago [-]
If the consumers had criticality they’d be people, not consumers, and we wouldn’t have marketing the way we do now.
cowboylowrez 4 hours ago [-]
Suno probably puts out better music than most folks who take up music. I mean statistically it has to, right? So once we go with "suno doesn't make music" then we also have to progress to "99 percent of human musicians don't make music", and all us amateur hacks will start objecting lol
fwip 2 hours ago [-]
Only in the same narrow way that a camera makes better art than a painter. Or the idea that a generic landscape print is better than a Rothko, because it looks more like other paintings that you like to look at.
cowboylowrez 50 minutes ago [-]
The camera just records reality, its by definition sort of neutral artistically speaking, its what it records that matters. Its quite different from this AI music thing, and I would consider it a very difficult task to discuss topics with someone trying to pretend it isn't.
And if you look a the dates, the rate at which they are doing it is actually increasing...
hansonkd 10 hours ago [-]
I mean more or less to an extent if you expand by what you mean by "out comes". With Amazon, a carpenter can type in any furniture he wants and have it arrive in a matter of hours. Almost equally as magically as well as disruptive. People still use carpenters to make tables, but for the majority, the ease of same-day delivery and cheaper prices are worth it for a sacrifice in personalization and character.
ashishmax31 9 hours ago [-]
>With Amazon, a carpenter can type in any furniture he wants and have it arrive in a matter of hours.
Sure but that doesnt require the person to be a carpenter though right? Anybody could do that.
bigstrat2003 9 hours ago [-]
> Yet here we are as software engineers...ready to surrender our craft to mere "prompts" with reckless abandon.
It's worth remembering that, contrary to what HN acts like, software engineers are not uniformly accepting of LLMs. They are still quite controversial in the industry. Now management, on the other hand, is completely sold on the supposed benefits of LLMs. So many engineers feel forced to use them, but it's not enthusiasm on their part which drives them to that.
ashishmax31 4 hours ago [-]
My experience has been the opposite, the HN crowd seems more sketical of vibe-coding and not reading a single line of code.
The twitter(x.com) and linkedin crowd has drank the cool-aid and are now swimming in it; Wild posts from FAANG engineers and managers saying they or their engineers dont read any of the code the LLM generates.
hardbass 7 hours ago [-]
This but for building houses would be the dream.
on_the_train 7 hours ago [-]
Music is already mostly a solved problem unfortunately. Try giving opus a few midi files of your favorite song. It'll write something new in that style with a competence humans would need several years of experience for.
prmph 4 hours ago [-]
Music is an art, not something needing to be "solved".
It' like saying compassion is a solved problem: just tell your problems to the LLM and have it gush out sympathy.
on_the_train 4 hours ago [-]
Might be an art for a few. It's mostly a commercial problem to be solved.
2 hours ago [-]
ontouchstart 2 hours ago [-]
I have been doing this for almost a year now by limiting myself to less powerful local models via llama.cpp and pi.
Fighting over engineering is hard. But it is also one of the opportunities for us humans in the era of abundance.
I did learn a lot about the ecosystem and myself from this experience.
lioeters 12 hours ago [-]
"Yup, looks good to me!" said the human reviewing thousands of lines of generated code everyday.
This has become common now, and I've seen it ruin large projects until nobody understands the codebase full of verbose nonsensical code and comments. It becomes impossible to further develop the project without AI. Contributors not using AI for any reason are all leaving because no one enjoys working with mindless robots (and humans) as colleagues. It's demeaning and devaluing their labor and humanity. They're not getting "left behind", they are leaving the madness of crowds behind.
> ..the possibility of runaway dynamics: once a critical threshold is crossed, small increases in adoption can trigger rapid population-level shifts toward *persistent dependence, with abrupt losses in cognitive competence*.
We can all see it happening. As this technology permeates society, the challenge to maintain control, coherence, and quality will become greater and ever more serious - for example, AI in healthcare, military, management and governance.
cowlevel 11 hours ago [-]
Even if the human's judgement is correct in every LGTM, there's a bias towards merging. Each merge that adds a feature but degrades understandability 1% will be merged. It won't be held back until it improves it by 5% instead. After 100 merges you have degraded understandability 100% even though nobody made an error.
californical 10 hours ago [-]
You’re completely right that nobody is stopping a PR that just ever so slightly decreases understandability of something. It may not even be obvious to the reviewer that it’s slightly less understandable overall… Doubly so if they’re just skimming due to the volume, and tripley so if they’re having agents do the most of the review, which lets the human reviewer’s guard down.
But before LLMs, it didn’t matter because on a good team you’d have a balance where maybe some PRs added complexity and some simplified it. Sometimes people would include a small related cleanup or refactor in a PR for code around what they were doing if they noticed complexity. That has become far less common, I think.
I know it’s not your point, but a 1% decrease 100 times is 0.99^100 = 0.366 which is a 63.4% decrease in understandability. Just throwing that out there
lioeters 10 hours ago [-]
Glad to see the math corrected, the sloppy calculation was bothering me also.
unpleasurable 10 hours ago [-]
I think it's about 63%
sowhat1 8 hours ago [-]
So exactly like before AI? ;)
Enterprise software has always been crap.
Buttons840 10 hours ago [-]
I hope there's still value to be found in a bunch of developers standing around a whiteboard and talking, and I think this scenario is where their ability and understanding really shows these days.
If we don't value that though--if we don't value understanding the system and just thinking through problems as a group--then let's stop pretending and just not do code reviews. Probably nobody cares and just types "LGTM" after having their own LLM do a spot check to cover their ass.
It's really easy to push developers to stop caring. If management says "go fast fast fast, use those LLMs!", then developers can't keep up, get frustrated, stop understanding the system, and at that point they're just there to collect a paycheck before society collapses. Or, if management is making dumb decisions all the time, same thing.
Hopefully as implementing ideas takes less and less time and resources, smart people who actually care will start building things with other smart people who care.
drakonka 8 hours ago [-]
I've had someone approve my PRs within a minute of me opening them, and likewise expect immediate approvals on his own PRs. When asked questions about parts of the code sometimes the answer would be along the lines of "Claude did it this way", or verbose AI-generated responses to the question.
nurettin 10 hours ago [-]
To be fair, models are more accurate than how they were earlier this year. In march I was still frustrated and was writing my own event loops because the model simply wasn't built for it. By January I was still hand checking statistical outputs to make sure it wasn't making any errors, and it was. After august it was pretty much over. There were no stats errors, code generated was shit but it could do what is asked. Instruction following is still a problem, and I sometimes catch silly things in the scrolling diffs, but it has been much easier since then.
hypfer 9 hours ago [-]
The most difficult part of LLMs and agents seems to be - surprisingly - humans.
Specifically, humans and their dysfunctions. Their fragility. Their ego tied to stuff.
This is an excellent post/reflection on the topic of solo LLM use.
The only gap I see is that it assumes that one works on something alone and not in a team. Though, that might be a different essay in itself.
Edit: Turns out that footnote 3 actually mentions that already. Nice. Good post
quibono 3 hours ago [-]
On the ego-effect, I sometimes feel like cataloguing some PR-review quirks that are mostly caused by humans being humans:
1. Smaller PRs (that can be easily read) tend to receive more comments and nitpicks than a large change since it's easier for us to have all the PR context in our heads at the same time.
2. Most people are ego-tied to their code even if they pretend that they are
impartial
3. Social dynamics around merging and reviewing code between senior/junior and older/newer employees
4. probably many more
wanderingmind 8 hours ago [-]
Cognitive surrender is intellectually convenient. As humans, its instinctive for us to chose convenience: path of least resistance. The tyranny of convenience is what drives most economic activity today, especially the gig economy[1]. Unless there are structural frameworks put in place to resist convenience (exams in schools), nature will always entice to chose the path of least resistance
> Once you have used a washing machine, laundering clothes by hand seems irrational, even if it might be cheaper.
Washing clothes by machine is orders of magnitude cheaper than by hand when you factor in the value of your time.
> After you have experienced streaming television, waiting to see a show at a prescribed hour seems silly, even a little undignified.
Undignified? What? Completely incoherent word slop. Good to see LLMs haven't completely taken that domain for themselves, I guess - a little hand-crafted slop now and then is a treat, I guess? Anyways, they're wrong. People still tune in to watch scheduled live events all the time.
> To resist convenience — not to own a cellphone, not to use Google
Neither of these things are a matter of convenience. A cellphone is at this point literally required to participate in a modern economy. Google is a tool that provides you with globally-retrieved information that you otherwise literally could not find on your own or without another similar tool.
I don't even disagree with the premise that convenience is a driving factor of our lives, but the justifications and conclusions drawn by this article are beyond stupid. Here's my view of it: the desire for convenience is what pushes us onwards as a species. Hunting for food is inconvenient, so we invented agriculture. Going down to the river or well for water is inconvenient, so we invented indoor plumbing. Repeat ad infinitum until we have modern civilization. There is one part the pessimistic view overlooks; that to achieve maximum convenience long-term requires more effort in the short-term. Developing the skill of programming is inconvenient in the short-term but convenient in the long-term because of the possibilities this opens up to you. This remains true even in the current LLM era, and it will remain true for as long as we are not replaced wholesale by LLMs.
rcxdude 4 hours ago [-]
>Washing clothes by machine is orders of magnitude cheaper than by hand when you factor in the value of your time.
I'll also note that it is less resource intensive overall. Handwashing dishes and clothes will use more water and more energy to heat that water compared to the machines that do the same job (people often assume the opposite).
quibono 3 hours ago [-]
I don't like the op-ed but I would agree with the main point or what I understand as the main point, even if the argumentation is weak).
It takes a mix of willpower / routine / systems / laws to keep people in check, otherwise we predictably go back to the path of least resistance.
7 hours ago [-]
gosen 7 hours ago [-]
What an amazing article! It's really hard to find the balance between "ship faster, fix later because AI will help" and "efficient thoughtful engineering" these days. The world is kind of pushing everybody to work faster because "with AI you're more efficient" and it's also pushing to surrender and just accept it from time to time. Maybe that's occasional thoughtful surrender mentioned in the article, or it's just lack of energy to resist to it at that point.
The article is inspiring though, I have more motivation to work on my thinking process and my engineering setup (and vibe limit haha) to be more thoughtful, thank you.
1. LLM usage, in particular unsupervised, tends to build up "cognitive debt"
2. It tries to characterize the concept, including to propose a score for it
3. Notes that the level at which one can let the agents loose (and allow some CD) differs quite a lot between core business domains and more peripheral ones
kombookcha 4 hours ago [-]
Cognitive debt bad.
tonyedgecombe 5 hours ago [-]
Ask your LLM.
skybrian 9 hours ago [-]
Making a quick prototype is probably the best way to find out if an idea is worth keeping. Vibe-coding it and then throwing it away seems like a good way to go?
knlb 9 hours ago [-]
Yeah, definitely. Also at times I've vibecoded successive iterations with better designs after seeing where the first ones collapsed -- the agents do a good job with a template and need much less prompting for the successive versions.
totetsu 11 hours ago [-]
I am often thinking recently of the littel loop diagrams from the "The Capability Trap" paper. We need to keep a loop reinvesting into our own Human capability in anything we're doing.
rambambram 4 hours ago [-]
At least "cognitive surrender" sounds catchier and more academic than "stupid on purpose".
ahartmetz 4 hours ago [-]
It is by accident though - at least from the user's POV. From the AI provider's POV though...
bheadmaster 7 hours ago [-]
I know I've cognitively surrendered when my first instinct is to ask AI to summarize the article.
I am losing more of what little focus I've had every day.
stego-tech 3 hours ago [-]
One aspect to resistance that is oft overlooked is knowing when to fail on purpose so you can succeed on a bigger issue elsewhere.
As an example, I’m juggling multiple projects at work and home. I have a finite amount of time, and access to AI. One camp would tell me to use AI for all of it to move faster, but that increases the risk of Cognitive Surrender. The other camp would tell me to do it all by hand, but I don’t have the time available for that; I must pick and choose.
So, I choose to cognitively surrender on the SEO task for our website because that’s a set of skills I don’t want to acquire and are better handled by AI anyhow. This leaves me more time to focus on learning more about Apple’s configuration profiles and improving our MDM suite, which are valuable skills that will still pay out in an AI-first world.
You have to pick and choose where to fight, otherwise you’ll expend all your energy on losses.
bbeonx 7 hours ago [-]
Maybe it's because I'm not working in the workforce (grad student), but I find my best workflow is to fight and argue with my agents throughout the entire process. Often times, since the agent is such a sycophant, I end up arguing with myself...I actually find this useful because it gets me to think about the other side of a problem and poke holes in it. This is not fast, but it is effective.
caporaltito 6 hours ago [-]
Excellent article.
ur-whale 7 hours ago [-]
> Resisting Cognitive Surrender
The economics of the whole affair unfortunately point squarely in the opposite direction.
lo_zamoyski 1 hours ago [-]
The primary reason you shouldn't surrender cognitively is that the truth matters. That's it.
Of course, when you're working on a system of high complexity - and one that is constantly changing to boot - with all sorts of historical reasons and what not that inform the why and so forth, you can't reasonably be expected to understand everything. This is what bracketing is for. When I acknowledge "this is the way X works", it doesn't mean I have to accept the proposition "this is the right way/best way to do X" or "X must be this way". A teacher or mentor may say it is, but in a theoretical context, unless it has been justified, you cannot claim that you know that what has been said is true. However, you can accept it as a provisional working hypothesis. And practically, it may work out and perhaps experience will lend some feeling of substance to the hypothesis, but you needn't make it doctrine.
Empirical science often has the same spirit. Perhaps the model isn't a true account of reality, but it puts handles on reality so that it can be instrumentalized.
There is an occupational danger here. If we focus exclusively on the instrumental, if we reduce all of human attitudes toward the world to use and utility, we not only vulgarize ourselves and our orientation toward the world and even others, but we risk trampling the truth and the good underfoot. We risk developing an indifference to the truth, because all that will matter is some measure of effectiveness. This is what one might call the textbook[0] definition of bullshit.
I simply don't care. As a student I thought that the mythical production code is perfect and error-free, but in reality 99% of systems are a hot potato game sprinkled with politics where you don't want the grenade to explode in your hands.
I mean, if the management doesn't see value in decent engineering, then who am I to butt myself into the conversation?
novaonilands 4 hours ago [-]
[flagged]
sjdh76 11 hours ago [-]
[dead]
jdw64 12 hours ago [-]
[dead]
augment_me 11 hours ago [-]
This is copium that is gone with the wind when the models perform up to the level of standard that the user requires. I'll give the author 6 months before they feel like its okay to just send it.
Karpathy had the same trajectory. Primeagean had the same trajectory. I think in the end everyone except for Jonathan Blow will have the same trajectory
knlb 9 hours ago [-]
Writing is hard, but what I was trying to say is that it's okay to just send it as long as you're doing it consciously and intentionally (and I do that frequently enough); but if it's not an explicit choice that can be problematic.
augment_me 9 hours ago [-]
I see, that makes sense, I think that applies to most things from eating food to watching media, etc.
However it takes more mental energy and people don't like that
willtemperley 11 hours ago [-]
If you don't understand the code, how can you know if it's good enough?
I guess you could ask the model. "Yes well done, you're a visionary software engineer".
TonyStr 7 hours ago [-]
I've been experimenting with vibe-coding some games. One of the games started performing very badly in a particular menu, and I began suspecting it's because of a shader that may be redundantly computed way too much on too many large surfaces. I began writing my suspicion to Claude, then stopped and decided to just describe the problem and see if it would come to the same conclusion as me. A few minutes later, it had profiled the game, found what contributed most to the poor performance, fixed the rendering and made the game performant again. I didn't have to inspect the code to do any of that. The current models (paired with coding harnesses) love to test and validate their results by running the games, taking screenshots, singling out individual parts, including logs to reproduce errors or other issues, and they love to create tests. The code is good enough in the same sense that the code I used to write was good enough if it did what we wanted. My non-technical bosses didn't have to inspect my code for that.
krilcebre 5 hours ago [-]
In my experience, it's usually good for finding out the bugs, but I take the fixes it provides with a grain of salt. There were numerous times where the fix could have lead to some more serious bugs because of some business logic special case scenarios that it didn't understand.
TonyStr 4 hours ago [-]
Yeah, I'm seeing this at work too. It's capable of adjusting behavior to meet some narrow success criteria, but it can easily introduce bigger problems in the process in large coupled code bases, which it doesn't catch on its own. Automated tests help a bit, but I find I still have to steer it very strongly at work.
willtemperley 6 hours ago [-]
It sounds like you understood what was happening here and used a tool effectively, nice.
That's very different from what a lot of vibe-coders do, like wholesale cloning and relicensing projects. This is happening in the GIS space, and these vibe-clone projects slightly hilariously inherit all the technical debt they didn't understand.
Nothing wrong with using LLMs, but I agree with Andrej Karpathy: "you can outsource your thinking, but not your understanding".
TonyStr 4 hours ago [-]
I think that quote is very apt. I experimented with making realistic video game grass (inspired by Ghost of Tsushima), and let it take the reins. But its solution became very contrived and hard to steer in the direction I wanted without me learning the specifics of how grass rendering should work. Still, it seems that I can limit my understanding to the higher levels of abstractions in most cases now. For example, I made it clump grass together according to a voronoi diagram, yet I have no idea how the voronoi algorithm works.
augment_me 9 hours ago [-]
Who cares if it's good enough? Send the product, if people use it, it's good enough. If not, let users debut it and give "feedback", problem solved. You then advocate to these users, make them feel important, make them your cash cows, rinse and repeat.
Every team I have been on have been more successful when they gave up on the product ideals and just ensured that the customers were hooked and satisfied.
alpaca128 6 hours ago [-]
That mindset is why most software was already slop before AI. Users know when your team has turned into salespeople. And now that AI could enable you to do better in the same timeframe the response with this mindset is of course to care even less.
tikotus 7 hours ago [-]
More than one service has lost me as a customer due to this process. I don't want a company to use me to verify their slop.
bigstrat2003 9 hours ago [-]
> Who cares if it's good enough? Send the product, if people use it, it's good enough. If not, let users debut it and give "feedback", problem solved.
I care, because doing a sloppy job and seeing if people will tolerate it is not acceptable engineering practice. It's not engineering at all. More pragmatically, if you do the thing you are suggesting, you won't have users for much longer. They will abandon you and go to someone who actually makes good products.
augment_me 9 hours ago [-]
I don't believe this in practice. I am at a FAANG-like firm, and we are just maxxing the engagement and people use the product more and more the more we serve stuff to keep them hooked using their data and behaviour.
The quality of what we provide seems to matter 0, we are up about 40% issues in prod year-to-year but the users and their engagement is also growing.
I think there is a small subset of technical nerd/idealists who think that quality is all that matters but they are in their own little sandbox while 97% of users are in another one.
willtemperley 8 hours ago [-]
> I think there is a small subset of technical nerd/idealists who think that quality is all that matters
This is just a derogatory comment.
It is exactly this kind of arrogance that is killing open source. In my area, GIS, there are foundational projects that are necessarily intricate. For example it might be necessary to make a four-dimensonal transform of a location, that takes into account point movement due to the Glacial Isostatic Adjustment. There are hundreds of other complex examples I could come up with.
The libraries are intricate and quality does matter.
Unfortunately the lynchpin maintainer has just gone on indefinite strike, partly because open-source vibe-coders are wholesale copying and relicensing some of these libraries in their quest to ... do what?
I actually I don't know what they want. Take all the oxygen from the room as far as I can tell.
augment_me 7 minutes ago [-]
Well I think both the maintainer and the people copying the software will realize that the code has 0 value, but the community and peoples attention has some value.
The oxygen on the room here are peoples ego. Use your tool and interact with your community while you have their attention.
Rapzid 7 hours ago [-]
> I am at a FAANG-like firm
Also known as not-FAANG
> The quality of what we provide seems to matter 0
Good for you! There are lots of low skill development jobs before AI where low knowledge coders cranked out WordPress sites for marketing, advertising, and small businesses. Sounds like you got in on the ground floor!
Meanwhile companies like Amazon, one of the As in actual FAANG, do have demands on their software quality. To the extent they essentially said "knock off the tokenmaxxing productivity theater" in a recent, publicized internal memo.
Horses for courses. Sometimes you need an thoroughbred and sometimes a pack mule.
And there's nothing wrong with being a pack mule! And nobody expects a pack mule to understand the work of a thoroughbred! Nobody expects too much from a jackass after all.
TonyStr 7 hours ago [-]
You are attacking the person rather than the idea.
puelocesar 6 hours ago [-]
> The quality of what we provide seems to matter 0, we are up about 40% issues in prod year-to-year but the users and their engagement is also growing.
Sorry for going into a tangent here, but I'm actually unsure if I hate the structures and economic incentives that makes this possible, or if I hate you specifically for defending something so rotten
Next time I rage at some broken UI in some damn service I'm forced to use because that's where people are, I'll remember this comment
tonyedgecombe 4 hours ago [-]
It's all incentives. Blaming the GP for their slop is like blaming your labrador for getting fat when you overfeed it.
EA-3167 11 hours ago [-]
In the narrow case of your profession perhaps. Beyond the slipshod world of most software is a universe of standards, licenses and liability.
The lawyers and judges who trusted an ML system to be up to snuff are currently pounding sand.
augment_me 9 hours ago [-]
I worked for one of the largest device manufacturers for a certain kind of sensor.
The requirements/standards for this sensor was made by this company 20 years ago. All the tests and requirements were made by this same company in a way that would minimize the costs for us to keep making new revision we could sell, and make it the hardest for any competitor to enter the market. Another competitor started with a better product abroad, so we copied it, but it didn't pass the tests, so then they installed an owner of the certification house that was a cousin of one of the VPs. No problems passing tests after that.
You think this is all controlled and nice and standardized but it's all just a market from software to hardware.
intended 9 hours ago [-]
While quite believable, there is a diversity of industries and standard adherence / enforcement no?
Markets also range from under regulated to over regulated, and are usually in a transition from one end to another (while contending with other forces).
intended 9 hours ago [-]
Why would Prime be in that list. He had an entire arc where he tried to make it work and then went back to the drawing board.
I am not saying he writes all code by hand, of course. Just that he is furthest in figuring out some sort of mid point between full comprehension and cognitive surrender.
I have missed the more recent content, so perhaps I have missed something, but this was my read last I watched.
gunslinger_meta 4 hours ago [-]
I'm also not sure why anyone would attach importance to his opinions as a software engineer, he's exceptional in generating entertaining content but as an engineer he does not seem above average. Definitely not even close to the category of Jonathan Blow or Andrej Karparthy
Rendered at 15:30:24 GMT+0000 (Coordinated Universal Time) with Vercel.
When I'm working on my own, I scrutinize every line of backend code and frequently send the LLM back for rework. Frontend code I keep an eye on just to make sure the LLM isn't getting stupid (which it occasionally does, even at the frontier), but mostly defer. The CSS I don't even look at.
This isn't too different from having employees. How much do I review other people's work? Depends on the employee, depends on how critical the work is.
Most of the non-technical people I see using LLMs to create software are not engaging in "cognitive surrender" as far as I can tell.
They didn't read the code before and they don't now. However, asking questions and improving understanding of the system and architecture is actually faster and easier for them now than it used to be.
They're evaluating what they can see, i.e, judging results. They build things for their own use. They test them, iterate on them to meet their needs and try to achieve their vision. This is roughly the same process they followed when they used to outsource to an agency except the loop is much tighter.
It seems to work well, mainly hurts when the problems are invisible, so security, backups, breaking migrations, perf, maintenance over time (language / runtime updates, dep updates, server updates etc). All the stuff that hurts if you have never been responsible for the uptime of something before. Not knowing that software is never "done". But junior devs trip on those exact same things.
I agree with the tail of the article; tooling and processes that make that stuff more visible would close a lot of the gaps.
If we continue from your "water comes from the tap" example this situation feels like someone who has no experience of plumbing deciding they're going to replace their bathroom without bothering to find out how any of it works. They'll possibly end up with a bathroom that superficially works but they're also liable to find problems down the line with leaks, things not draining properly, or water pressure issues.
Some of the people writing software purely via LLMs are even turning this into a business, which I guess would be analogous to deciding you're going to run sewage treatment for a small city without understanding anything about how that works. It'll mainly hurt when the problems are invisible, so food poisoning, or inlets backing up and overflowing into a nearby river.
In the bathroom remodel analogy, the AI is the plumber that you hire to do the work for you. You tell it what you want, it figures out the details necessary to get it to you. If what you want isn't possible, or requires large changes, a good plumber will raise the issue to you and explain the constraints they are working under, ask for guidance, and offer recommendations.
Sometimes the AI does things in a weird or inefficient way, or neglects details that become important later, or misunderstands something important about the task resulting in an outcome other than what you wanted, but that's also the same as the plumber. This is why people usually want to be at home and keep an eye on the people working on their improvement projects or repairs, because they want to make sure that the guy they hired is replacing the broken faucet, not the functional but odd-looking one.
There is still value in knowing the underlying systems well enough to present a good design vision to the plumber up-front. It's still possible to have a competent executor produce a bad idea, which is how we get stuff like Groverhaus, the Doom Bathroom, and McMansion Hell, which are examples of "slop architecture" in exactly the same way as we are seeing "slop software".
I agree it's pretty bloody cheeky to run an agency like that though. The only ways out I can see are licensing or real liability.
But let he who has never done dodgy home improvements cast the first stone.
People looking for self-contained, spot solutions to localized problems will have much more capacity to solve them locally, while people with in-depth expertise will be able to build systems that are even more reliable, performant, and scalable than what they were building before.
So we'll get the benefits of disintermediation for the basics, and much greater capacity to deal with effectively with complexity for the not-so-basics.
One of the things I've built in my current role is tooling to help understand current architecture, and to resolve the differences between things as-documented, as-implemented, and as-actually-behaving. It's used for a few things, including updating documentation, but it's been most impactful when onboarding new engineers. They can simply ask questions about how things work and why, and get explanations and deep dives wherever they need it.
Is the article claiming that?
> Not knowing all the details seems more like the default human condition, in the sense that most people don't know exactly where their water comes from or where their waste goes exactly. "Water comes from the tap, food comes from the store".
Sure. The ability to specialize, and on the flip-side not have to care about a lot of things, remains one of the biggest driving forces of human civilization. But in this article, I don't think the person worried about cognitive surrender is the "most people" in relation to food or water. They're the farmer or the water infrastructure engineer! They absolutely for sure do need to know those things. And all around us we see a lure of systems that invite them to surrender.
> Most of the non-technical people I see using LLMs to create software are not engaging in "cognitive surrender" as far as I can tell.
Maybe not. But the might be cognitively surrendering when they use the LLM to summarize the day's news or write a message to a friend, or even plan dinner for the week. I'd say they are. I agree that it's not clear-cut.
> They're evaluating what they can see, i.e, judging results. They build things for their own use. They test them, iterate on them to meet their needs and try to achieve their vision.
No. They have things built, then they test them. This is different. It might be OK, but it's not the same as building.
> This is roughly the same process they followed when they used to outsource to an agency except the loop is much tighter.
Yes. But they wouldn't usually outsource summarizing a letter from grandma, or the political agenda of a candidate/party in an upcoming election. That's surrender.
> It seems to work well, mainly hurts when the problems are invisible, so security, backups, breaking migrations, perf, maintenance over time (language / runtime updates, dep updates, lib updates, server updates etc).
… or ones own abilities deteriorate.
> Tooling and processes that make that stuff more visible would close a lot of the gaps.
Not the problem of one's own skills though.
Fair. Maybe I "cognitively surrendered" a long time ago just as a consequence of working in large teams :(
I hit the breaking point of not being able to read all the code well before LLMs came along. Also people would leave and the documentation for their subsystems would be wrong or incomplete, so there were always mysteries to chase down (during an incident, naturally) even in code written by hand.
Gets to the point, you don't exactly read the MRs, you're mainly checking their approach against what you talked about earlier and making sure it doesn't look way more gnarly than it's supposed to.
Working with agents doesn't feel that different.
> No. They have things built, then they test them. This is different. It might be OK, but it's not the same as building.
It does bug me sometimes when people say "yeah we built the deck..." and they got a contractor to do it. But then we do say things like "Nolan made the film" even though he didn't hold a camera or build the sets. To me, one-shotting doesn't feel like building, but engaging in a loop and applying control to get the outcome you want, does. Regardless of the exact toolset.
I don't personally see super hard lines between using a compiler, a 4GL, a sql query, stitching together 10 libraries and a framework I didn't write, using an LLM or even directing a team.
> Not the problem of one's own skills though.
I read somewhere once we can't weld Saturn V engines anymore, or at least not economically. We would also probably have a lot of trouble finding people who could do all the engine calcs the way they did. It doesn't necessarily mean that rocket progress is stalled.
> Yes. But they wouldn't usually outsource summarizing a letter from grandma, or the political agenda of a candidate/party in an upcoming election. That's surrender.
Agree 100% on the letter from Grandma.
I do say that, too. But I'm not in the film industry. My understanding is that those who are, don't use words as crude as "made" to describe a role in the making of a film. I believe they say that he wrote, directed or produced a film.
We are not talking about some other field of expertise here, where we are outsiders – the production of food, the supply of water, or the making of movies – we're talking about our own field of expertise.
They didn't read code before, they just wrote it? That makes no sense.
Previously, pretty much everyone all agreed on YAGNI and KISS because code was so expensive to create and more expensive to maintain. Now the code generation is "free" - but at a design level, YAGNI and KISS do still matter. But they are enforced now only by sheer will power. You have to actively restrain the code from being created if you don't want it. Perfectly good code - I can't even fault it. I just know, if it doesn't need to exist yet, it's a mistake to create it earlier than needed.
> La perfection est atteinte non pas lorsqu'il n'y a plus rien à ajouter, mais lorsqu'il n'y a plus rien à retirer.
> Perfection is attained not when there is nothing more to add, but when there is nothing more to remove.
On the same theme:
> the supreme goal of all theory is to make the irreducible basic elements as simple and as few as possible without having to surrender the adequate representation of a single datum of experience
- Einstein
> I would have written a shorter letter, but did not have the time
- Pascal
> Simplicity is a great virtue but requires hard work to achieve and education to appreciate, and, to make matters worse, complexity sells better.
- Dijkstra.
When endless bootstrapping and framing is free, you end up generating the most beautiful scaffold while ignoring the building behind the scaffold.
Now would be the ideal time for everybody to give it negative feedback when bad code is returned but I'm afraid not enough people care and we'll just settle with what works just like we always have.
The design machines aren't good enough yet; and their progress has little relationship to LLMs at all. Designing computer programs and the proper level of abstraction for the easiest maintenance, flexibility and debugging is simply an open problem. There's nothing to train LLMs on that is smaller than the size of an entire program, and its history of code changes.
Get a few of those, figure out a way to analyze and reduce the patterns and lessons in them, and train on that maybe. Until then, there's no way for an LLM to figure out what lacking a particular abstraction is going to end up costing.
You can choose to ignore the fine details and just tell it what to build, or you can spend more time and effort and tell it how to do things, you can ask it to create a plan, review the plan and ask for changes etc, you can review the implementation and ask for changes there. You can simply tell it you don't like how a feature is implemented and ask it for suggestions on how to improve it, include what you don't like about it so it has direction.
It's all about the effort you put in. LLMs are able to write clean beautiful and efficient code, they just can't necessarily do exactly what you want on the first try without any direction. That's why you should figure out what you want, ask for what you want and make sure it's actually building what you want.
Keeping the volume of code down is definitely a challenge though.
At the same time I can't help but wonder that this is almost too much for us to cope, the LLMs are _too_ easy to use. I think we're very good at pretending there isn't a problem here but there is definitely going to be some cognitive decline happening.
We want to pretend like reading over the output of an LLM is nearly the same as us making those changes ourselves but I think there's a lot of value in those boring and manual steps. And most people (including me) will be too lazy to not eventually offload most things to the LLMs.
In a way this is similar to how watching a lecture tends to give one an "impression" of understanding that doesn't correlate with reality. We need hands-on work and some course work to get our monkey brains engaged.
I can only imagine what effects this is going to have in education.
Our cognition didn't evolve to produce arguments for online discussions, but for making crude tools and hunting. As we stopped applying our intelligence towards these uses as a result of our tools, we simply shifted it to new problems that were unthinkable previously.
A caveman probably wouldn't care too much about algebra, a topic most children learn today.
We should stop thinking about the problems AI trivialized and, under the assumption that AI won't fully replace human capability (a key assumption), start thinking about the problems that were previously out of reach. Education would then start focusing on that.
Taking clothing for example, there's more available now than ever. It's disposable at great cost to the environment and the workers that produce it, but very little to you.
Yet, I bet you cannot evaluate garments, construction, or fabric quality. You may not know how to wash woolens and linens. You may have only worn smelly plastic workout clothing and plastic suits for formal events.
You may never have worn a piece of clothing that actually fit your body.
You may not realize that you can alter any clothing, that most clothing was intended to be altered once upon a time. Or that older garments were both more expensive and higher quality, because the expectation was the most important items would need to last for a decade.
The Luddite's central objections about automation was about declining working conditions (objecting to child labour, for instance), bad pay, and declining quality. They used the 'obnoxious' machines they broke. They saw what was being lost and demanded better (without demanding no machines at all).
Later, the rise of unionization brought about many of the changes they'd asked for decades before.
I hope this current technologic movement doesn't end where fast fashion has, with influencers buying a new outfit [repo] every day for a year. A vortex of undifferentiated slop for its own sake.
I can also recommend the book "Blood in the Machine" by Brian Merchant [0][1] which is precisely about this.
[0] https://www.hachettebookgroup.com/titles/brian-merchant/bloo...
[1] https://www.bloodinthemachine.com/about
It'll take a particularly dumb and lazy AI to replace me.
There’s one thing that’s bothering me though — all of these have us consuming what LLMs produce. I learned and internalized how things worked by building systems. At least within the constraints of the abstractions I was using. This makes me think of TikTok and shorts. Everybody is just watching. Maybe that’s not charitable though. Maybe the abstraction has now changed from an “os/gui/event loop” to “llm” and the rest is just accidental complexity.
Automatic control is good for standard case, and the expected errors. For weird cases, for now humans are better to detect them and press the big red button.
But.. we do monitor whether our prayers are effective .. right? Is that a smitable offense?
After that disappointment, I too was "curious": was the author going to talk about "curiosity", that magic virtue that we hackers believe, lifts design into the divine (but more often than not kills the pussy)
Think of platforms like Shopify. Shopify allows huge number of external users to write code customizing their web shops. But Shopify engineers do not need to review and understand the users' code, because there is a boundary which limits what the external code can do. The external code can be A Big Ball of Mud without making the whole platform a A Big Ball of Mud.
But UI on top of a good API should be unable to cause problems; the API must prevent invalid state etc. So letting LLM’s write UI code for CRUD-apps (which is most of them…) seems fine to me.
That said, the core apps are still full of AI code. I don't see a reality where hand-written code can or will be curated and sequestered. It's unenforceable, too slow, and too expensive.
In order to be able to enforce this hard boundary, it means that one "agent generated code" in one place, not only "can call a strictly defined set of APIs" - but, more importantly, it implements an isolated piece of functionality. Think something like: I let the AI write the dashboards, so the dashboards may be slow, crappy, incorrect, whatever - but it doesn't impact the rest of the product. However, I can't let AI write "metrics collection" because that not only impacts dashboard, but potentially impacts core product efficiency too (memory consumption, cpu, IO, etc.)
The only way things will evolve is towards maximizing AI generated code and minimizing human involvement because that is the most efficient mode for getting product to market. Any trend to the contrary is temporary, paperclip maximizer will maximize. Any code that humans touch directly will be considered implicitly too risky to run in any production environment.
We'll just have to hope things don't break too much and when they do, we'll just have to accept it because no human will be able to (or allowed to, or even trusted to) fix it.
Its just as much as an ego trip...as the Principal Engineers that roam around here, after they survived three layoffs and outputted front end code for six years straight at a FAANG...
Prove me wrong...
But oh well, who was she, just the engineer who wrote the software for the Apollo 11. Probably she had no idea what she was talking about.
/s
Not to be confused with Margaret Hamilton the actress who played the Wicked Witch of the West in the film The Wizard of Oz.
Code is not like craftsmanship anyway. The end-user doesn't see any difference whether I use microservices or hexagonal design or react or htmx or how I structure the code or which language I use. All they see is the design and the stability. Since the average developer was terrible at design anyway, the average design output from LLMs is actually an improvement in most cases, even if it always has that generic AI generated look. Stability... was not good before AI either. We have become accustomed to SAP taking 10 seconds to load each component on the page, or Atlassian spewing error messages in your face when you click submit on a ticket. It seems that the aspects that are most valued by our end users are QA and UX, not the craftsmanship of our code.
But of course, every time consumers voice their preferences is actually "a loud minority" or an "echo chamber" :)
And if you look a the dates, the rate at which they are doing it is actually increasing...
Sure but that doesnt require the person to be a carpenter though right? Anybody could do that.
It's worth remembering that, contrary to what HN acts like, software engineers are not uniformly accepting of LLMs. They are still quite controversial in the industry. Now management, on the other hand, is completely sold on the supposed benefits of LLMs. So many engineers feel forced to use them, but it's not enthusiasm on their part which drives them to that.
The twitter(x.com) and linkedin crowd has drank the cool-aid and are now swimming in it; Wild posts from FAANG engineers and managers saying they or their engineers dont read any of the code the LLM generates.
It' like saying compassion is a solved problem: just tell your problems to the LLM and have it gush out sympathy.
Fighting over engineering is hard. But it is also one of the opportunities for us humans in the era of abundance.
I did learn a lot about the ecosystem and myself from this experience.
This has become common now, and I've seen it ruin large projects until nobody understands the codebase full of verbose nonsensical code and comments. It becomes impossible to further develop the project without AI. Contributors not using AI for any reason are all leaving because no one enjoys working with mindless robots (and humans) as colleagues. It's demeaning and devaluing their labor and humanity. They're not getting "left behind", they are leaving the madness of crowds behind.
> ..the possibility of runaway dynamics: once a critical threshold is crossed, small increases in adoption can trigger rapid population-level shifts toward *persistent dependence, with abrupt losses in cognitive competence*.
We can all see it happening. As this technology permeates society, the challenge to maintain control, coherence, and quality will become greater and ever more serious - for example, AI in healthcare, military, management and governance.
But before LLMs, it didn’t matter because on a good team you’d have a balance where maybe some PRs added complexity and some simplified it. Sometimes people would include a small related cleanup or refactor in a PR for code around what they were doing if they noticed complexity. That has become far less common, I think.
I know it’s not your point, but a 1% decrease 100 times is 0.99^100 = 0.366 which is a 63.4% decrease in understandability. Just throwing that out there
Enterprise software has always been crap.
If we don't value that though--if we don't value understanding the system and just thinking through problems as a group--then let's stop pretending and just not do code reviews. Probably nobody cares and just types "LGTM" after having their own LLM do a spot check to cover their ass.
It's really easy to push developers to stop caring. If management says "go fast fast fast, use those LLMs!", then developers can't keep up, get frustrated, stop understanding the system, and at that point they're just there to collect a paycheck before society collapses. Or, if management is making dumb decisions all the time, same thing.
Hopefully as implementing ideas takes less and less time and resources, smart people who actually care will start building things with other smart people who care.
Specifically, humans and their dysfunctions. Their fragility. Their ego tied to stuff.
This is an excellent post/reflection on the topic of solo LLM use. The only gap I see is that it assumes that one works on something alone and not in a team. Though, that might be a different essay in itself.
Edit: Turns out that footnote 3 actually mentions that already. Nice. Good post
1. Smaller PRs (that can be easily read) tend to receive more comments and nitpicks than a large change since it's easier for us to have all the PR context in our heads at the same time.
2. Most people are ego-tied to their code even if they pretend that they are impartial
3. Social dynamics around merging and reviewing code between senior/junior and older/newer employees
4. probably many more
[1] https://www.nytimes.com/2018/02/16/opinion/sunday/tyranny-co...
> Once you have used a washing machine, laundering clothes by hand seems irrational, even if it might be cheaper.
Washing clothes by machine is orders of magnitude cheaper than by hand when you factor in the value of your time.
> After you have experienced streaming television, waiting to see a show at a prescribed hour seems silly, even a little undignified.
Undignified? What? Completely incoherent word slop. Good to see LLMs haven't completely taken that domain for themselves, I guess - a little hand-crafted slop now and then is a treat, I guess? Anyways, they're wrong. People still tune in to watch scheduled live events all the time.
> To resist convenience — not to own a cellphone, not to use Google
Neither of these things are a matter of convenience. A cellphone is at this point literally required to participate in a modern economy. Google is a tool that provides you with globally-retrieved information that you otherwise literally could not find on your own or without another similar tool.
I don't even disagree with the premise that convenience is a driving factor of our lives, but the justifications and conclusions drawn by this article are beyond stupid. Here's my view of it: the desire for convenience is what pushes us onwards as a species. Hunting for food is inconvenient, so we invented agriculture. Going down to the river or well for water is inconvenient, so we invented indoor plumbing. Repeat ad infinitum until we have modern civilization. There is one part the pessimistic view overlooks; that to achieve maximum convenience long-term requires more effort in the short-term. Developing the skill of programming is inconvenient in the short-term but convenient in the long-term because of the possibilities this opens up to you. This remains true even in the current LLM era, and it will remain true for as long as we are not replaced wholesale by LLMs.
I'll also note that it is less resource intensive overall. Handwashing dishes and clothes will use more water and more energy to heat that water compared to the machines that do the same job (people often assume the opposite).
It takes a mix of willpower / routine / systems / laws to keep people in check, otherwise we predictably go back to the path of least resistance.
The article is inspiring though, I have more motivation to work on my thinking process and my engineering setup (and vibe limit haha) to be more thoughtful, thank you.
https://epiq.sh/blog/cost-of-cognitive-debt.html
1. LLM usage, in particular unsupervised, tends to build up "cognitive debt"
2. It tries to characterize the concept, including to propose a score for it
3. Notes that the level at which one can let the agents loose (and allow some CD) differs quite a lot between core business domains and more peripheral ones
I am losing more of what little focus I've had every day.
As an example, I’m juggling multiple projects at work and home. I have a finite amount of time, and access to AI. One camp would tell me to use AI for all of it to move faster, but that increases the risk of Cognitive Surrender. The other camp would tell me to do it all by hand, but I don’t have the time available for that; I must pick and choose.
So, I choose to cognitively surrender on the SEO task for our website because that’s a set of skills I don’t want to acquire and are better handled by AI anyhow. This leaves me more time to focus on learning more about Apple’s configuration profiles and improving our MDM suite, which are valuable skills that will still pay out in an AI-first world.
You have to pick and choose where to fight, otherwise you’ll expend all your energy on losses.
The economics of the whole affair unfortunately point squarely in the opposite direction.
Of course, when you're working on a system of high complexity - and one that is constantly changing to boot - with all sorts of historical reasons and what not that inform the why and so forth, you can't reasonably be expected to understand everything. This is what bracketing is for. When I acknowledge "this is the way X works", it doesn't mean I have to accept the proposition "this is the right way/best way to do X" or "X must be this way". A teacher or mentor may say it is, but in a theoretical context, unless it has been justified, you cannot claim that you know that what has been said is true. However, you can accept it as a provisional working hypothesis. And practically, it may work out and perhaps experience will lend some feeling of substance to the hypothesis, but you needn't make it doctrine.
Empirical science often has the same spirit. Perhaps the model isn't a true account of reality, but it puts handles on reality so that it can be instrumentalized.
There is an occupational danger here. If we focus exclusively on the instrumental, if we reduce all of human attitudes toward the world to use and utility, we not only vulgarize ourselves and our orientation toward the world and even others, but we risk trampling the truth and the good underfoot. We risk developing an indifference to the truth, because all that will matter is some measure of effectiveness. This is what one might call the textbook[0] definition of bullshit.
[0] https://www2.csudh.edu/ccauthen/576f12/frankfurt__harry_-_on...
I mean, if the management doesn't see value in decent engineering, then who am I to butt myself into the conversation?
Karpathy had the same trajectory. Primeagean had the same trajectory. I think in the end everyone except for Jonathan Blow will have the same trajectory
However it takes more mental energy and people don't like that
I guess you could ask the model. "Yes well done, you're a visionary software engineer".
That's very different from what a lot of vibe-coders do, like wholesale cloning and relicensing projects. This is happening in the GIS space, and these vibe-clone projects slightly hilariously inherit all the technical debt they didn't understand.
Nothing wrong with using LLMs, but I agree with Andrej Karpathy: "you can outsource your thinking, but not your understanding".
Every team I have been on have been more successful when they gave up on the product ideals and just ensured that the customers were hooked and satisfied.
I care, because doing a sloppy job and seeing if people will tolerate it is not acceptable engineering practice. It's not engineering at all. More pragmatically, if you do the thing you are suggesting, you won't have users for much longer. They will abandon you and go to someone who actually makes good products.
The quality of what we provide seems to matter 0, we are up about 40% issues in prod year-to-year but the users and their engagement is also growing.
I think there is a small subset of technical nerd/idealists who think that quality is all that matters but they are in their own little sandbox while 97% of users are in another one.
This is just a derogatory comment.
It is exactly this kind of arrogance that is killing open source. In my area, GIS, there are foundational projects that are necessarily intricate. For example it might be necessary to make a four-dimensonal transform of a location, that takes into account point movement due to the Glacial Isostatic Adjustment. There are hundreds of other complex examples I could come up with.
The libraries are intricate and quality does matter.
Unfortunately the lynchpin maintainer has just gone on indefinite strike, partly because open-source vibe-coders are wholesale copying and relicensing some of these libraries in their quest to ... do what?
I actually I don't know what they want. Take all the oxygen from the room as far as I can tell.
The oxygen on the room here are peoples ego. Use your tool and interact with your community while you have their attention.
Also known as not-FAANG
> The quality of what we provide seems to matter 0
Good for you! There are lots of low skill development jobs before AI where low knowledge coders cranked out WordPress sites for marketing, advertising, and small businesses. Sounds like you got in on the ground floor!
Meanwhile companies like Amazon, one of the As in actual FAANG, do have demands on their software quality. To the extent they essentially said "knock off the tokenmaxxing productivity theater" in a recent, publicized internal memo.
Horses for courses. Sometimes you need an thoroughbred and sometimes a pack mule.
And there's nothing wrong with being a pack mule! And nobody expects a pack mule to understand the work of a thoroughbred! Nobody expects too much from a jackass after all.
Sorry for going into a tangent here, but I'm actually unsure if I hate the structures and economic incentives that makes this possible, or if I hate you specifically for defending something so rotten
Next time I rage at some broken UI in some damn service I'm forced to use because that's where people are, I'll remember this comment
The lawyers and judges who trusted an ML system to be up to snuff are currently pounding sand.
The requirements/standards for this sensor was made by this company 20 years ago. All the tests and requirements were made by this same company in a way that would minimize the costs for us to keep making new revision we could sell, and make it the hardest for any competitor to enter the market. Another competitor started with a better product abroad, so we copied it, but it didn't pass the tests, so then they installed an owner of the certification house that was a cousin of one of the VPs. No problems passing tests after that.
You think this is all controlled and nice and standardized but it's all just a market from software to hardware.
Markets also range from under regulated to over regulated, and are usually in a transition from one end to another (while contending with other forces).
I am not saying he writes all code by hand, of course. Just that he is furthest in figuring out some sort of mid point between full comprehension and cognitive surrender.
I have missed the more recent content, so perhaps I have missed something, but this was my read last I watched.