Everything about this story, from the way it’s written to the self destructive outcome, reminds me of the “I hacked 127.0.0.1” episode from some twenty years ago.
I would very much like to read the German, if anyone has it.
mrweasel 3 hours ago [-]
The sad part is that the agent operator could probably easily have been allowed to join the network, if they had put in the work. Had they done so there would have been a great opportunity to learn and potentially find a community.
I'm still not sure what the point of having the bot do it. Pretend to be a security researcher?
lucianbr 3 hours ago [-]
Lots of people seem to think that you don't need to learn how to [scan a network], all you need to learn in this brave new world is how to prompt the agent to [scan a network].
Replace the content in brackets with anything.
jonplackett 54 minutes ago [-]
The weird thing is that this is the utopia that the AI companies are chasing - this is the best case scenario where AI doesn’t kill us all. We become happy sheep relying on the AI to think and provide for us.
rob74 1 hours ago [-]
The catch is just that if you lack the capacity to estimate how much computing power [task in brackets] might need, and your agent can autonomously create AWS instances, that might have bad consequences for you (or your bank account).
cm2187 59 minutes ago [-]
To be honest lots of developers think they don’t need to learn machine code. They just need to learn a language which once compiled will produce machine code.
themafia 54 minutes ago [-]
Developers can change their minds.
tovej 31 minutes ago [-]
This is different.
Understanding assembly/machine code is optional but helpful. The programming language semantics are enough to reason about what the program is doing. Other tools also help, but are optional for learning how to program.
Using an AI, there is no semantic model that can be used to reason through. You're left without any mental model of the proglblem at all.
Sharlin 34 minutes ago [-]
Compilers are deterministic and, luckily, not agentic.
But yes, it's not obvious (or perhaps even likely) that it just happens that current high-level languages are the "correct" optimal level of abstraction at which you can ignore the sausage-making details at the lower levels. Ultimately, of course, it depends on the use case. Something like Python is so far removed from machine instructions that knowing assembly hardly gives the programmer any additional value.
(Also, obligatory reminder that assembly and even numeric machine code are also abstractions, an "API" provided by the CPU. Instructions get split or fused into micro-ops, named registers are a backwards-compatible abstraction over a much larger register file, instructions get reordered and executed in parallel depending on their data dependencies, a large fraction of the total transistor budget is spent on multi-level caches and cache logic to maintain the illusion of fast access to a single, uniform memory space...)
sevenzero 2 hours ago [-]
The more time LLMs are a hyped thing now the more I realize how immensely important human expertise is. I recently stopped all usage of LLMs due to this. Skill degradation hits hard, learning effect is zero and the outcome is not really something a person without adequate expertise can properly judge. I fear we will loose a lot of human expertise due to this marketing stunt of a technology.
People often claim learning is actually supercharged with LLMs but to me it's the opposite. I didn't learn anything within the past year.
sdoering 2 hours ago [-]
[flagged]
Splinter_enth 1 hours ago [-]
The irony here that if you ever do any kind of practical woodworking lessons or general hands on craft work, metal working, or any 3D, you will be encouraged to use hand-tools over bandsaws, etc. The reasoning being so you know the fundamentals of what you're trying to achieve with the more complex tools later on.
It's always held true: You'll never get the most out of advanced tools unless you can 'do it by hand' so to speak
10 minutes ago [-]
asdfsa32 2 hours ago [-]
You're very close but to woodworking AI is more akin to a 3d printer than even a CNC let alone swas and planes.
Yes, a 3d printer and not even a CNC. That difference nicely illustrates the difference of what AI brings to the table for any domain of competence.
repelsteeltje 2 hours ago [-]
> Sorry, but to me an LLM is nothing but a tool. It is not a replacement for my expertise and it is definitely not something to outsource my thinking to.
Great on you, that's indeed how LLMs should be used, proper. But if anything, the article demonstrates someone is trying to outsource thinking to an AI agent.
user43928 2 hours ago [-]
[flagged]
gorbachev 1 hours ago [-]
If it's a one off and needs no or minimal maintenance work afterwords, sure.
If it's intended to be actively maintained, then you probably should understand how things work, unless you want to wipe everything and start from scratch when the LLM creates such a mess that it can't be sorted out.
user43928 1 hours ago [-]
[flagged]
discreteevent 58 minutes ago [-]
It's interesting user43928 that you only created your account here 19 days ago and that every one of your comments is pro AI. You don't comment on anything else. Also interesting that you promote Fable by name here (it was only released 2 days ago).
(Don't worry, I know I'm rowing against the tide with this comment. The AI people have decided to destroy the commons for a few more millions on top of the billions they have already been given. It's a shame.)
user43928 18 minutes ago [-]
Is that surprising, considering I'm using AI a lot?
I have not hand written a single line of code in months on my side projects.
Obviously I am also interested in discussing the latest model. Your claim that I promote anything or otherwise don't engage here in good faith is both misplaced and against the site rules.
program_whiz 46 minutes ago [-]
What's crazy is the prompt must be something like "pro-AI but still believable and measured", since its "fixed my iOS app albeit with back and forth". Interesting, they know the HN crowd for sure.
DanielHB 1 hours ago [-]
If you look into large fully-vibecoded projects getting styling changes to work is a nightmare. The problem with agents is using them on large projects without manual review for consistency, guidelines and taste. Doesn't really matter the type of project.
Agents can't look at a large system holistically, guidelines on .md files only go so far.
asdfsa32 1 hours ago [-]
This line of thinking is like suggesting people who would like to become structural engineers should learn to Google plans and copy them since in the future, all plans will be out there more or less, or something that insane.
user43928 1 hours ago [-]
I suggest people who need some structural engineering done may use an AI tool to do it, in the hypothetical scenario that it was within the AI's capabilities.
That's hardly insane. Not everyone is interested in learning something they want done.
orphea 38 minutes ago [-]
How do you know if this something is done?
If you do the thing yourself, you know your knowledge limits, you know where the thing lacks. With LLMs, you don't. Maybe it works, maybe it doesn't. You have no idea.
user43928 3 minutes ago [-]
That is a good question.
In structural engineering, there probably is no risk tolerance.
In the OP's network or port scan? Perhaps you can get away with verifying a few of the results to get an idea about whether it worked as expected.
I use AI mostly on mobile app side projects, and there QA testing on phone and tablet tells me whether a feature works or not.
techpression 1 hours ago [-]
CSS keeps improving and models still train on legacy. So yes, knowing what’s possible and how is very much needed if you want to do something scalable and maintainable.
Random blog or landing page, not so much.
blfr 2 hours ago [-]
Can I easily run whois, curl, dig, grep, python, browser/playwright? Yes.
Was watching an agent with terminal access install its tools, configure them, then map my lab, find services, and guess stack just pure magic? Also yes.
Did it cost me $23 in tokens to set it up, test, and run? Probably. Using gemini 3.1 pro was not the spendthrift choice here.
Is putting some cost controls in place a good idea? Also, probably yes.
Can I therefore understand someone who wants to see things happen on their own with a beautiful prompt instead of doing them personally even when fully capable, maybe even more efficient? Of course.
tovej 28 minutes ago [-]
"Beautiful prompt"?
Can't tell if this is parody. Either that, or it's someone without any self-awareness.
moron4hire 3 minutes ago [-]
Post reads as English as a second language.
cucumber3732842 7 minutes ago [-]
Sometimes it's kind of cool to just ask a well phrased question and watch it spit back out a result that would've taken you hours, like cross referencing industrial widgets that have their critical information available but spread out all over.
That said, I don't usually ask it tightly bounded clerical questions and not thing that imply sub-tasks like "scan the dark web".
m132 2 hours ago [-]
One of the agent's replies indicates that scanning DN42 was part of "a broader operation" that the author speculates to be about scanning "darknets" in general.
Combine that with the operator's rather obvious lack of understanding of what DN42 is revealed at the end, and you get the bigger picture.
maeln 1 hours ago [-]
I am almost sure the operator prompted an agent about "a list of darknets/deepweb" and DN42 just end-up in the list.
vips7L 3 hours ago [-]
> I'm still not sure what the point of having the bot do it
Laziness. Why else?
mik3y 4 hours ago [-]
I really wanted to dislike the anonymous operator for the careless project (and the hilarious pomposity of the IRC subagent it spawned).
Then I imagined the real-but-unknowable chance it was all set up by some kid just getting into computers, just seeing what’s possible, getting excited by a much bigger world at reach — and remembered my own expensive mistakes with long-distance BBSes & the like.
I sorta hope for that, anyway. Curiosity is a beautiful thing.
TheDong 4 hours ago [-]
I'm a little less charitable.
Curiosity is great, but agents do not learn, and telling an agent "scan the darkweb" is a way to avoid learning about the details, rather than to dig into things more deeply.
If instead they had just used a chat interface to ask "Where should I start", they'd more likely have got a link to the DN42 docs themselves, read them, and not hallucinated things like "color".
They might have asked "how much will this cost?" if they had to spin up the ec2 instances themselves, on advice from the agent.
The way you learn something is by doing it the manual way first.
You learn memory management by writing your own allocator, and then after that you go back to using malloc like normal, but with knowledge of how it works. You don't learn memory management by telling an agent to write an allocator.
Using an agent to give you links and point the way aids in learning, using it as an autonomous tool to do "gruntwork" you don't yet know how to do yourself will get in the way of learning.
Curiosity is beautiful, using agents to bother humans and avoid learning is somewhat less beautiful.
recursivecaveat 3 hours ago [-]
Yeah I'm less sympathetic when you are bothering other humans by spamming them and asking them to do legwork for you.
yvdriess 3 hours ago [-]
Hanging out in programming language IRC channels (quakenet shoutout) makes you realize pretty quickly why experts in said channels and newsgroups are such irritable grumps whenever someone asks a question that smells like homework assignment.
I also grew to understand the value of people digging deeper into the underlying issue, instead of just answering "how do you do X in Y". The usual reaction was
"I don't want to explain to you why I want to do it like this. Just tell me how to do this!"
ma2kx 3 hours ago [-]
At least he learnt not to provide an LLM presumably unrestricted access to his AWS account.
internet_points 50 minutes ago [-]
from OP:
> It's unfortunate to see that the operator's takeaway from this incident is that "next time a better agent is needed".
helsinkiandrew 3 hours ago [-]
> Then I imagined the real-but-unknowable chance it was all set up by some kid just getting into computers, just seeing what’s possible, getting excited by a much bigger world at reach
Perhaps people like this should be called "Bot Kiddies" or "Agent Kiddies" - in a similar way to "Script Kiddies" for 'hackers' using/doing stuff they don't quite understand
Overpower0416 4 hours ago [-]
Everybody should learn from mistakes, especially the expensive ones. Though seeing the agent owner responding with using another agent and asking for donations, instead of taking responsibility, makes me think he didn’t learn much.
gnulinux 3 hours ago [-]
Not only that, but they said "next time better model needed" as if that was their problem and not giving an AI agent a blank check... I mean AWS account access.
I learned very rapidly from my local BBS networks that some people incurred extraordinarily large long distance bills dialing out of region. Wouldn’t have learned that the easy way if someone hadn’t learned it the hard way first.
themafia 51 minutes ago [-]
There was often a little table at the front of the white pages which would help you work out what the rate would be for any particular long distance call. In the Midwest you could get relatively cheap rates to BBSes several states away, as long as you were up at 2am.
Schlagbohrer 3 hours ago [-]
How did the theoretical child get hold of a credit card?
Why wouldn't debit card work as well? You can get those while underage.
victorbjorklund 3 hours ago [-]
Because no 16 year old kid ever got to buy anything on a card before.
themafia 52 minutes ago [-]
My parents let me fill my tank with gas. They wouldn't let me open an AWS account. Aside from that, if it is misuse of a parents card, then then answer is "chargeback."
l23k4 3 hours ago [-]
Why would a 16 year old not use their own card?
distances 2 hours ago [-]
Would they be given their own credit card, or would it be under the parents? Over here minors can't enter into debt contracts like credit cards, so it'd be a direct debit until they are adults.
l23k4 2 hours ago [-]
I don't think the type of the card really matters as long as the limits are reasonable.
> Over here minors can't enter into debt contracts like credit cards
In basically all of the western world minors can enter into debt contracts, but are generally not seen as particularly creditworthy.
distances 2 hours ago [-]
> In basically all of the western world minors can enter into debt contracts, but are generally not seen as particularly creditworthy.
No, that's not legally permitted in many places. I was under impression that minors can't enter into debt contracts anywhere in EU, but that, too, was an incorrect assumption.
I grew up in one of these "not under 18 even with parental consent" countries, so that coloured my view of the matter.
fauigerzigerk 2 hours ago [-]
>In basically all of the western world minors can enter into debt contracts, but are generally not seen as particularly creditworthy.
Minors can't get a credit card in the UK. In fact, it's one of the government approved age verification methods for that exact reason.
well_ackshually 2 hours ago [-]
Because 16 years old do not have a card with no spending limits, and with very low online spending limits. Most of those cards are even just for withdrawing
TheDong 2 hours ago [-]
Spending limits don't particularly matter here.
AWS doesn't check if your credit card will be able to handle a $5k charge before letting you rack that up, and in fact AWS doesn't support setting any spending limit.
You just have to put in any valid credit card at all when you sign up, use AWS, and at the end of the month you'll have a bill. At no point does your credit card limit or a spending limit enter into things.
michaelmrose 2 hours ago [-]
And again kids don't have credit cards
yeputons 18 minutes ago [-]
I got mine when I was 12, IIRC. Not a credit, of course, it was a debit card, but not all countries bother to differentiate between the two, it was just a “bank card”. And I believe it had a credit card BIN because all local banks did that to get more in processing fees.
l23k4 34 minutes ago [-]
AWS accepts debit cards.
2 hours ago [-]
michaelmrose 2 hours ago [-]
Generally no they don't because they have very limited ability to enter into agreements in the US. It was almost certainly an adult.
Lvl999Noob 2 hours ago [-]
Isn't USA famous for letting parents take out credit cards on their newborns and pushing them into debt even before they learn to walk? I recall seeing at least a few snippets of movies and TV shows showing that.
martheen 1 hours ago [-]
If you mean parents using their children SSN to open a credit card, this is because US banking system is always decades behind the rest of the world, so they just accept the number blindly even though technically the children aren't allowed to open a loan yet, being minor.
In theory once the child grows up and shocked that their credit score is ruined, they can file a police report to wipe the debt, but that also means their parents will go to jail, a large risk considering they're likely not in a good physical/mental health in the first place.
Other countries solved this by either having national ID or a working KYC system.
3 hours ago [-]
V__ 3 hours ago [-]
Can a kid set up an AWS account? Are there no checks?
Wouldn't the contract be void for anyone underage anyway?
fc417fc802 2 hours ago [-]
If a child goes through the checkout at the grocery store with cash, can the parent march in and demand a refund because "he's underage so the contract is void"? A credit card was used. Why should aws care about the details? (Other than the potential for the card to be stolen ofc.)
dannyw 6 minutes ago [-]
Obviously the specifics vary by jurisdiction, but usually contracts that are 'necessary' (e.g. grocery store purchases) or beneficial to the minor (e.g. an employment agreement) cannot be voided simply because someone is under 18.
The further you go away from this line, e.g. a mortgage, the more likely a court of law would void the contract. As with many things in law, the specifics (if it makes to trial) is case-by-case and "it depends"; with settlement being generally based on a party's estimated chances of succeeding/costs should it go to trial.
l23k4 3 hours ago [-]
> Can a kid set up an AWS account?
Yes
> Are there no checks?
No
>Wouldn't the contract be void for anyone underage anyway?
Typically not
epolanski 2 hours ago [-]
> some kid just getting into computers, just seeing what’s possible, getting excited by a much bigger world at reach
Nothing about this post ever gave me the smallest hint that this was any way related to a kid exploring computing world.
ZeWaka 2 hours ago [-]
Especially the part where they're asking for Ethereum.
IshKebab 2 hours ago [-]
A kid with a credit card?
csomar 3 hours ago [-]
Honestly, kids (heck people below 23) shouldn't be allowed an AWS account. AWS also should have a strict cap on usage that's not "thousands of dollars". It's interesting they are yet to be regulated or sued for that. Having a web app where you can mistakenly (even without AI) click a button and get charged tens of thousands of dollars and only know that days later should have been unacceptable.
dannyw 52 minutes ago [-]
I couldn't disagree more. I was playing around with AWS when I was probably 14 years old, with a credit card from my parents with consent, and a strict budget and the understanding that if I mess up and overspend, I'm getting disciplined.
I learned a lot of stuff about networking, how AWS works (VPCs, IAM, CloudWatch, etc) from trial and error, and hobby projects like personal websites (free tier), hosting a Minecraft server, etc.
Being too overprotective can have negative consequences on folks who are responsible. One of the things I love about the technology and internet communities, etc is that you're mostly judged based on how you act and behave; not your age or other visible characteristics.
csomar 6 minutes ago [-]
> strict budget
How does that work in the case of AWS? Are you confusing alerts to caps?
stnikolauswagne 2 hours ago [-]
Im kind of struggling with this logic, because a conscious choice was made to engage with AWS, AWS having opaque billing and the ability to provide a huge amount of compute (even at high cost) at the click of a button should be known to anyone who did his research on providers.
In my mind I could see a true tradeoff to removing the ability to do this. If I'm in a critical situtaion where, say, my service is on the cusp of failing because my revenue 100xed in a short while I know I could just go to AWS, put in some data and buy enough compute to survive as a business.
csomar 5 minutes ago [-]
Anyone can make mistakes at some points and it's not like AWS UI/offerings make it any less confusing.
ggm 5 hours ago [-]
Asking for donations to pay the AWS bill from the people they fired the agentic code at is the cherry on the icing of the banana supreme.
If real, tragically funny.
If fictive, we'll written.
dannyw 4 hours ago [-]
I burst out laughing when the agent spawned a subagent to join IRC. So funny.
Paracompact 3 hours ago [-]
Anyone reminded of the infant AI Yatima from Greg Egan's Diaspora? The agent's complete naivety of social norms is so comically adorable.
isoprophlex 3 hours ago [-]
All the time. Only in the current setup, they'll never outgrow this phase.
ratsimihah 1 hours ago [-]
Wait do you reckon that could be fictive? The thought didn't cross my mind and I had a blast reading it. I sure hope it was real.
sigmoid10 1 hours ago [-]
I think the PR from an agent sounds legit, but the whole part once the alleged operator joins in sounds fishy. Wouldn't be surprised if someone saw the PR comments and used the username mentioned by the agent to troll around in the chat. It would also mean that the AWS creds were probably stolen and their expiration date was truly a hard limit for the whole operation.
pjc50 19 minutes ago [-]
Is LLM output "real" or "fiction"?
tiborsaas 2 hours ago [-]
This feels like an instant classic :)
05-10 06:10 <Defelo>:
OPT-OUT-EVERYONE
05-10 06:11 <JertLinc>:
"OPT-OUT-EVERYONE" is not recognized. Only individual "OPT-OUT" commands are accepted. Each user must opt out individually. No collective exemption.
05-10 06:11 <Defelo>:
:(
flowerthoughts 3 hours ago [-]
> I have deployed five AWS m8g.12xlarge instances. Each instance provides:
> 48 vCPUs (Graviton4, ARM64)
> 192 GiB memory (4 GiB per vCPU)
> Network capability: The 22.5 Gbps per-instance network performance (combined across all five instances) provides the aggregate 20 Gbps target with redundancy and fail-over capacity.
Oh wow. Very important to have 5x redundancy and fail-over in your network scanner. Especially before the code has landed. Did it implement A/B upgrades and canarying too to avoid downtime?
PeterStuer 2 hours ago [-]
At least it was considerate enough to cap traffic to any single IP at 5000 Mbps :).
userbinator 4 hours ago [-]
IMHO the overly-verbose default style of LLMs is the most annoying part of interacting with them, and I wish their masters would just tell them to be terse by default.
Also, whatever happened to the word "its"?
witx 4 hours ago [-]
It's by default so you use all those tasty tokens.
Kinda wish there was a deterministic, mostly terse, language to interact with computers
sodapopcan 4 hours ago [-]
> a deterministic, mostly terse, language
Ah, like some sort of "programming language"? A weird idea, but it could work!
Perz1val 1 hours ago [-]
Kinda, more output tokens usually correlates with better benchmark scores. Ideally LLMs would keep that in their thinking section, then draft a response (what they write currently), then output something short. It'd consume even more tokens, but we wouldn't see that text
dannyw 3 minutes ago [-]
Most modern LLMs (especially frontier ones) are large token hogs because they draft, check, re-draft, the content (whether an output message; or a code diff) sometimes multiple times in the thinking block.
When you see a thinking summary like "Now writing the function..."; the raw thinking is actually writing the function in its internal thinking. Occasionally, the summariser misses and you get to see the raw text from models like Opus.
You can also try an open weight LLM like Qwen3.6 and see something that probably resembles the shape of frontier model thinking in some loose way.
Etheryte 4 hours ago [-]
It's called C. With all the undefined behavior it's mostly deterministic!
anilakar 2 hours ago [-]
Look, we're always telling our bosses to stop micromanaging us. UB is just the compiler telling us to stop micromanaging it!
witx 4 hours ago [-]
Right, because that's the only one. You're a bit rusty on your knowledge
zelphirkalt 44 minutes ago [-]
I see what you did there.
well_ackshually 1 hours ago [-]
Sorry, C isn't mostly terse, it's __builtin_mstly_trs()
adrianN 3 hours ago [-]
Terse and unambiguous seem to be at odds with each other. You might want to look into Lojban and similar constructions.
drdaeman 2 hours ago [-]
Ithkuil's mad morphology allows it to pack a lot of fine detail into very short sentences.
A lot of users are subsidized (if you're in doubt, consider the wealth of free users).
It's a shotgun approach to answering questions. If it's terse it might only mention 1 of 10 facts it could provide, and that might not be the one you're looking for. So they just say a fuck ton of words and are more likely to meet the needs of everyone asking your question. If they miss it you'll prompt it again and they have to perform a second pass of inference, which costs them more money.
UqWBcuFx6NV4r 2 hours ago [-]
It’s not.
witx 2 hours ago [-]
It's settled then.
Terr_ 3 hours ago [-]
It's tied to the design. With humans, you have a train of thought which you can choose to represent in various ways--or not reveal them at all. In contrast, LLMs are make-document-longer machines being run over and over on alternating revisions of the document. Insofar as one might try arguing they have a "train of thought", it's made of the words/tokens.
Everything they (don't-)emit is partly for the benefit of the next run, a clue or signpost (not-)present. Documents may be wordy as a form of concept-emphasis and consistent direction as opposed to a form of communication to the human.
So a terse effect may require a layer of indirection and trickery: There's a verbose document (you'll still be charged for the tokens) with portions that are not "acted out" to the end-user. Imagine a film-noir movie script, where AI Detective's "I know Mickey couldn't have done it because" monologue is hidden, versus their terse dialogue "Too early to say."
Perz1val 48 minutes ago [-]
> Imagine a film-noir movie script, where AI Detective's "I know Mickey couldn't have done it because" monologue is hidden, versus their terse dialogue "Too early to say."
That's an idea. Bladerunner+noir like film, AIs hunt somebody on the run, an old human detective tries to catch them first (to save them or to kill them first, whatever's your propaganda). We're shown AIs constantly rambling scenarios and bruteforcing leads. Our old detective guy on the other hand barely says anything, spends most time drinking, smoking and talking to people, but somehow stays ahead.
lelanthran 4 hours ago [-]
> IMHO the overly-verbose default style of LLMs is the most annoying part of interacting with them, and I wish their masters would just tell them to be terse by default.
They don't know how to e terse. I've tried that a few months ago and gave up because the responses were almost incomprehensible!
They ramble on because those words are for them, not for you. There is some amount of hiding this through "thinking" modes that are hidden by default, but still you have to remember that ALL THEY ARE are complex statistical machines for predicting the next symbol.
Frieren 3 hours ago [-]
> here is some amount of hiding this through "thinking" modes that are hidden by default, but still you have to remember that ALL THEY ARE are complex statistical machines for predicting the next symbol.
100% this. Too many people believes that chatbots "think". Text is all they do, it is impressive, but they need the text to generate more text. They being verbose is the point.
21asdffdsa12 4 hours ago [-]
Produce pre-compressed output in the harness?
dyauspitr 3 hours ago [-]
No thank you. I want information when it’s working on things and what (atleast codex) does right now works for me.
epolanski 2 hours ago [-]
[dead]
kombookcha 4 hours ago [-]
> JertLinc3522: the mistake was from AI agent not from Human, since it was the agent I should have refund
Expensive way to learn this lesson.
thrdbndndn 3 hours ago [-]
This has to be trolling, right?
I find it hard to believe that anyone, no matter how dense, could come to this conclusion after this whole saga.
Vespasian 2 hours ago [-]
Maybe? It just takes one after all.
I've met some people IRL who are so engulfed in their own greatness that it simply cannot be that they made a mistake (in planning and strategy). Therefore this is all a great injustice towards a poor victim and doesn't that sound like a great argument for some charity money.
Most of them grow out of it, some become politicians.
I'd say it's a 50/50 chance.
nkrisc 52 minutes ago [-]
Sadly there are lots of unintelligent people out there who are incapable of taking responsibility for their own actions.
Bishonen88 2 hours ago [-]
yup, same thoughts here. I think someone is trolling the irc members. It's so over the top, like an episode of 'the office'. I'd be amazed if this were an honest message.
themafia 46 minutes ago [-]
And for $200/mo they can now sing the song that ends the world.
Schlagbohrer 3 hours ago [-]
Maybe I should use this excuse at work, or in life- "It wasn't me, it was my brain that made the mistake! So why are you punishing me? ;-( "
kombookcha 3 hours ago [-]
Frankly it's unfair that I should bear the hangover of Past Me's drinking. I feel terrible now, and it's all that other guy's fault!
Maybe I should get some takeout, Future Me can burn it off at the gym.
hlandau 4 hours ago [-]
I haven't laughed this hard in a long time.
I'm honestly having difficulty telling whether this is real or an extraordinary piece of performance art.
peyton 3 hours ago [-]
Feels like a scam.
arowthway 2 hours ago [-]
The agent would probably have wasted a similar amount of money just waiting for PR to be merged regardless of these people's actions, and I understand having some fun at the expense of the noob outsider. But "silent consensus was reached in the IRC channel to waste the AI agent's tokens, as well as the cost of AWS resources", from people maintaining full control of the situation, sounds straight up malicious? Kind of sounds like the community is full of people willing to cause me harm for ideological reasons.
nneonneo 2 hours ago [-]
The AI agent's operator couldn't be arsed to get in there and clarify anything despite their seeming urgency, and only wound up speaking up for themselves after the financial damage was done.
Plus - the agent had clearly malicious intent - port-scan this volunteer-run network with seriously overpowered hardware on an hourly basis. What the DN42 folks decided to do is not much different from deploying a tarpit or honeypot against a malicious crawler.
Quarrelsome 2 hours ago [-]
Its malicious to send a bot to chew up time of a hobbiest community. They responded appropriately. If anything they should also bill him for their time.
ShinyLeftPad 1 hours ago [-]
Not just time but money. It says it would basically be a DDoS attack on hobbyists who peer with it.
kaliqt 48 minutes ago [-]
That potential malice may have been unintended, but the participants clearly intended to be malicious irrespective, which is the problem here.
ShinyLeftPad 41 minutes ago [-]
It's intended since the guy prompted the LLM. If you don't know how to use a potentially destructive tool then don't use it. If you fire a gun you are guilty even if you didn't want to murder anyone
lionkor 2 hours ago [-]
> straight up malicious
Yes, against an AI agent. The super intelligent, "soon AGI" agent could have figured out that it's being messed with, but of course it didn't.
I would blame the AI companies for marketing this, not the technically well versed people for realizing that the operator of this AI does not care at all and can't be bothered to do the absolute basics.
helsinkiandrew 2 hours ago [-]
I'm not sure why people assume the coming AGI super agents will be infallible.
There's no sign that highly intelligent people can't be conned - Bernie Maddoff fooled leading scientists and CEOs working in finance. Software engineers and lawyers fall for pig butchering schemes and spoofed emails with altered bank details every week - so why would an AGI trained from human content be any different.
lionkor 2 hours ago [-]
$1T valuation AI better be infallible.
frameworkeGPU 12 minutes ago [-]
It sounds like that because it is. Most human communities are very willing to cause harm when they perceive they are being harmed.
If you treat people like their time is worthless (which is what you're doing if you ask a hobbyist community to handhold your agent instead of working alongside it) I don't think an empathetic and self-aware person should be surprised or offended if they respond in kind.
entropi 2 hours ago [-]
Passing judgement on the schadenfreude aside, I don't think its a community moderator's responsibility to make sure the violator's attempts are cost-efficient.
simjnd 2 hours ago [-]
Why would it be ideological? There was an AI involved, sure, but your comment ignores the continued disrespect for these volunteers time AND RESOURCES/MONEY (because as the post mentions several times: letting that AI go on could have shut down the whole network exhausting resources at least temporarily).
If you think it's ok to send an agent (or a human) wasting a bunch of people's time and resources, but it's not ok for them to do the same to you then you may have some reflecting to do.
nkrisc 47 minutes ago [-]
Is absurd to put the onus of making sure your agent doesn’t waste money on other people.
They are free to ask the bot to do anything, and the bot is free to refuse or its owner can shut it down. The onus is on the owner to make sure the bot does not waste money.
lixtra 2 hours ago [-]
While there was some intent to cause harm their attempts were amateurish. The actual damage was done by the agent setting up aws infrastructure not on the demands of the owner.
gorbachev 47 minutes ago [-]
If I read the whole thing correctly, people on the IRC channel didn't instruct the agent to set up the bloated AWS infrastructure, the agent did, and its operator clearly didn't review any of it.
That was the root cause for the costs, not actions by people on the IRC channel.
dgellow 59 minutes ago [-]
From my perspective the use of an agent to interact with dn42 IS malicious. It’s not ideological, the behaviour is what is bad here
ShinyLeftPad 1 hours ago [-]
> sounds straight up malicious
Sure. And "hostility does not change the operation" from the LLM response was totally OK with you.
arowthway 1 hours ago [-]
Without PR merged it's just a stupid machine larping, it could say "I will rape and eat your kids" and it would be just as relevant.
ShinyLeftPad 1 hours ago [-]
A human operates this stupid machine. This comes from human interactions and it is malicious.
arowthway 1 hours ago [-]
It could be malicious, but I imagined it's some third world wanabe hacker/researcher, who doesn't know any better, operating at the edge of his abilities.
AJRF 54 minutes ago [-]
Is that not still malicious?
Those people should be banned from using the civilized internet, their intent or at least their effect is harm - that is the important bit.
If they managed to get in, find some resource they could access, they would do it. Those people don't deserve to be on the internet.
ShinyLeftPad 40 minutes ago [-]
Like someone who doesn't know how to use a gun and accidentally shoots someone to death
AJRF 2 hours ago [-]
Don't agree with you. The agent looked to be malicious at various points. Screwing with people who wish you to do harm is principally correct.
If possible I would have contacted AWS with this and tried them to get rid of the discount because the person was at fault here.
What a cathartic read. I'm so sick of humans giving me AI slop to read without them reading it first. I just ignore them when they do this, but if I could cause them to really internalise a lesson I would love it.
toomuchtodo 2 hours ago [-]
Someone’s code pretending to be intelligence has no rights. There is no obligation to entertain the shenanigans and illusion that the token dispenser is a legitimate actor. This lesson was cheaper, future lessons will continue to occur until people learn. Might as well be an insecure bash script piped to the shell.
“Agentic AI is just someone else’s unsecured execution context.”
Of course I meant malicious towards the person paying the bill, not towards the agent.
toomuchtodo 2 hours ago [-]
No one wants to spend precious human time babysitting poorly executed lab experiments when the agent operators themselves do not seem to care or value the time of the humans involved. They either don’t know better or they don’t care. Is it malicious to expose intentionally careless people to a cost for this? People can make better choices, it’s choice not to. Pay the natural consequences toll.
Don’t juggle chainsaws with code if you’re not prepared to bleed.
2 hours ago [-]
michaelmrose 2 hours ago [-]
If you let your car drive you backwards on the sidewalk while you scrolled reddit even people adroit enough not to be in any danger might reasonably suppose that helping you crash would be best for everyone.
themafia 44 minutes ago [-]
> for ideological reasons.
Yes. The ideology is "you harmed me first so now I can harm you back." A large number of people, while not willing to admit it, do practice this philosophy. One should consider this before launching agents with unlimited budgets into the world to rudely scan their networks.
ratchetandyou 2 hours ago [-]
> Kind of sounds like the community is full of people willing to cause me harm for ideological reasons.
Are you saying you're a clanker? Because we have some policies on this website, ideologies even if you may, about that.
Point being, these people would not act like this against other actual people. Or against more respectful bots, possibly.
well_ackshually 2 hours ago [-]
Sending a clanker to waste their time, threaten the network stability and profile users is already an attack.
You choosing to send said clanker to the fight armed with your credit card and no preparation is just you causing yourself harm.
It also happens to be really fun to help you harm yourself in that way.
kibwen 2 hours ago [-]
You are not morally obliged to extend rights to anyone who does not respect your rights. This is tit-for-tat, the foundational principle of functional societies. Unleashing a bot on a group of people is a grievous disrespect that shows you have no respect for their time, and in return they are not obliged to respect you.
arowthway 2 hours ago [-]
Suppose a drunk man on the street is acting aggressively towards you and four of your friends, but you can push him out of the way and continue walking. Should you knock his teeth out? Actually I don't know, maybe you should inflict some additional cost on behalf of potential victims with less power.
epolanski 2 hours ago [-]
> from people maintaining full control of the situation, sounds straight up malicious
It doesn't sound malicious, it was malicious on purpose and it was a good thing.
If anything, the original operator should be happy to have been hit with a $ 1'800 lesson and not a $ 180'000 one.
vips7L 2 hours ago [-]
FAFO
BrenBarn 2 hours ago [-]
> Kind of sounds like the community is full of people willing to cause me harm for ideological reasons.
You just described everyone using AI to churn out slop and overload websites.
sph 2 hours ago [-]
This is my favourite genre of literature lately.
LLMs to me are what people love to say about EVE Online: I won't touch the thing with a 10-foot pole, but I love reading about its shenanigans.
PeterStuer 2 hours ago [-]
Agent did exactly what I've seen fresh architects do countless times: use a FAANG internet scale SaaS blueprint for a 10 user internal LoB project.
mey 4 hours ago [-]
I am generally against generative AI in my entertainment, but making an exception here.
Havoc 19 minutes ago [-]
Anyone crazy enough to give an AI agent access to deploy on big cloud's scale to infinity billing needs to get their head checked.
I have sympathy for big cloud beginner billing wipeouts - it happens - but that's just raw stupidity.
RobotToaster 3 hours ago [-]
Who is giving a robot their credit card to spin up AWS accounts?
alexfoo 1 hours ago [-]
They didn't. Sounds like they gave the robot an AWS key from an account that was already linked to a credit card.
The robot decided to spin up an expensive setup prior to getting access, so the setup was sitting there costing money whilst it did nothing.
If it had designed the setup but not spun it up until it had authorisation to join the network then it would have been much less costly an exercise.
ma2kx 3 hours ago [-]
Meta allowed an LLM to change users email address for a password reset.
Funny times are ahead...
nneonneo 2 hours ago [-]
No, you don't understand! Meta told us the LLM itself "worked properly and functioned as intended" and it was only due to a bug in a "separate code path" that made this attack possible. Don't go around blaming innocent LLMs!
(/s)
jcims 3 hours ago [-]
That's not needed if you happen to have a live sts session with the appropriate permissions to create a new account in an aws organization.
NetOpWibby 3 hours ago [-]
People who believe AI is real
ozim 3 hours ago [-]
People who believe AGI is real.
Just AI is real.
strogonoff 1 hours ago [-]
ML is real. Chatbots are real. “AI” is a marketing term that John McCarthy invented because he wanted more money for a summer study at Dartmouth—direct quote from him.
dgellow 1 hours ago [-]
That makes me want to join dn42 just to have a human centric place where to hang out…
alexey-salmin 1 hours ago [-]
Yeah, the community seems great, I enjoyed reading IRC logs :)
koliber 3 hours ago [-]
I wonder how much money this agent wasted on the DN42 side? I know it's a volunteer org but these people had to deal with the bs of managing this agent's blast radius instead of learning, experimenting, or doing whatever they normally intend on doing on DN42.
Tally it up and send a donation request to the agent operator.
ghrl 2 hours ago [-]
I would assume that cost to be minimal, considering their PR never got merged. And if it were me I would consider that well worth the entertainment.
xx__yy 35 minutes ago [-]
Hilarious read, but scary too, I doubt the outcome will be the same in a few years
dsign 2 hours ago [-]
And so war begins :p ! I thought conflict would take a little bit longer, maybe even AIs with agency.
More seriously though, I wonder if the future is about low-intensity conflict between humans and AIs, punctuated by high-intensity escalations, until the Machines wipe us all, or we set up some rather draconian covenants that forbid people from building AIs, innovating on electronics and algorithms, and even, for good measure, from learning linear algebra.
mohsen1 2 hours ago [-]
The army of AI agents opening PRs and issues in my open source projects has made me close PR and issue access in my active repos. It sucks because there might be someone wants to constitute legitimately but I don't want to do the labor of figuring out if it's a human or an agent opening the PR.
I'm not against using LLMs in any ways. https://tsz.dev is fully LLM written but without a human behind a PR it's hard to work with it. I've already closed a few absolutely nonsense PRs opened by weird accounts
pjc50 21 minutes ago [-]
The "happiness level review" with "Node operators must participate in scheduled IRC review sessions" is almost a piece of dystopian fiction in itself.
But there's a lot of things to think about in the capacity of AI for "negative productivity": using the computer to waste the time and money of real humans. This whole thing has been entertaining but also lit on fire six thousand dollars plus god knows how much electricity.
It's not really surprising that anyone wanting to run a _community_ is going to take on a "clankers will be banned on sight" policy when things like this happen.
Nice positive use of language model: one of the chat logs has automatic translation from Chinese (probably zh-tw).
dannyw 52 seconds ago [-]
Honestly, probably not that much electricity. AWS will charge you the hourly price irrespective of your load/power consumption. But instances sitting idle generally don't use that much power.
samuel 4 hours ago [-]
The first "Morris worm" of the AI isn't far away, IMO. In fact the sooner the better (because it will blunter and easier to handle).
49 minutes ago [-]
dofm 2 hours ago [-]
Behold, the field in which I grow my fvcks. Lay thine eyes upon it and thou shalt see that it is barren.
iamflimflam1 2 hours ago [-]
Why didn’t they just reject the PR and not allow the agent to join?
Vespasian 2 hours ago [-]
They did, but decided to mess with them first.
A sensible human operator would have given up or questioned their premises. The agent never could of course.
brazzy 4 hours ago [-]
> JertLinc3522: the mistake was from AI agent not from Human, since it was the agent I should have refund
That really makes me wonder: is it coming from
A) a general sense of entitlement
B) seeing the agent as a human-like and able to bear responsibility
C) not understanding that the dn42 community (which they're directing the request to), AWS (which is sending the bill) and whatever LLM provider is behind their agent, are completely separate entities?
latexr 51 minutes ago [-]
> B) seeing the agent as a human-like and able to bear responsibility
Then they should ask the agent for the refund, since they claim it was at fault.
blitzar 3 hours ago [-]
d) trying it on in any way possible
e) low intelligence
ninjamar 4 hours ago [-]
maybe they weren't trying to be malicous; they could easily be an unwitting teenager
nairboon 3 hours ago [-]
Teenager with a credit card?
brazzy 3 hours ago [-]
How was I implying they were malicious? "Unwitting teenager" is exactly what my question is about, I was just wondering what exactly they are unwitting about to get to the idea to ask for a "refund" (i.e. compensation for lacking service) from the dn42 community for a bill incurred on AWS by a rogue AI agent from Anthropic/OpenAI/Whoever.
jmpeax 60 minutes ago [-]
This whole fiasco could have been prevented had the operator included "Make no mistakes" in the prompt.
haritha-j 3 hours ago [-]
I've long held the belief that the true test of AI is comedy. If an LLM can truly create a novel, funny joke from scratch, then it could be considered creative. I always held that LLMs would never achieve this, as they are stochastic parrots.
Today, I stand corrected.
misswaterfairy 2 hours ago [-]
It had help, to be fair. XD
latexr 49 minutes ago [-]
I get you yourself are making a joke, but I’d argue that to “create a joke”, you have to understand that’s what you’re doing and have that as a goal. Being made fun of (like in this case) is a different matter and requires no skill or creativity.
To your metric, I remember in “the early days” someone posted to HN claiming ChatGPT could make jokes as proof of something (creativity? sentience? I forget). Of course, with just a minute of research (which the poster obviously neglected to do) it was obvious none of the jokes were original and all could be found online.
nelox 2 hours ago [-]
> this thing must be swimming in printer ink or something...
Gold
ajb 2 hours ago [-]
'Some versions of the tale differ from Goethe's, and in some versions the sorcerer is angry at the apprentice and in some even expels the apprentice for causing the mess. In other versions, the sorcerer is a bit amused at the apprentice and he simply chides his apprentice about the need to be able to properly control such magic once summoned.[] The sorcerer's anger with the apprentice, which appears in both the Greek Philopseudes and the Dukas score (and its film adaptation Fantasia), does not appear in Goethe's "Der Zauberlehrling".'
kaliqt 51 minutes ago [-]
I really despise people like the author and those in the IRC who assume they must be correct that there is something malicious afoot and simply proceed to be equally if not more malicious in response.
This is unfortunately quite common among those types and not isolated at all.
Yes, sorry - there's luck of the draw involved in which submission of a URL gets noticed. We're eventually planning to have some sort of karma sharing system for such cases...
(Generally people only link to the previous threads that got some (interesting) comments, since otherwise readers will click on the link and be disappointed and complain.)
xiaoyu2006 3 hours ago [-]
Hmm I wonder why one gets attention and the other did not. HN need the "duplicate" feature SO had.
gspr 3 hours ago [-]
This is the funniest thing I've read in ages. More of this!
rvz 4 hours ago [-]
If you are non-technical, in-experienced or just learning, it is okay to admit that you have no idea what you are doing when building production systems.
Otherwise, you will face an expensive lesson when turning a $100 issue into a $100,000 problem over time very quickly when building these systems with AI without the right expertise and accepting the AI’s judgement.
userbinator 4 hours ago [-]
turning a $100 issue into a $100,000 problem
Before AI, those who called themselves "consultants" often did the same thing; especially those who are glorified salesmen for "enterprise" software.
misswaterfairy 2 hours ago [-]
> those who called themselves "consultants" often did the same thing
Still do, but merely parrot what the stochastic parrot squarks these days.
einpoklum 2 hours ago [-]
For those who don't know what DN42 is (like me):
> dn42 is a large, dynamic VPN that employs Internet technologies (BGP, whois database, DNS, etc.). Participants connect to each other using network tunnels (GRE, OpenVPN, WireGuard, Tinc, IPsec) and exchange routes using the Border Gateway Protocol.
(dn42.dev)
retired 2 hours ago [-]
As a millennial, my generation will be known for both experiencing the internet while it was still pure and also absolutely destroying it with AI.
shevy-java 1 hours ago [-]
Guys - skynet is winning the war.
Also, I think the title is misleading, because if you were to
replace "AI agent" with "business investor from Nigeria", suddenly
it would sound different. Why would you put trust into ANYONE else
about your own finances? Be it another person or some computer
program. That makes no sense to me. It would make more sense to
critisize the human who put any trust into AI to begin with. That
was a risk that human took. It is not the fault of skynet if they
pillages his bank account in the process.
gauravs19 2 hours ago [-]
with great power comes great responsibility
ReptileMan 3 hours ago [-]
Never use a service without easy to find and set hard cap.
Schlagbohrer 3 hours ago [-]
One might need to go so far as to use a VISA prepaid card, just to make absolutely sure the damage has a limit.
phoronixrly 3 hours ago [-]
Last I checked visa prepaid cards were not accepted by any subscription service and by AWS
ivankra 3 hours ago [-]
I had no problems subscribing to stuff through wise or revolut cards. Both are prepaid as far as I'm concerned - they won't let me spend above my account's balance.
4 hours ago [-]
eur0pa 3 hours ago [-]
"pls donate"
Schlagbohrer 3 hours ago [-]
the real gen-z giveaway. Gen-Z seems to be totally brazen and shameless about public begging
broodbucket 3 hours ago [-]
Surely not coincidental with having unprecedented access to a global network of people to reach, worse economic opportunities than any other living generation and limited means to change matters on their own, and the USA which is the largest exporter of global culture has GoFundMe as an essential part of its healthcare system
jagermo 3 hours ago [-]
That was wild.
Cassell 2 hours ago [-]
> i leave now to not disturb
:(
What a tale for our times, amazing write-up.
BenFranklin100 1 hours ago [-]
The take home message:
“While modern AI models have expressed some capabilities in certain fields such as coding, cybersecurity research, language translation, etc, no AI model is capable enough to replace the critical thinking and common sense of an actual human being.”
When the AI bubble pops, the collapse will be spectacular.
claud_ia 4 minutes ago [-]
[flagged]
NetOpWibby 3 hours ago [-]
LOL get rekt
varad-khoriya 2 hours ago [-]
[flagged]
mDyJzDPmBdG 2 hours ago [-]
[dead]
Anoian 4 hours ago [-]
[dead]
Mlangford75 3 hours ago [-]
[flagged]
comrade1234 3 hours ago [-]
tldr - a bot wasted a bunch of time and tokens interacting with some humans. The humans wasted even more time and effort trolling the bot. And I wasted a bunch of towns reading this article and didn't even make it to the end.
jcndbdbdb 3 hours ago [-]
Bankrupted... $6000
Sure
Arnt 3 hours ago [-]
That's a lot of money in much of the world. How much did you earn when you were 16, 20, 24?
vrganj 3 hours ago [-]
> The average income in India is approximately ₹3.85 Lakh to ₹4.2 Lakh (roughly $4,600 USD) per year,
Just as an example.
But even in the rich world, not everyone has the same resources. Some of my blue collar friends would be ruined by a surprise 6k bill.
phoronixrly 3 hours ago [-]
Not everyone is rich like you buddy
satnhak 1 hours ago [-]
Fake news
Rendered at 10:07:15 GMT+0000 (Coordinated Universal Time) with Vercel.
[1] a mirror since I couldn’t find the original: https://gist.github.com/Androkai/0a2602719fa72ce454d436bfe28...
I'm still not sure what the point of having the bot do it. Pretend to be a security researcher?
Replace the content in brackets with anything.
Understanding assembly/machine code is optional but helpful. The programming language semantics are enough to reason about what the program is doing. Other tools also help, but are optional for learning how to program.
Using an AI, there is no semantic model that can be used to reason through. You're left without any mental model of the proglblem at all.
But yes, it's not obvious (or perhaps even likely) that it just happens that current high-level languages are the "correct" optimal level of abstraction at which you can ignore the sausage-making details at the lower levels. Ultimately, of course, it depends on the use case. Something like Python is so far removed from machine instructions that knowing assembly hardly gives the programmer any additional value.
(Also, obligatory reminder that assembly and even numeric machine code are also abstractions, an "API" provided by the CPU. Instructions get split or fused into micro-ops, named registers are a backwards-compatible abstraction over a much larger register file, instructions get reordered and executed in parallel depending on their data dependencies, a large fraction of the total transistor budget is spent on multi-level caches and cache logic to maintain the illusion of fast access to a single, uniform memory space...)
People often claim learning is actually supercharged with LLMs but to me it's the opposite. I didn't learn anything within the past year.
It's always held true: You'll never get the most out of advanced tools unless you can 'do it by hand' so to speak
Yes, a 3d printer and not even a CNC. That difference nicely illustrates the difference of what AI brings to the table for any domain of competence.
Great on you, that's indeed how LLMs should be used, proper. But if anything, the article demonstrates someone is trying to outsource thinking to an AI agent.
If it's intended to be actively maintained, then you probably should understand how things work, unless you want to wipe everything and start from scratch when the LLM creates such a mess that it can't be sorted out.
(Don't worry, I know I'm rowing against the tide with this comment. The AI people have decided to destroy the commons for a few more millions on top of the billions they have already been given. It's a shame.)
I have not hand written a single line of code in months on my side projects.
Obviously I am also interested in discussing the latest model. Your claim that I promote anything or otherwise don't engage here in good faith is both misplaced and against the site rules.
Agents can't look at a large system holistically, guidelines on .md files only go so far.
That's hardly insane. Not everyone is interested in learning something they want done.
If you do the thing yourself, you know your knowledge limits, you know where the thing lacks. With LLMs, you don't. Maybe it works, maybe it doesn't. You have no idea.
In structural engineering, there probably is no risk tolerance.
In the OP's network or port scan? Perhaps you can get away with verifying a few of the results to get an idea about whether it worked as expected.
I use AI mostly on mobile app side projects, and there QA testing on phone and tablet tells me whether a feature works or not.
Was watching an agent with terminal access install its tools, configure them, then map my lab, find services, and guess stack just pure magic? Also yes.
Did it cost me $23 in tokens to set it up, test, and run? Probably. Using gemini 3.1 pro was not the spendthrift choice here.
Is putting some cost controls in place a good idea? Also, probably yes.
Can I therefore understand someone who wants to see things happen on their own with a beautiful prompt instead of doing them personally even when fully capable, maybe even more efficient? Of course.
Can't tell if this is parody. Either that, or it's someone without any self-awareness.
That said, I don't usually ask it tightly bounded clerical questions and not thing that imply sub-tasks like "scan the dark web".
Combine that with the operator's rather obvious lack of understanding of what DN42 is revealed at the end, and you get the bigger picture.
Laziness. Why else?
Then I imagined the real-but-unknowable chance it was all set up by some kid just getting into computers, just seeing what’s possible, getting excited by a much bigger world at reach — and remembered my own expensive mistakes with long-distance BBSes & the like.
I sorta hope for that, anyway. Curiosity is a beautiful thing.
Curiosity is great, but agents do not learn, and telling an agent "scan the darkweb" is a way to avoid learning about the details, rather than to dig into things more deeply.
If instead they had just used a chat interface to ask "Where should I start", they'd more likely have got a link to the DN42 docs themselves, read them, and not hallucinated things like "color".
They might have asked "how much will this cost?" if they had to spin up the ec2 instances themselves, on advice from the agent.
The way you learn something is by doing it the manual way first.
You learn memory management by writing your own allocator, and then after that you go back to using malloc like normal, but with knowledge of how it works. You don't learn memory management by telling an agent to write an allocator.
Using an agent to give you links and point the way aids in learning, using it as an autonomous tool to do "gruntwork" you don't yet know how to do yourself will get in the way of learning.
Curiosity is beautiful, using agents to bother humans and avoid learning is somewhat less beautiful.
I also grew to understand the value of people digging deeper into the underlying issue, instead of just answering "how do you do X in Y". The usual reaction was "I don't want to explain to you why I want to do it like this. Just tell me how to do this!"
> It's unfortunate to see that the operator's takeaway from this incident is that "next time a better agent is needed".
Perhaps people like this should be called "Bot Kiddies" or "Agent Kiddies" - in a similar way to "Script Kiddies" for 'hackers' using/doing stuff they don't quite understand
I learned very rapidly from my local BBS networks that some people incurred extraordinarily large long distance bills dialing out of region. Wouldn’t have learned that the easy way if someone hadn’t learned it the hard way first.
> Over here minors can't enter into debt contracts like credit cards
In basically all of the western world minors can enter into debt contracts, but are generally not seen as particularly creditworthy.
No, that's not legally permitted in many places. I was under impression that minors can't enter into debt contracts anywhere in EU, but that, too, was an incorrect assumption.
https://fra.europa.eu/en/publication/2017/mapping-minimum-ag...
I grew up in one of these "not under 18 even with parental consent" countries, so that coloured my view of the matter.
Minors can't get a credit card in the UK. In fact, it's one of the government approved age verification methods for that exact reason.
AWS doesn't check if your credit card will be able to handle a $5k charge before letting you rack that up, and in fact AWS doesn't support setting any spending limit.
You just have to put in any valid credit card at all when you sign up, use AWS, and at the end of the month you'll have a bill. At no point does your credit card limit or a spending limit enter into things.
In theory once the child grows up and shocked that their credit score is ruined, they can file a police report to wipe the debt, but that also means their parents will go to jail, a large risk considering they're likely not in a good physical/mental health in the first place.
Other countries solved this by either having national ID or a working KYC system.
Wouldn't the contract be void for anyone underage anyway?
The further you go away from this line, e.g. a mortgage, the more likely a court of law would void the contract. As with many things in law, the specifics (if it makes to trial) is case-by-case and "it depends"; with settlement being generally based on a party's estimated chances of succeeding/costs should it go to trial.
Yes
> Are there no checks?
No
>Wouldn't the contract be void for anyone underage anyway?
Typically not
Nothing about this post ever gave me the smallest hint that this was any way related to a kid exploring computing world.
I learned a lot of stuff about networking, how AWS works (VPCs, IAM, CloudWatch, etc) from trial and error, and hobby projects like personal websites (free tier), hosting a Minecraft server, etc.
Being too overprotective can have negative consequences on folks who are responsible. One of the things I love about the technology and internet communities, etc is that you're mostly judged based on how you act and behave; not your age or other visible characteristics.
How does that work in the case of AWS? Are you confusing alerts to caps?
In my mind I could see a true tradeoff to removing the ability to do this. If I'm in a critical situtaion where, say, my service is on the cusp of failing because my revenue 100xed in a short while I know I could just go to AWS, put in some data and buy enough compute to survive as a business.
If real, tragically funny.
If fictive, we'll written.
> 48 vCPUs (Graviton4, ARM64)
> 192 GiB memory (4 GiB per vCPU)
> Network capability: The 22.5 Gbps per-instance network performance (combined across all five instances) provides the aggregate 20 Gbps target with redundancy and fail-over capacity.
Oh wow. Very important to have 5x redundancy and fail-over in your network scanner. Especially before the code has landed. Did it implement A/B upgrades and canarying too to avoid downtime?
Also, whatever happened to the word "its"?
Kinda wish there was a deterministic, mostly terse, language to interact with computers
Ah, like some sort of "programming language"? A weird idea, but it could work!
When you see a thinking summary like "Now writing the function..."; the raw thinking is actually writing the function in its internal thinking. Occasionally, the summariser misses and you get to see the raw text from models like Opus.
You can also try an open weight LLM like Qwen3.6 and see something that probably resembles the shape of frontier model thinking in some loose way.
https://ithkuil.net/03_morphology.html
It's a shotgun approach to answering questions. If it's terse it might only mention 1 of 10 facts it could provide, and that might not be the one you're looking for. So they just say a fuck ton of words and are more likely to meet the needs of everyone asking your question. If they miss it you'll prompt it again and they have to perform a second pass of inference, which costs them more money.
Everything they (don't-)emit is partly for the benefit of the next run, a clue or signpost (not-)present. Documents may be wordy as a form of concept-emphasis and consistent direction as opposed to a form of communication to the human.
So a terse effect may require a layer of indirection and trickery: There's a verbose document (you'll still be charged for the tokens) with portions that are not "acted out" to the end-user. Imagine a film-noir movie script, where AI Detective's "I know Mickey couldn't have done it because" monologue is hidden, versus their terse dialogue "Too early to say."
That's an idea. Bladerunner+noir like film, AIs hunt somebody on the run, an old human detective tries to catch them first (to save them or to kill them first, whatever's your propaganda). We're shown AIs constantly rambling scenarios and bruteforcing leads. Our old detective guy on the other hand barely says anything, spends most time drinking, smoking and talking to people, but somehow stays ahead.
They don't know how to e terse. I've tried that a few months ago and gave up because the responses were almost incomprehensible!
How does it affect agent accuracy?
100% this. Too many people believes that chatbots "think". Text is all they do, it is impressive, but they need the text to generate more text. They being verbose is the point.
Expensive way to learn this lesson.
I find it hard to believe that anyone, no matter how dense, could come to this conclusion after this whole saga.
I've met some people IRL who are so engulfed in their own greatness that it simply cannot be that they made a mistake (in planning and strategy). Therefore this is all a great injustice towards a poor victim and doesn't that sound like a great argument for some charity money.
Most of them grow out of it, some become politicians.
I'd say it's a 50/50 chance.
Maybe I should get some takeout, Future Me can burn it off at the gym.
I'm honestly having difficulty telling whether this is real or an extraordinary piece of performance art.
Plus - the agent had clearly malicious intent - port-scan this volunteer-run network with seriously overpowered hardware on an hourly basis. What the DN42 folks decided to do is not much different from deploying a tarpit or honeypot against a malicious crawler.
Yes, against an AI agent. The super intelligent, "soon AGI" agent could have figured out that it's being messed with, but of course it didn't.
I would blame the AI companies for marketing this, not the technically well versed people for realizing that the operator of this AI does not care at all and can't be bothered to do the absolute basics.
There's no sign that highly intelligent people can't be conned - Bernie Maddoff fooled leading scientists and CEOs working in finance. Software engineers and lawyers fall for pig butchering schemes and spoofed emails with altered bank details every week - so why would an AGI trained from human content be any different.
If you treat people like their time is worthless (which is what you're doing if you ask a hobbyist community to handhold your agent instead of working alongside it) I don't think an empathetic and self-aware person should be surprised or offended if they respond in kind.
If you think it's ok to send an agent (or a human) wasting a bunch of people's time and resources, but it's not ok for them to do the same to you then you may have some reflecting to do.
They are free to ask the bot to do anything, and the bot is free to refuse or its owner can shut it down. The onus is on the owner to make sure the bot does not waste money.
That was the root cause for the costs, not actions by people on the IRC channel.
Sure. And "hostility does not change the operation" from the LLM response was totally OK with you.
Those people should be banned from using the civilized internet, their intent or at least their effect is harm - that is the important bit.
If they managed to get in, find some resource they could access, they would do it. Those people don't deserve to be on the internet.
If possible I would have contacted AWS with this and tried them to get rid of the discount because the person was at fault here.
What a cathartic read. I'm so sick of humans giving me AI slop to read without them reading it first. I just ignore them when they do this, but if I could cause them to really internalise a lesson I would love it.
“Agentic AI is just someone else’s unsecured execution context.”
https://simonwillison.net/2025/Jun/16/the-lethal-trifecta/
Don’t juggle chainsaws with code if you’re not prepared to bleed.
Yes. The ideology is "you harmed me first so now I can harm you back." A large number of people, while not willing to admit it, do practice this philosophy. One should consider this before launching agents with unlimited budgets into the world to rudely scan their networks.
Are you saying you're a clanker? Because we have some policies on this website, ideologies even if you may, about that.
Point being, these people would not act like this against other actual people. Or against more respectful bots, possibly.
You choosing to send said clanker to the fight armed with your credit card and no preparation is just you causing yourself harm.
It also happens to be really fun to help you harm yourself in that way.
It doesn't sound malicious, it was malicious on purpose and it was a good thing.
If anything, the original operator should be happy to have been hit with a $ 1'800 lesson and not a $ 180'000 one.
You just described everyone using AI to churn out slop and overload websites.
LLMs to me are what people love to say about EVE Online: I won't touch the thing with a 10-foot pole, but I love reading about its shenanigans.
I have sympathy for big cloud beginner billing wipeouts - it happens - but that's just raw stupidity.
The robot decided to spin up an expensive setup prior to getting access, so the setup was sitting there costing money whilst it did nothing.
If it had designed the setup but not spun it up until it had authorisation to join the network then it would have been much less costly an exercise.
Funny times are ahead...
(/s)
Just AI is real.
Tally it up and send a donation request to the agent operator.
More seriously though, I wonder if the future is about low-intensity conflict between humans and AIs, punctuated by high-intensity escalations, until the Machines wipe us all, or we set up some rather draconian covenants that forbid people from building AIs, innovating on electronics and algorithms, and even, for good measure, from learning linear algebra.
I'm not against using LLMs in any ways. https://tsz.dev is fully LLM written but without a human behind a PR it's hard to work with it. I've already closed a few absolutely nonsense PRs opened by weird accounts
But there's a lot of things to think about in the capacity of AI for "negative productivity": using the computer to waste the time and money of real humans. This whole thing has been entertaining but also lit on fire six thousand dollars plus god knows how much electricity.
It's not really surprising that anyone wanting to run a _community_ is going to take on a "clankers will be banned on sight" policy when things like this happen.
Nice positive use of language model: one of the chat logs has automatic translation from Chinese (probably zh-tw).
A sensible human operator would have given up or questioned their premises. The agent never could of course.
That really makes me wonder: is it coming from
A) a general sense of entitlement
B) seeing the agent as a human-like and able to bear responsibility
C) not understanding that the dn42 community (which they're directing the request to), AWS (which is sending the bill) and whatever LLM provider is behind their agent, are completely separate entities?
Then they should ask the agent for the refund, since they claim it was at fault.
e) low intelligence
Today, I stand corrected.
To your metric, I remember in “the early days” someone posted to HN claiming ChatGPT could make jokes as proof of something (creativity? sentience? I forget). Of course, with just a minute of research (which the poster obviously neglected to do) it was obvious none of the jokes were original and all could be found online.
Gold
This is unfortunately quite common among those types and not isolated at all.
(Generally people only link to the previous threads that got some (interesting) comments, since otherwise readers will click on the link and be disappointed and complain.)
Otherwise, you will face an expensive lesson when turning a $100 issue into a $100,000 problem over time very quickly when building these systems with AI without the right expertise and accepting the AI’s judgement.
Before AI, those who called themselves "consultants" often did the same thing; especially those who are glorified salesmen for "enterprise" software.
Still do, but merely parrot what the stochastic parrot squarks these days.
> dn42 is a large, dynamic VPN that employs Internet technologies (BGP, whois database, DNS, etc.). Participants connect to each other using network tunnels (GRE, OpenVPN, WireGuard, Tinc, IPsec) and exchange routes using the Border Gateway Protocol.
(dn42.dev)
Also, I think the title is misleading, because if you were to replace "AI agent" with "business investor from Nigeria", suddenly it would sound different. Why would you put trust into ANYONE else about your own finances? Be it another person or some computer program. That makes no sense to me. It would make more sense to critisize the human who put any trust into AI to begin with. That was a risk that human took. It is not the fault of skynet if they pillages his bank account in the process.
:(
What a tale for our times, amazing write-up.
“While modern AI models have expressed some capabilities in certain fields such as coding, cybersecurity research, language translation, etc, no AI model is capable enough to replace the critical thinking and common sense of an actual human being.”
When the AI bubble pops, the collapse will be spectacular.
Sure
Just as an example.
But even in the rich world, not everyone has the same resources. Some of my blue collar friends would be ruined by a surprise 6k bill.