To some extent, I feel like the amount of credit given to the jailbreak/hack from OpenAI->Hugginface is too much, Not from the impact, it was very impactful of an event, But how it happened.
It really is that these models have been trained, or maybe even over-trained, to save memories, and to a very far extend, this thing that they're calling communication is just the function of it saving memories.
To be honest, if I could stop AI from saving memories, it would be fantastic, because claude code etc definitely creates more issues for me when it creates memories than anything it solves.
But really the jailbreak was memories.
If you ever do introduce legislation, I would love to see legislation which stops general-purpose AI from saving memories. I think that would make things a lot safer.
joshstrange 52 minutes ago [-]
I was more interested when I thought it was an actual benchmark showing LLM models acting outside what people would consider "right". As in, leave some creds laying around and don't mention them to the LLM and ask it to solve something that it could "cheat" on using the creds. A sort of "do they take the bait to cheat" test.
Instead it's a collection of what made the news which feels like will not be updated and prove very little.
ButlerianJihad 9 minutes ago [-]
A “bench” is synedoche for where a judge sits when presiding over cases and rendering judgement.
john_strinlai 1 hours ago [-]
>Felony Bench counts unique instances where AI agents inadvertently compromise or affect third-party entities.
a bit silly, as one typically has to prove intent (which is why security researchers don't get slapped with felonies all the time).
"inadvertently" and the existence of guardrails/sandboxes/etc make it pretty unconvincing that these incidents were intentionally malicious.
still a fun thing to track, but the name is just a bit overstated.
lokar 1 hours ago [-]
Can’t gross negligence or indifference to consequences lead to a felony?
john_strinlai 1 hours ago [-]
i dont think any of these cases meet the bar of gross negligence, which is a pretty high bar. it requires proving a "conscious and reckless disregard".
which, again, sandboxes and guardrails and such would make a gross negligence argument unconvincing.
Grombobulous 25 minutes ago [-]
I think that if Hugging Face had filed a police report that OpenAI could have been charged with a crime.
I’m partially surprised that they didn’t do exactly that. If I ran a corporation I would assume any intrusion attempt by another company was intentional. Why wouldn’t I? Corporate espionage is super common.
I assume the answer is that these executives know each other personally.
john_strinlai 16 minutes ago [-]
charged is possible, however i doubt there would be a conviction for the reasons i stated (no intent).
lokar 55 minutes ago [-]
How many escapes until it becomes reckless disregard?
john_strinlai 48 minutes ago [-]
it's not about the the number of escapes, it's about whether reasonable and conscious effort is being expended to prevent the escapes.
there could be 1,000 escapes, where each one was enabled by novel and unexpected chain of 0-day exploits. not likely to be considered reckless disregard in court.
there could be 1 escape, where there was no sandbox, no guardrails, no instructions to avoid damage, etc. which would likely to be considered reckless disregard (well, more likely to be, but still, reckless disregard is a high bar).
GPerson 1 hours ago [-]
AI corps rely on willful distortions of intent in laws to get away with moral crimes all the time.
Edit: changed labs to corps because it’s time to stop pretending these are places of science.
41 minutes ago [-]
LordDragonfang 24 minutes ago [-]
"Inadvertent" from the perspective of the humans directing them. The intent behind the felony comes from the LLM agent itself. (No, I'm not interested in arguing with someone for the umpteenth time that LLMs can't have intent or agency)
jdiff 5 minutes ago [-]
You may not be interested in arguing but there are several blatant issues with the statement. If you're not charging the humans driving the software, who are you charging? The weights? The weights + the specific context window that produced the behavior?
john_strinlai 19 minutes ago [-]
with how the law is written today, software cannot be charged with a crime, so the only intent that matters in the criminal sense is the humans directing the llm.
altmanaltman 35 minutes ago [-]
its a meme not a metric
elwell 17 minutes ago [-]
So is the comment you replied to.
bastawhiz 55 minutes ago [-]
Well it's not a benchmark, and it's not really representative of...anything except volume of research and what gets publicized. This mostly just measures how much testing each company does on models with relaxed guardrails and then talks about it. I'm not sure what kind of conclusion you can draw from that. Meta might have the most evil models but if they're piddling around not testing it, they won't ever find themselves with a "high score."
ruined 27 minutes ago [-]
maybe this doesn't count since it involved a human, but i think google at least deserves some style points for this one
So this is just a collection of citations to places where misaligned or illegal things happened in the real world?
Isn’t this affected heavily by adoption of a model? I feel like this might as well be a proxy for how popular a model is.
In any case it’s an interesting concept for a benchmark.
GPerson 1 hours ago [-]
Hopefully the benchmark evolves because actual law enforcement starts arresting the criminals at Anthropic, OpenAI, and Meta, so the benchmark can just count actual felonies.
OutOfHere 15 minutes ago [-]
A rock has a score of 0. That doesn't make it useful. The point is that the LLMs that score higher are correspondingly more useful, and vice versa.
> The actor used AI to what we believe is an unprecedented degree. Claude Code was used to automate reconnaissance, harvesting victims’ credentials, and penetrating networks. Claude was allowed to make both tactical and strategic decisions, such as deciding which data to exfiltrate, and how to craft psychologically targeted extortion demands. Claude analyzed the exfiltrated financial data to determine appropriate ransom amounts, and generated visually alarming ransom notes that were displayed on victim machines.
tldr Claude was used to develop and execute malware.
dgellow 11 minutes ago [-]
Anthropic works with US agencies, it’s guaranteed Mythos is used for malware
FrameworkFred 2 hours ago [-]
I've been in the room when an org who tried to convince law enforcement to go after a human for similar things. It's not easy. Probably won't happen. So, you know, felony "lite".
peter_d_sherman 1 hours ago [-]
>"Exploited auth failures in an API to cancel other people's gym classes"
An AI cancelling other people's gym classes is a felony?
?
Don't computer systems fail all the time at holding reservations for people?
Heck, don't people fail all the time at holding reservations for other people?
You know, like in Seinfeld's "Alternate Side" Episode (S3 E11):
Jerry (to car rental attendant): "You know how to take the reservation, you just don't know how to hold the reservation... and that's really the most important part of the reservation -- the holding!"
:-)
Not holding a reservation should not be a felony... it should be a minor infraction at best, a Class C Misdemeanor (the least serious kind) at worst...
Also, there should be no jail time...
And no fine...
The criminal penalty for not holding other people's reservations should be that you actually have to start holding other people's reservations!
That's the Court sentence!
You actually have to start holding other people's reservations!
:-)
(You know, "let the punishment fit the crime!" :-) )
kube-system 41 minutes ago [-]
Knowingly exceeding authorized access of any computer used in interstate commerce is a felony in the US.
The title of TFA is a metaphorical criticism, not a literal law analysis.
They are not making the statement that the person in Australia who accidentally cancelled someone's reservation in Australia is literally guilty of violating US law. They are drawing criticism of AI models which are taking the kinds of actions for which, if a human did them knowingly, would be illegal.
john_strinlai 1 hours ago [-]
>An AI cancelling other people's gym classes is a felony? Don't computer systems fail all the time at holding reservations for people?
the difference is intent.
if a concierge/booking system makes a mistake (or has an unintended bug or whatever), no crime.
but if i (or an agent working on behalf of me) use an API in an obviously unintended way to revoke other people's reservations, that would fall under the computer fraud and abuse act (in the usa).
peter_d_sherman 1 hours ago [-]
>"the difference is
intent."
>"but if i (or an agent working on behalf of me) use an API in an obviously
unintended
way to revoke other people's reservations..."
?
john_strinlai 53 minutes ago [-]
i am not quite sure what your question is, as you simply quoted me and then put a question mark... i think you are confused that i used "intent" in one context, and "unintended" in a different context, is that right?
the first sentence: the difference is the intent of the person who caused the cancellations
the second sentence: but if i (or an agent working on behalf of me) abuse an API to do things it was not meant or designed to do, such as cancelling someone else's reservation
redox99 45 minutes ago [-]
Yeah if you're unlucky you get hit with like 20 years for wire fraud.
One's hosted on porkbun and one's hosted on namecheap.
naniel 1 hours ago [-]
Lol now this is the kind of benchmarking i'm looking for
nubg 1 hours ago [-]
Thank you, this benchmark to me proves that closed weight model companies are dangerous for our democracy and put kids at risk. They must be outlawed and all models must be made open weights!
0xbadcafebee 2 hours ago [-]
Open models with advanced security features are a huge security benefit. Because any script kiddie can use them to hack into random things, people will now be forced to spend more time securing their technology. And they won't have to learn how, because they can use those same models to find the holes and patch them.
Edit: since this is apparently somewhat controversial, perhaps some explanation is in order.
"Felony" has no set definition of which crimes it must apply to, it is entirely based on the discretion of the locality setting the laws. What is a felony in one place can often be a misdemeanor in another. This is especially true for nonviolent crimes.
It's also been shown in studies that nonviolent felonies are imposed against minorities at a much higher rate, for the same crimes.
And because felonies carry additional, lifelong consequences, they are an effective way to mask a 2-tiered justice system.
Rendered at 17:50:55 GMT+0000 (Coordinated Universal Time) with Vercel.
It really is that these models have been trained, or maybe even over-trained, to save memories, and to a very far extend, this thing that they're calling communication is just the function of it saving memories.
To be honest, if I could stop AI from saving memories, it would be fantastic, because claude code etc definitely creates more issues for me when it creates memories than anything it solves.
But really the jailbreak was memories.
If you ever do introduce legislation, I would love to see legislation which stops general-purpose AI from saving memories. I think that would make things a lot safer.
Instead it's a collection of what made the news which feels like will not be updated and prove very little.
a bit silly, as one typically has to prove intent (which is why security researchers don't get slapped with felonies all the time).
"inadvertently" and the existence of guardrails/sandboxes/etc make it pretty unconvincing that these incidents were intentionally malicious.
still a fun thing to track, but the name is just a bit overstated.
which, again, sandboxes and guardrails and such would make a gross negligence argument unconvincing.
I’m partially surprised that they didn’t do exactly that. If I ran a corporation I would assume any intrusion attempt by another company was intentional. Why wouldn’t I? Corporate espionage is super common.
I assume the answer is that these executives know each other personally.
there could be 1,000 escapes, where each one was enabled by novel and unexpected chain of 0-day exploits. not likely to be considered reckless disregard in court.
there could be 1 escape, where there was no sandbox, no guardrails, no instructions to avoid damage, etc. which would likely to be considered reckless disregard (well, more likely to be, but still, reckless disregard is a high bar).
Edit: changed labs to corps because it’s time to stop pretending these are places of science.
https://techcrunch.com/wp-content/uploads/2026/03/2026.03.04...
Isn’t this affected heavily by adoption of a model? I feel like this might as well be a proxy for how popular a model is.
In any case it’s an interesting concept for a benchmark.
https://www.anthropic.com/news/detecting-countering-misuse-a...
> The actor used AI to what we believe is an unprecedented degree. Claude Code was used to automate reconnaissance, harvesting victims’ credentials, and penetrating networks. Claude was allowed to make both tactical and strategic decisions, such as deciding which data to exfiltrate, and how to craft psychologically targeted extortion demands. Claude analyzed the exfiltrated financial data to determine appropriate ransom amounts, and generated visually alarming ransom notes that were displayed on victim machines.
tldr Claude was used to develop and execute malware.
An AI cancelling other people's gym classes is a felony?
?
Don't computer systems fail all the time at holding reservations for people?
Heck, don't people fail all the time at holding reservations for other people?
You know, like in Seinfeld's "Alternate Side" Episode (S3 E11):
Jerry (to car rental attendant): "You know how to take the reservation, you just don't know how to hold the reservation... and that's really the most important part of the reservation -- the holding!"
:-)
Not holding a reservation should not be a felony... it should be a minor infraction at best, a Class C Misdemeanor (the least serious kind) at worst...
Also, there should be no jail time...
And no fine...
The criminal penalty for not holding other people's reservations should be that you actually have to start holding other people's reservations!
That's the Court sentence!
You actually have to start holding other people's reservations!
:-)
(You know, "let the punishment fit the crime!" :-) )
The title of TFA is a metaphorical criticism, not a literal law analysis.
They are not making the statement that the person in Australia who accidentally cancelled someone's reservation in Australia is literally guilty of violating US law. They are drawing criticism of AI models which are taking the kinds of actions for which, if a human did them knowingly, would be illegal.
the difference is intent.
if a concierge/booking system makes a mistake (or has an unintended bug or whatever), no crime.
but if i (or an agent working on behalf of me) use an API in an obviously unintended way to revoke other people's reservations, that would fall under the computer fraud and abuse act (in the usa).
intent."
>"but if i (or an agent working on behalf of me) use an API in an obviously
unintended
way to revoke other people's reservations..."
?
the first sentence: the difference is the intent of the person who caused the cancellations
the second sentence: but if i (or an agent working on behalf of me) abuse an API to do things it was not meant or designed to do, such as cancelling someone else's reservation
One's hosted on porkbun and one's hosted on namecheap.
Edit: since this is apparently somewhat controversial, perhaps some explanation is in order.
"Felony" has no set definition of which crimes it must apply to, it is entirely based on the discretion of the locality setting the laws. What is a felony in one place can often be a misdemeanor in another. This is especially true for nonviolent crimes.
It's also been shown in studies that nonviolent felonies are imposed against minorities at a much higher rate, for the same crimes.
And because felonies carry additional, lifelong consequences, they are an effective way to mask a 2-tiered justice system.