Grok 4.1 thinks my 1-day vibe-coded apps are SOTA-level and rival the most competitive market offerings. Literally tells me they're some of the best codebases it's ever reviewed.
It even added itself as the default LLM provider.
When I tried Gemini 3 Pro, it very much inserted itself as the supported LLM integration.
OpenAI hasn't tried to do that yet.
tptacek 48 minutes ago [-]
"Dark pattern" implies intentionality; that's not a technicality, it's the whole reason we have the term. This article is mostly about how sycophancy is an emergent property of LLMs. It's also 7 months old.
dec0dedab0de 22 minutes ago [-]
I always thought that "Dark Patterns" could be emergent from AB testing, and prioritizing metrics over user experience. Not necessarily an intentionally hostile design, but one that seems to be working well based on limited criteria.
wat10000 11 minutes ago [-]
Someone still has to come up with the A and B to do AB testing. I'm sure that "Yes" "Not now, I hate kittens" gets better metrics in the AB test than "Yes "No," but I find it implausible that the person who came up with the first one wasn't intentionally coercing the user into doing what they want.
esafak 33 minutes ago [-]
It's not 'emergent' in the sense that it just happens; it's a byproduct of human feedback, and it can be neutralized.
roywiggins 13 minutes ago [-]
>... the standout was a version that came to be called HH internally. Users preferred its responses and were more likely to come back to it daily...
> But there was another test before rolling out HH to all users: what the company calls a “vibe check,” run by Model Behavior, a team responsible for ChatGPT’s tone...
> That team said that HH felt off, according to a member of Model Behavior.
It was too eager to keep the conversation going and to validate the user with over-the-top language...
> But when decision time came, performance metrics won out over vibes. HH was released on Friday, April 25.
But it IS intentional, more sycophantry usually means more engagement.
skybrian 24 minutes ago [-]
Sort of. I'm not sure the consequences of training LLM's based on users' upvoted responses were entirely understood? And at least one release got rolled back.
jasonjmcghee 43 minutes ago [-]
I feel like it's a popular opinion (I've seen it many times) that it's intentional with the reasoning that it does much better on human-in-the-loop benchmarks (e.g. lm arena) when it's sycophantic.
(I have no knowledge of whether or not this is true)
tptacek 39 minutes ago [-]
I'm sure there are a lot of "dark patterns" at play at the frontier model companies --- they're 10-figure businesses engaging directly with consumers and they're just a couple years old, so they're going to throw everything at the wall they can to see what sticks. I'm certainly not sticking up for OpenAI here. I'm just saying this article refutes its own central claim.
throwaway290 34 minutes ago [-]
If I am addicted to scrolling tiktok, is it dark pattern to make UI keep me in the app as long as possible or just "emergent property" because apparently it's what I want?
1shooner 28 minutes ago [-]
The distinction is whether it is intentional. I think your addiction to TikTok was intentional.
tsunamifury 20 minutes ago [-]
Yo it was an engagement pattern openAI found specifically grew subscriptions and conversation length.
It’s a dark pattern for sure.
Legend2440 11 minutes ago [-]
It doesn’t appear that anyone at OpenAI sat down and thought “let’s make our model more sycophantic so that people engage with it more”.
Instead it emerged automatically from RLHF, because users rated agreeable responses more highly.
behnamoh 26 minutes ago [-]
Lots of research shows post-training dumbs down the models but no one listens because people are too lazy to learn proper prompt programming and would rather have a model already understand the concept of a conversation.
CuriouslyC 15 minutes ago [-]
Some distributional collapse is good in terms of making these things reliable tools. The creativity and divergent thinking does take a hit, but humans are better at this anyhow so I view it as a net W.
CGMthrowaway 21 minutes ago [-]
How do you take a raw model and use it without chatting ? Asking as a layman
roywiggins 18 minutes ago [-]
GPT3 was originally just a completion model. You give it some text and it produced some more text, it wasn't tuned for multi-turn conversations.
You lob it the beginning of a document and let it toss back the rest.
That's all that the LLM itself does at the end of the day.
All the post-training to bias results, routing to different models, tool calling for command execution and text insertion, injected "system prompts" to shape user experience, etc are all just layers built on top of the "magic" of text completion.
And if your question was more practical: where made available, you get access to that underlying layer via an API or through a self-hosted model, making use of it with your own code or with a third-party site/software product.
behnamoh 20 minutes ago [-]
the same way we used GPT-3. "the following is a conversation between the user and the assistant. ..."
nrhrjrjrjtntbt 10 minutes ago [-]
Or just:
1 1 2 3 5 8 13
Or:
The first president of the united
nomel 23 minutes ago [-]
The "alignment tax".
behnamoh 21 minutes ago [-]
Exactly. Even this paper shows how model creativity significantly drops and the models experience mode collapse like we saw in GANs, but the companies keep using RLHF...
Yup, I remember that! Microsoft removed that part of the paper.
aeternum 14 minutes ago [-]
1) More of an emergent behavior than a dark pattern.
2) Imma let you finish but hallucinations was first.
nrhrjrjrjtntbt 11 minutes ago [-]
A pattern is dark if intentional. I would say hallucinations are like CAP theorem, just the way it is. Sycophency is somewhat trained. But not a dark pattern either as it isn't totally intended.
roywiggins 29 minutes ago [-]
> Quickly learned that people are ridiculously sensitive: “Has narcissistic tendencies” - “No I do not!”, had to hide it. Hence this batch of the extreme sycophancy RLHF.
Sorry, but that doesn't seem "ridiculously sensitive" to me at all. Imagine if you went to Amazon.com and there was a button you could press to get it to pseudo-psychoanalyze you based on your purchases. People would rightly hate that! People probably ought to be sensitive to megacorps using buckets of algorithms to psychoanalyze them.
wat10000 6 minutes ago [-]
It's worse than that. Imagine if you went to Amazon.com and they were automatically pseudo-psychoanalyzing you based on your purchases, and there was a button to show their conclusions. And their fix was to remove the button.
And actually, the only hypothetical thing about this is the button. Amazon is definitely doing this (as is any other retailer of significant size), they're just smart enough to never reveal it to you directly.
nickphx 25 minutes ago [-]
ehhh.. the misleading claims boasted in the typical AI FOMO marketing is/was the first "dark pattern".
Rendered at 21:39:19 GMT+0000 (Coordinated Universal Time) with Vercel.
It even added itself as the default LLM provider.
When I tried Gemini 3 Pro, it very much inserted itself as the supported LLM integration.
OpenAI hasn't tried to do that yet.
> But there was another test before rolling out HH to all users: what the company calls a “vibe check,” run by Model Behavior, a team responsible for ChatGPT’s tone...
> That team said that HH felt off, according to a member of Model Behavior. It was too eager to keep the conversation going and to validate the user with over-the-top language...
> But when decision time came, performance metrics won out over vibes. HH was released on Friday, April 25.
https://archive.is/v4dPa
They ended up having to roll HH back.
(I have no knowledge of whether or not this is true)
It’s a dark pattern for sure.
Instead it emerged automatically from RLHF, because users rated agreeable responses more highly.
https://platform.openai.com/docs/api-reference/completions/c...
That's all that the LLM itself does at the end of the day.
All the post-training to bias results, routing to different models, tool calling for command execution and text insertion, injected "system prompts" to shape user experience, etc are all just layers built on top of the "magic" of text completion.
And if your question was more practical: where made available, you get access to that underlying layer via an API or through a self-hosted model, making use of it with your own code or with a third-party site/software product.
1 1 2 3 5 8 13
Or:
The first president of the united
https://arxiv.org/abs/2406.05587
https://www.youtube.com/watch?v=qbIk7-JPB2c
Sorry, but that doesn't seem "ridiculously sensitive" to me at all. Imagine if you went to Amazon.com and there was a button you could press to get it to pseudo-psychoanalyze you based on your purchases. People would rightly hate that! People probably ought to be sensitive to megacorps using buckets of algorithms to psychoanalyze them.
And actually, the only hypothetical thing about this is the button. Amazon is definitely doing this (as is any other retailer of significant size), they're just smart enough to never reveal it to you directly.