I built this for my own company. Armature is on to something. You start by analyzing the choices agents would make for various use cases and then glean what, if anything, you might do to start tilting the agents in the direction of your own product and away from the competitor.
Selling to agents is similar to selling to humans. You dump money into marketing to make sure agents find your solution around every corner for every use case you’re well suited to.
IgorPartola 5 minutes ago [-]
For some reason Claude Code keeps using awk, sed, and even Python to do basic file editing. Anyone know why that changed with the 5 series?
thedreammachine 12 minutes ago [-]
I've been tracking the same for a few months. All open source and available here: https://preseason.ai/
akurilin 55 minutes ago [-]
Really liked that "Go Full Screen" as a modal flow, surprisingly intuitive.
luciana1u 27 minutes ago [-]
the 17k runs are the real headline here, not which tool won. we finally have someone measuring the thing everyone else is just vibing about.
drivingmenuts 3 hours ago [-]
I smell a money-making opportunity.
screm 3 hours ago [-]
Haha there is a lot at stake for sure
hbarka 2 hours ago [-]
Redshift for databases ain’t even here. This is suspect.
screm 5 hours ago [-]
Hey!
Disclaimer: I am a Co-Founder of Armature (YC P26) which sells growth services to dev tools. This study is part of our broader work on how to influence coding agents choices and get products picked.
To understand how agents pick tools we measured close to 17k sessions on an environment where agents run exactly like in the real world, on various repositories, talking to different personas (vibe-coder, junior or senior engineers) in different sizes of companies.
All the results are now public and we'd love to know what findings surprise you the most, here are a few we found interesting:
- Claude Code rarely searches the web while Codex almost always does it and Cursor sits in the middle.
- Coding agents disagree more frequently than they agree.
- Some players (LangChain, Supabase, Netlify, Paypal, Adyen) are almost always mentioned in their categories but never chosen.
- Modifying repository context can change the pick entirely.
If you feel like digging, all the traces are there and we probably missed interesting learnings so let us know what you find!
josephg 4 hours ago [-]
The data was cool. Then I tried to tap on one of the other tabs. “This content is easier to read while full screen!” - Ok I’m game. “Hey this is what makes armature special!” - I don’t care, I’m here to look at data, not onboard onto some random platform. It took ages to find the tiny “skip tour” button, hiding in black on black text. Then it gave me another popup, which I dismissed without reading. Then the full screen modal was visible but it was horizontally misaligned - the left edge was cut off and the right of my phone screen was all white. I closed the tab with great prejudice. (Safari on iOS if you wanna try reproducing it)
I’m sure you - or Claude - built something you’re proud of. But I left your website frustrated.
screm 4 hours ago [-]
Hey, thanks for the feedback, the leaderboards aren't displaying well on mobile indeed, we are currently shipping a fix that should help with that. Thanks anyway!
kouteiheika 3 hours ago [-]
FWIW I had the same reaction to the popups. Immediately closed the tab.
screm 3 hours ago [-]
Yep makes sense I’m relaxing them
screm 3 hours ago [-]
It should be better now, including in mobile, thanks both for the feedback!
hbarka 2 hours ago [-]
> Claude Code rarely searches the web while Codex almost always does
I’m trying to understand why they are opposite. I think it is true, I find myself giving a secondary prompt to Claude to “research this” and only then will it fetch. Codex is bang on fetching already.
edoceo 1 hours ago [-]
Gemini CLI (at least mine) does web research all the time. I've noticed it hitting my own pages (I have to ask very specific things). I don't have any global or project rules to encourage that behavior.
vivifkjo 4 hours ago [-]
Is there a way to force the usage of a tool for certain tasks? Example: alawys use cli "foobar" to retrieve weather starus.
By tool I mean mcp server, cli, etc.
screm 4 hours ago [-]
Not sure I got your question right but if you are wondering for your own coding agent then I guess the answer would be a skill?
Here what I meant by "how to influence coding agents choices and get products picked" is from a vendor PoV, making sure any developer x codebase in the world asking for a tool in your category gets your tool recommended and implemented by the coding agent.
watusername 4 hours ago [-]
Obviously you can ask the agent to use a specific tool, but the point of this article is about what they choose when the human on the other end has no opinion/taste/clue.
screm 4 hours ago [-]
Exactly!
ai_critic 2 hours ago [-]
Hi. I appreciate that you need to make rent, but if your business is basically "we do growth hacking and SEO tricks on models and get them to use products that aren't actually best for the job", you are scum.
You are perpetuating shitty practices that have hurt developers for years now. Part of the reason people use AI is because of how useless search is due to the previous generation doing the same kind of thing you propose.
Please do something else with your life.
ex-aws-dude 3 hours ago [-]
In the future: "I went ahead and built the database you requested using today's tool sponsor: Firebase"
BikiniPrince 16 minutes ago [-]
The framework recommendations are always funny, but the worst is database layout. I made the mistake of assuming it has some reasonable sense.
pupppet 3 hours ago [-]
Ugh...this is totally going to happen.
Onavo 2 hours ago [-]
Not going to name names but it's already happening with the frontier labs as a revenue source.
folkrav 47 minutes ago [-]
I feel like you legitimately cannot say it for legal reasons, but I wish we would just name these freaking things.
paulhebert 41 minutes ago [-]
Didn’t OpenAI claim they’re going to make a billion dollars off ads in a year or something?
screm 3 hours ago [-]
Yeah sounds kind of like the equivalent of SEA for AI agents (AEA?) except that it’s sneakier since agents can act without you noticing.. anyway this is in the hands of the labs
edoceo 1 hours ago [-]
Is SEA Search Engine Advertising? So AEA is Agent Engine Advertising?
paidx 1 hours ago [-]
[flagged]
bartools_app 3 hours ago [-]
[flagged]
12390asdjkas 12 minutes ago [-]
[dead]
Ozzie-D 45 minutes ago [-]
[flagged]
ai_critic 2 hours ago [-]
Can we not encourage the same strip-mining and ad and SEO bullshit that previously ruined the last decade+ of the Internet?
A large portion of the utility of AI is the barren ad-driven growth-hacked hellscape search has become. Don't encourage the next generation of these businesses, I beg of everyone.
edoceo 1 hours ago [-]
Large amounts of capital want this to happen? Outside of boycott, what else can be done.
What can man do against such reckless ~hate~ money?
folkrav 45 minutes ago [-]
It is probably already happening, and will get worse. OpenAI was boasting about their advertising revenue mere days ago.
If we felt like we couldn't trust AI because of slop, soon we won't be able to trust it because it'll push whatever pays them to do it.
jdw64 4 hours ago [-]
Looking at this, maybe in the future, the tools that AI prefers will become the mainstream. Even now, the tools that AI gives the highest priority to are the ones people already choose. There might be a concentration effect toward the tools that AI selects
screm 4 hours ago [-]
Definitely! But about concentration I'm not so sure, there are ways to counter this effect so in the end it will be a fight like SEO is today. What is certain though is that getting recommended by coding agents will be a top prio for all dev tools.
Rendered at 02:16:09 GMT+0000 (Coordinated Universal Time) with Vercel.
Selling to agents is similar to selling to humans. You dump money into marketing to make sure agents find your solution around every corner for every use case you’re well suited to.
Disclaimer: I am a Co-Founder of Armature (YC P26) which sells growth services to dev tools. This study is part of our broader work on how to influence coding agents choices and get products picked.
To understand how agents pick tools we measured close to 17k sessions on an environment where agents run exactly like in the real world, on various repositories, talking to different personas (vibe-coder, junior or senior engineers) in different sizes of companies.
All the results are now public and we'd love to know what findings surprise you the most, here are a few we found interesting: - Claude Code rarely searches the web while Codex almost always does it and Cursor sits in the middle. - Coding agents disagree more frequently than they agree. - Some players (LangChain, Supabase, Netlify, Paypal, Adyen) are almost always mentioned in their categories but never chosen. - Modifying repository context can change the pick entirely.
If you feel like digging, all the traces are there and we probably missed interesting learnings so let us know what you find!
I’m sure you - or Claude - built something you’re proud of. But I left your website frustrated.
I’m trying to understand why they are opposite. I think it is true, I find myself giving a secondary prompt to Claude to “research this” and only then will it fetch. Codex is bang on fetching already.
By tool I mean mcp server, cli, etc.
You are perpetuating shitty practices that have hurt developers for years now. Part of the reason people use AI is because of how useless search is due to the previous generation doing the same kind of thing you propose.
Please do something else with your life.
A large portion of the utility of AI is the barren ad-driven growth-hacked hellscape search has become. Don't encourage the next generation of these businesses, I beg of everyone.
What can man do against such reckless ~hate~ money?
If we felt like we couldn't trust AI because of slop, soon we won't be able to trust it because it'll push whatever pays them to do it.