I would never trust an AI to make a scientific conclusion, but they're very good at some things:
- Plotting with natural language: "Create a scatter plot of column A versus column B, log scale Y and use stars sized by column C." Every plotting library (except maybe ggplot2 / plotnind) sucks, and the boilerplate never gets less tedius.
- Literature search, e.g. "Have there been any peer reviewed papers where they applied X method to Y domain"
- Complex visualization, e.g. "Make a self contained HTML file with a 3D interactive visualization of this graph. Add ball and spring position optimization and let me move nodes"
- Trying random stuff that would normally be too time consuming to be worth it, eg "Here is a dataset. Try regression methods A, B, and C and report a table." In this case, usually it will screw up and overfit, but it's a good smoke test.
Note how in all of these, it's being used for tedious things that provide some value, but ultimately they are only providing information to make a decision on.
Granted, I would like to have a research assistant I could train, but if your company will not create a position then there is not much you can do.
ozgung 3 hours ago [-]
Yes to all but I have trouble using them effectively for my research side projects or Kaggle competitions. Claude Code/Codex designs and runs experiments quickly, invalidates my initial idea using some obscure criteria than changes direction, and after long hours and days of experiments we end up with poor results. There must be a better agentic workflow for research.
StableAlkyne 3 hours ago [-]
> invalidates my initial idea using some obscure criteria than changes direction
You might be giving it too much leeway.
Opus and Fable are more trustworthy, but even today I find they very quickly go off the rails for research tasks if I don't give them short chunks of work.
They're not ready for prime time automation of research (and before someone mentions the math results, that required absurd amounts of computation time and has immediate feedback in the form of Lean)
tensor 2 hours ago [-]
Yes, I find DeepSeek can be very precious about its ideas and I have to insist multiple times to do the experiments I ask for despite it thinking them not worth doing. Often my experiments do give good results and I swear the model gets salty about it.
I guess when you ask these things to take on the role of a senior scientist it picks up some natural stubbornness in the training data.
lxe 2 hours ago [-]
When it comes to difficult optimization/search problems, doing research in a single Claude/Codex thread is definitely a struggle. There are ways to bypass some of the pitfalls by creating a agentic swarm with hard guardrails such as context resets and changing the way the harness works with memory, etc.
lxe 2 hours ago [-]
So you're not wrong in most of these conclusions, however this is very surface level use case for these tools when it comes to scientific research.
> I would never trust an AI to make a scientific conclusion, but they're very good at some things
> In this case, usually it will screw up and overfit, but it's a good smoke test.
> it's being used for tedious things that provide some value
I see a lot of comments like this about AI use in general, and I suspect there’s a kind of learned helplessness combined with anchoring to earlier experiences. Someone tries a model, encounters a few failures, and forms a fairly permanent view of what these systems can do.
My concern is that people sometimes turn an expectation of failure into a permanent ceiling on how much they’ll even experiment with these tools. Their limited use then keeps confirming their limited expectations.
EDIT: This came off as criticizing the user, but this is entirely to validate my own thesis that for quite a long time nobody has been able to crack the usability challenge when it comes to non-coding knowledge work for these harnesses. It's fascinating because it seems like it's such an easy problem to solve, but in reality it's a complex product problem.
StableAlkyne 2 hours ago [-]
> this is very surface level use case for these tools
Correct. As of Q4 2026, in my experience they are only trustworthy for surface level use cases in the domain of computational chemistry / in-silico high throughout screening, unless you are working on a highly constrained, immediately verifiable and well-studied problem like protein docking or structure-property relations.
> anchoring to earlier experiences.
I have observed the models improving over time. I am basing my post on things at the level of Opus 5.5 / GPT-6.1 Sol / Fable 5.1
> Their limited use then keeps confirming their limited expectations.
Every time a new model comes out or a harness I use gets a new feature, I do try to push it until it breaks. A swarm of GPT Lunas at max context is highly effective at finding information or condensing highly disparate data sources; that wasn't even a thing a year ago in most harnesses.
But even the smartest models like Fable will get sidetracked, or make bad decisions because they do not have enough context of the problem domain. As of Q4 2026.
I'd put their autonomous intelligence in this particular domain at the level of a well-read first year grad student. Phenomenal at knowing things, but unable to question their own assumptions or invalidate their own work unless you hold their hand in the form of a prompt, skill, or direct instruction.
lxe 2 hours ago [-]
Thanks for the clarification, this is such a spot on response that actually mirrors my experience considerably.
> unless you hold their hand in the form of a prompt, skill, or direct instruction.
The hand-holding and inability to execute automated tasks is a very well known failure mode.
The requirement for me to prod, redirect and continuously steer is the number one pitfall when I did the Muse Open Kernel challenge using a long-horizon search/improvement swarm (or whatever the term is... just don't call it 'RSI').
Using plateau avoidance techniques such as https://arxiv.org/abs/2610.07447 and building harnesses around these is what helped me.
StableAlkyne 2 hours ago [-]
Very interesting article, I'm going to give it a read over lunch. Thanks!
nyc_data_geek 2 hours ago [-]
Experimenting with these tools, checking their work and spending that time is not free of cost.
knottn 2 hours ago [-]
Didn’t read your whole post, but the first point is wrong.
“Plotting with natural language: "Create a scatter plot of column A ver….”
No, the tedium of boilerplate code was a feature. Charts were generated by template so styling was consistent, making charts a bit more bland but easier to consume and check. You cannot achieve that same level of consistency or predictability with ai without reinventing a wheel. The models change too fast, and they all want to differentiate themselves for brand reasons, and the emphasis is on fast and easy, not on slow and foolproof, there’s no documentation of the work the way code provides documentation and an easy audit trail. You can restrict the ai by giving specs (use ggplot, use color template foo, create readable code with unit tests), but these are all similar tradeoffs as you might make with human labor too.
What llm do is to simply lower human value, not remove it entirely.
Just as results needed checking and replication before llm, they will continue to need checking and replication by humans. The humans won’t be compensated nearly as much, but humans are humans and so are not.
StableAlkyne 58 minutes ago [-]
> Charts were generated by template so styling was consistent, making charts a bit more bland but easier to consume and check
This is why reading the whole post is important :)
You seem to think I'm suggesting the use an image generator for plotting. Don't do that, it will screw that up, image models are not made for that. The use for AI here is to save 5 minutes writing the code to generate the plot, or to edit the style (again via code) of the plot.
If you do plt.scatter(x,y, c="blue", marker="o") that is a function call in Matplotlib. Its API notoriously sucks. Most Python plotting libraries just create convenience functions around this library because although the API (actually there are two and the documentation is not consistent on which to use) is hot garbage, and the documentation is labyrinthine, the plotting engine itself is powerful.
> What llm do is to simply lower human value, not remove it entirely.
The value in a plot is not the exact code used to make it (beyond the code being correct). The value is in the visualization giving the reader an "ah-ha" moment. I would much rather take the 5 minutes I save writing a function call, and use it to spend more time examining the plot itself.
ch4s3 3 hours ago [-]
> Literature search, e.g. "Have there been any peer reviewed papers where they applied X method to Y domain"
This is something the LLMs are really good for, at least for compiler papers.
jhbadger 2 hours ago [-]
And like everything LLM related this is something that people may have had bad experience with in the past but which work well now -- a year or two ago half those papers would be "hallucinated" non-existent papers but this rarely happens these days.
bicepjai 3 hours ago [-]
They are great computer interface assistants, but forget scientific conclusion would never trust then for committing in right git branch
w10-1 2 hours ago [-]
So: code + econ; researchers and organizations slated for overwhelm
Mostly quotes:
- 207K consumer user claude sessions (78K code, remainder chat+cowork) in Aug+Sep '26 classified per 7K SciNet taxonomy
- largest domains: computer science (11.9%), engineering (8.4%), and medicine (7.7%)
- higher ratio of usage/researcher: computer science, mathematics, statistics; social sciences
- "Usage is also high for the social sciences" ... "Relative usage is low in [...] the social sciences, with the exception of economics"
- Claude use and AI capability: "the top 20% of tasks also draw two to four times the expected use" "Health sciences researchers use Claude relatively little compared with the AI capability scores of their tasks"
- "Theoretical AI capability also predicts how much researchers allow Claude to work autonomously"
- "productivity gains tend to materialize only after organizations restructure around them (David 1990; Brynjolfsson, Rock, and Syverson 2021)"
- "A surge in new research papers could advance a field faster but also overwhelm scientists’ capacity to absorb or verify the results."
- "As AI becomes more capable and autonomous, AI systems could take the lead in verification, refereeing, and communicating important results, which would ease the absorption problem discussed above"
- "Fifth, more autonomous AI systems may increase the challenge of ensuring that AI-driven research remains aligned with human goals"
- "One effect may be a change in researchers’ job satisfaction."
dreambuffer 2 hours ago [-]
Don't like this style of article where the "How" is "in what manner do they use it?" and not "what techniques do they use?". I would prefer to see specific lab setups, how they use and coordinate agents, what kind of protocol they have for asking questions and verifying information, and so on.
lxe 2 hours ago [-]
Because it's written for marketing and engagement, not to actually teach anyone anything.
FailMore 3 hours ago [-]
This article talks about the outputs being Claude artifacts. I am curious if anyone has feedback on using Claude artifacts? What do you like/not like?
Artifacts are amazing, I think they are not getting enough attention. They recently cloned most of the Google Docs / Notion features that I was using those connectors for. Comments are very useful for steering.
The missing feature for docs is now GDrive, ie organizing and cataloging your artifacts to build a coherent long-lived KB. But for that you can still export.
Anthropic added a while back the ability to let clients provide their own API key, again I think that’s under-explored and will become more relevant as AI gets more penetration.
Building out quick SPAs, HTML pages, etc is a great use, I think the innovation will be making these things supportable and usable for more than one-shot single-user artifacts. Eg user login, analytics, etc.
bemtevi 7 hours ago [-]
AI in science is a lot different than AI in progamming in some ways. It seems pretty harmless in programming a lot of the times because it's used to speed up rote tasks like writing boilerplate. But in science, if it does give scientists a boost in discovery, I think it's likely to lead to an even more cavalier attitude about the power that such discoveries can have, and thus lead to much more abuse of power by the wealthy and malicious actors.
leonpillow 2 hours ago [-]
Using privacy-preserving methods, we study 207,000 Claude and Claude Code research sessions to understand how Claude is used across scientific fields and research tasks
zach_moore 2 hours ago [-]
So they violated user's preference settings but they "preserved privacy"
5 hours ago [-]
nsagent 4 hours ago [-]
> Using privacy-preserving methods, we study 207,000 Claude and Claude Code research sessions to understand how Claude is used across scientific fields and research tasks.
Did they get permission to use researchers' data or did they monitor and extract data without consent? Considering the kerfuffle OpenAI generated with their math proofs, you'd think Anthropic would take a hint and be clear that they did the research with consent, though that assumes they got consent.
Seriously, all the industry researchers at these companies went through IRB training during their PhD; have they just conveniently forgotten?
adrianN 3 hours ago [-]
Isn’t there something in the terms of service that gives them essentially blanket permission to read you conversations?
evan_a_a 2 hours ago [-]
Yep, exactly, this is already covered by the privacy policy.
that's such a long read. can't they make it more digestable/readable on the blog and then leave the rest into a paper?
wslh 3 hours ago [-]
Am I the only one who finds Claude overly assertive and confident for research? I tend to get more value out of other models (e.g. Gemini 4 Argon) even if they make mistakes, and then use Claude to revise and polish.
7373737373 2 hours ago [-]
I agree, LLMs in general tend to misuse words and concepts like "proof" - when they have only partially or sloppily carried out an (heuristic) argument, or in situations where a proof is not possible at all, only empirical evidence. This has improved with recent models but has not been entirely solved
kian 2 hours ago [-]
"less in the health and lab-based sciences"... Wow. Got to love how they report the intended effects of their biology/chemistry query censorship as if it was some neutral result, without any mention of its likely cause.
`In other news, we report that there seems to be significantly less cybersecurity research done on our platform than other areas of computer science` /s
MrMan 7 hours ago [-]
[dead]
asnrt 4 hours ago [-]
[flagged]
Rendered at 19:47:27 GMT+0000 (Coordinated Universal Time) with Vercel.
I would never trust an AI to make a scientific conclusion, but they're very good at some things:
- Plotting with natural language: "Create a scatter plot of column A versus column B, log scale Y and use stars sized by column C." Every plotting library (except maybe ggplot2 / plotnind) sucks, and the boilerplate never gets less tedius.
- Literature search, e.g. "Have there been any peer reviewed papers where they applied X method to Y domain"
- Complex visualization, e.g. "Make a self contained HTML file with a 3D interactive visualization of this graph. Add ball and spring position optimization and let me move nodes"
- Trying random stuff that would normally be too time consuming to be worth it, eg "Here is a dataset. Try regression methods A, B, and C and report a table." In this case, usually it will screw up and overfit, but it's a good smoke test.
Note how in all of these, it's being used for tedious things that provide some value, but ultimately they are only providing information to make a decision on.
Granted, I would like to have a research assistant I could train, but if your company will not create a position then there is not much you can do.
You might be giving it too much leeway.
Opus and Fable are more trustworthy, but even today I find they very quickly go off the rails for research tasks if I don't give them short chunks of work.
They're not ready for prime time automation of research (and before someone mentions the math results, that required absurd amounts of computation time and has immediate feedback in the form of Lean)
I guess when you ask these things to take on the role of a senior scientist it picks up some natural stubbornness in the training data.
> I would never trust an AI to make a scientific conclusion, but they're very good at some things
> In this case, usually it will screw up and overfit, but it's a good smoke test.
> it's being used for tedious things that provide some value
I see a lot of comments like this about AI use in general, and I suspect there’s a kind of learned helplessness combined with anchoring to earlier experiences. Someone tries a model, encounters a few failures, and forms a fairly permanent view of what these systems can do.
My concern is that people sometimes turn an expectation of failure into a permanent ceiling on how much they’ll even experiment with these tools. Their limited use then keeps confirming their limited expectations.
EDIT: This came off as criticizing the user, but this is entirely to validate my own thesis that for quite a long time nobody has been able to crack the usability challenge when it comes to non-coding knowledge work for these harnesses. It's fascinating because it seems like it's such an easy problem to solve, but in reality it's a complex product problem.
Correct. As of Q4 2026, in my experience they are only trustworthy for surface level use cases in the domain of computational chemistry / in-silico high throughout screening, unless you are working on a highly constrained, immediately verifiable and well-studied problem like protein docking or structure-property relations.
> anchoring to earlier experiences.
I have observed the models improving over time. I am basing my post on things at the level of Opus 5.5 / GPT-6.1 Sol / Fable 5.1
> Their limited use then keeps confirming their limited expectations.
Every time a new model comes out or a harness I use gets a new feature, I do try to push it until it breaks. A swarm of GPT Lunas at max context is highly effective at finding information or condensing highly disparate data sources; that wasn't even a thing a year ago in most harnesses.
But even the smartest models like Fable will get sidetracked, or make bad decisions because they do not have enough context of the problem domain. As of Q4 2026.
I'd put their autonomous intelligence in this particular domain at the level of a well-read first year grad student. Phenomenal at knowing things, but unable to question their own assumptions or invalidate their own work unless you hold their hand in the form of a prompt, skill, or direct instruction.
> unless you hold their hand in the form of a prompt, skill, or direct instruction.
The hand-holding and inability to execute automated tasks is a very well known failure mode.
The requirement for me to prod, redirect and continuously steer is the number one pitfall when I did the Muse Open Kernel challenge using a long-horizon search/improvement swarm (or whatever the term is... just don't call it 'RSI').
Using plateau avoidance techniques such as https://arxiv.org/abs/2610.07447 and building harnesses around these is what helped me.
“Plotting with natural language: "Create a scatter plot of column A ver….”
No, the tedium of boilerplate code was a feature. Charts were generated by template so styling was consistent, making charts a bit more bland but easier to consume and check. You cannot achieve that same level of consistency or predictability with ai without reinventing a wheel. The models change too fast, and they all want to differentiate themselves for brand reasons, and the emphasis is on fast and easy, not on slow and foolproof, there’s no documentation of the work the way code provides documentation and an easy audit trail. You can restrict the ai by giving specs (use ggplot, use color template foo, create readable code with unit tests), but these are all similar tradeoffs as you might make with human labor too.
What llm do is to simply lower human value, not remove it entirely.
Just as results needed checking and replication before llm, they will continue to need checking and replication by humans. The humans won’t be compensated nearly as much, but humans are humans and so are not.
This is why reading the whole post is important :)
You seem to think I'm suggesting the use an image generator for plotting. Don't do that, it will screw that up, image models are not made for that. The use for AI here is to save 5 minutes writing the code to generate the plot, or to edit the style (again via code) of the plot.
If you do plt.scatter(x,y, c="blue", marker="o") that is a function call in Matplotlib. Its API notoriously sucks. Most Python plotting libraries just create convenience functions around this library because although the API (actually there are two and the documentation is not consistent on which to use) is hot garbage, and the documentation is labyrinthine, the plotting engine itself is powerful.
> What llm do is to simply lower human value, not remove it entirely.
The value in a plot is not the exact code used to make it (beyond the code being correct). The value is in the visualization giving the reader an "ah-ha" moment. I would much rather take the 5 minutes I save writing a function call, and use it to spend more time examining the plot itself.
This is something the LLMs are really good for, at least for compiler papers.
Mostly quotes:
- 207K consumer user claude sessions (78K code, remainder chat+cowork) in Aug+Sep '26 classified per 7K SciNet taxonomy
- largest domains: computer science (11.9%), engineering (8.4%), and medicine (7.7%)
- higher ratio of usage/researcher: computer science, mathematics, statistics; social sciences
- "Usage is also high for the social sciences" ... "Relative usage is low in [...] the social sciences, with the exception of economics"
- Claude use and AI capability: "the top 20% of tasks also draw two to four times the expected use" "Health sciences researchers use Claude relatively little compared with the AI capability scores of their tasks"
- "Theoretical AI capability also predicts how much researchers allow Claude to work autonomously"
- "productivity gains tend to materialize only after organizations restructure around them (David 1990; Brynjolfsson, Rock, and Syverson 2021)"
- "A surge in new research papers could advance a field faster but also overwhelm scientists’ capacity to absorb or verify the results."
- "As AI becomes more capable and autonomous, AI systems could take the lead in verification, refereeing, and communicating important results, which would ease the absorption problem discussed above"
- "Fifth, more autonomous AI systems may increase the challenge of ensuring that AI-driven research remains aligned with human goals"
- "One effect may be a change in researchers’ job satisfaction."
As a hobby I run https://smalldocs.org / https://github.com/espressoplease/smalldocs an agent agnositc artifact layer, so I'm looking for feedback on artifact use in general.
Here are some sample scientific SmallDoc artifacts:
Palmer penguins: body mass, flipper length and species - https://smalldocs.org/s/YrJDCfCD8-6j_uRE0pNCbI#k=odJRxWOIySV...
AlphaFold 3: capabilities, evidence and operational limits - https://smalldocs.org/s/rRRuByQhmhZNBH_4RNCx1t#k=S0Jhyf72Q6M...
The missing feature for docs is now GDrive, ie organizing and cataloging your artifacts to build a coherent long-lived KB. But for that you can still export.
Anthropic added a while back the ability to let clients provide their own API key, again I think that’s under-explored and will become more relevant as AI gets more penetration.
Building out quick SPAs, HTML pages, etc is a great use, I think the innovation will be making these things supportable and usable for more than one-shot single-user artifacts. Eg user login, analytics, etc.
Did they get permission to use researchers' data or did they monitor and extract data without consent? Considering the kerfuffle OpenAI generated with their math proofs, you'd think Anthropic would take a hint and be clear that they did the research with consent, though that assumes they got consent.
Seriously, all the industry researchers at these companies went through IRB training during their PhD; have they just conveniently forgotten?
https://www.anthropic.com/legal/privacy
`In other news, we report that there seems to be significantly less cybersecurity research done on our platform than other areas of computer science` /s