It's not just predicting content but also the form/shape of the response, highly shaped by RLHF preferences, which seems to encourage longer form responses which need padding with filler to generate the "preferred" amount of verbiage.

It seems most of these models have been trained to interpret questions as wanting a long-form authoritative/comprehensive-sounding response (even when midwit filler is all it has to offer), rather than initiating a short-turn conversation which would better allow you to steer the conversation. Apparently Muse is much better in this regard, but so far I've resisted signing up for any Meta/FaceBook service, and not inclined to break this winning streak!

I also try to keep the llm-enhanced (or even fully llm-generated) content separate from my human writing - because it is true that the failure modes are different. But think LLM writing might be useful even if it is not consumed directly - it can enhance LLM answering your questions.

(this isn't a new observation / thing, see https://randsinrepose.com/archives/nadd/ originally from somewhere in the 2000s)

But now with AI it's easy to generate a lot of prose. I'm thinking for example of the merge request descriptions my colleague submits. And here's the thing, it's not even bad or incorrect - I read it thinking, if I were to really sit down for it, get into The Zone etc, I could crank this out myself.

But I don't because honestly the MR is relatively trivial and can be described in just a sentence or two. Style and accuracy aside, my current take is that LLMs write things that humans wouldn't bother with, because it's too much effort and there's not enough value.

I mean it's an MR for an enterprise library project, the chances that anyone will ever read that MR description again once it's merged is close to zero.

That said: LLMs themselves don't get that, they will read everything you tell them to with the same amount of attention. Granted, their context windows are limited (and performance declines the more you put in it), so they too will use some tools to skim and pick out the important parts, but I think that's a temporary thing. The huge amount of LLM prose will end up being consumed by other LLMs. I don't know enough about the subject whether that's a good or a bad thing though, right now I think it just adds to the noise until it becomes noise in, noise out.

> how awful everything is to read lately > flooded with AI generated content, all of it terrible > everything sounding identical and everything being a rambling unconcise mess.

I can see the benefits of AI and enjoy using the technology - but, like many have said, it's wreaking havoc on the "content" that's out there nowadays. Gotta take some bad with the good, I suppose.

AI has made it even worse and lots of content on the web, for both these reasons, is not worth you time (now wonder users don't read most of the text on our apps!)

If you built a model only based on long form journalism and The New Yorker voice, including both the template structure of the article and the grammar and syntax, the model would replicate the patterns it is built upon. You could do the same for high vocabulary rap.

When you stuff all of English or all of human thought into it with somewhat equal weight, including very technical writing the sum is a lack of style and elegance and personality.

The second problem is that writing often doesn’t happen linearly. You write bookends and fill in, edit and revise. You sort of know where you’re going and can then make decisions how to get there. Language models may benefit from being redesigned to perform passes. Select a template of some sort, fill in the mad lib, then run a detail pass, then run a persuasion / emotion pass, then run a style pass. Current model writing very much sounds like I would expect autocomplete to.

"You can be pro AI productivity but anti-AI creativity. Automating creativity is the death of art."

Meaning you reject the creative value of AI but values it's ability to drive a car safely, or write code for your job, or turn off the lights when you leave the room. objective goal oriented task completion great.

But AI draws your a picture? What's the point. AI writes you a song? No feeling. AI creativity isn't interesting.

In AI-generated entertainment, the names "Marcus", "Chloe", "Sterling", "Meridian", and "Apex" keep appearing. Those aren't common names, but LLMs seem to prefer them.

OP’s linked article mentions the importance for them of reading their writing aloud. Humans construct sentences with a (mostly unconscious) sonic influence. How things sound really matters.

My theory is that LLM prose sounds slop-like because the base unit for an LLM is not a single sound.

Thus, perhaps, you get the uncanny, meandering, hollow-feeling outputs, which feel vapid to human readers, without the reader really knowing quite why.

I think this is also part of the reason why LLMs are rewarded for and thus more strongly exhibit their faux-punchy constructions, such as the infamous “it’s not [X], it’s [Y].” The transformer model can, with current tokenization, articulate some features of “good writing”, but not quite all.

I think there are a number of fairly simple SOLNs to this (or ways to test this hypothesis) without making tokenizers purely syllabic across the board, which would probably be a mistake.

Whether prose output quality even matters for models which are not explicitly trained for creative writing is highly debatable, though.

For me it's mostly the latter, which is a natural consequence of LLM writing. When the (time) cost-per-word goes down, there is less incentive to keep signal-to-noise ratio high, so it ends up verbose and bloated. I also believe that this is (at least partially) upstream from everything sounding identical: take two articles on two different subjects - when SNR is high, there isn't really much room for them to become similar (The ingredient list for a risotto will never read the same as the ingredient list for pizza), but when 80% of them is fluff, they can blend together more easily (To take the recipe comparison, this would be the background/story part above the recipe), especially when written by the same LLM

My parents live on a rural property, and I'll help them do research on things like livestock practices or occasionally "prepper" related things to give them a sanity check before they spend money. I have to weed through a dozen or more obvious ChatGPT-created pages that are the first page or two of results on every search engine before I get to anything a human has touched. The vast majority of the LLM-created pages are either massively verbose and vague, or often wrong. But they're getting clicks. Very often, when asking an LLM the same question and to give citations, I'll get a combination of useless, irrelevant sites with no connection to the material, maybe one of these LLM-generated sites, and maybe one online forum.

I'm not aggressively pro-AI, I'm in the middle, but stuff like this has me worried that we're looking at the equivalent of the Dark Ages all over again. But instead of simply no one bothering to write anything down as was the case in the early Middle Ages, it's going to be the task of searching for real human work and thought in a sea of aggressively spammed LLM-generated slop pages.

Endless headlines like, "aArON SORkIn SAYs hE WiLL Never UsE ai to WRite" doesn't change that base fact. No one asked him to, other than clickbait journalists.

If you find this hard to believe, just think about relative market sizes.

Yeah, thanks dude. We appreciate your efforts to enshittify everything.

Edit: Sure, downvote, but the parent comment is really not very different from "arsonist puzzled and upset that so many buildings have burned down".

I've noticed this a lot in conservative spaces especially, when they say "I love Trump, but..." -- why the need to say how much you love AI before criticizing it? Who are you trying to convince?

I use LLMs constantly for work and they have genuinely improved my life. But I absolutely despise generated "content". Text, audio, video, images -- for the reasons you describe.

It's ok to have a nuanced and multi-faceted opinion and approach to this and not have to call yourself a raving fanboy.

Maybe that's the key ingredient that's missing from LLM-generated text. Even when there is actual content there, there's this lack of surprise / uncertainty in its delivery that we require to stay engaged. We may not know what will be communicated, but knowing exactly how it will be communicated just kills our innate interest.

For foreign text, sure. But I never was never able to read New York Times articles about some invention that started with Lucy leaving her home at seven o'clock.

If you have a specific question, reading LLM text can be more interesting than anything in the world. Even Wikipedia.

These aren’t deep AI hustle bro hacks, it’s literally just editing some text. If a big part of your day is reading AI-generated words, slap something together to make it do words gooder.

I find that it often isn't just too low, but also too high at the same time. It has a lot of information that is irrelevant, burying far too little relevant information for the work needed to extract it. I wonder if this is a side effect of attention mechanism, where the AI's ability to track information in a document is far less hampered by other information, leading to it organizing information to be both dense per word but any specific concept to be sparsely spread throughout the writing. I also wonder if this is why the order feels off, because the AI doesn't need for the information to be well ordered to the same extent that a human does, so the order of storing information is often overlooked when storing it into a document (and even information that needs to be ordered might have the ordering information separate from how it is ordered in the document).

Which requires a bunch of things an LLM lacks, such as theory of mind, knowing who it is talking to, and willingness/ability to engage in short turn conversation to assess and steer the conversation according to the feedback it is getting from the person it is talking to.

IMO, it's still better to not let an AI critique your writing style at all, only the facts and claims you're making against its context / MCPs. Your own writing style is a reflection of your mind's thought process and, especially on forums like this one, is the only way to imbue your ideas with your personality.

Now there's an idea. LLMs may have taken the em-dash from us, but the em-en dash is unknown to Strunk & White and might be human-exclusive for some time yet.

I'm all for having these bots process inquiries if it means a speedier resolution, but, please, anyone out there using these things, turn off the simulated sympathy. It's infuriating how condescending it comes across.

The typewriter was the beginning of the end of the art of handwriting.

Factories was the beginning of the end of hand-created goods.

AI has killed the art of handwritten code (which itself automated away other forms of art).

Look at programming. What's happening there is we are having agents produce 10-100x the code to where people give up on reading it and just hit OK. Look at math. Plenty of people there want to learn from the LLMs and all these crazy new math proofs. But those are the people who will probably be out of a job soon because the money in math is in producing proofs, and that's now an LLM-exclusive job.

I think LLMs are weird in that the first order effects are those of an incredible and amazing tool that anyone can learn things from, but the second order effects are going to be the exact opposite - people giving up on bothering with ideas. Even at my job, which is one of the few that has not gone crazy with AI, anytime I ask anyone a question, all I hear is "have you asked Claude?". There's no communication anymore, no pushback, no struggle. I worry that one day we'll look around and realize we've built an entirely separate second civilization of machines and robots that acts more like a neighboring country than a tool we can use to build humanity up.

-Just my rambling thoughts, not refined by an AI.

(I have to admit, I truly do like some of the music, so feel free to scorn me. If you want to be infested with a vicious ear-worm check this out: https://www.youtube.com/watch?v=zONGVaONKe8&list=RDzONGVaONK...).

Given the amount of angst being expressed in this thread about particular LLM-isms, I wonder why people don't simply prompt against the traits they hate. Yes, it will reduce output quality very slightly in some circumstances, but if it really gets your goat like this, surely it's worth it.

AI does the same with responses where the cadence is some type of formula that becomes identifiable.

Each of the big models has their own but an example of the cadence is

A) Agreement or Acknowledgement with the User

B) Specific detail gets a paragraph of detail related to the previous response

C) Overview of the conversation's current state

D) Prompting or Direction for this conversation to go

E) Landing on a beat. Wrap up the vibe. Recap with a zinger.

I cannot unsee it and a few of the models are annoying. I have attempted to build out the Agents and context to avoid the 'last beat one liner' without success because after 2-3 more turns, I get "That challenge was accepted and you stepped up. You're the winner this round".

There are no objective criteria that can say "this text is good" or "this text is bad". Sure, spelling and grammar errors... but we're talking about writing style, "voice", and whether the text is easy for humans to read.

Now, there are plenty of problems with LLM-generated code that can't be solved by having good tests. Is it maintainable? Is it written in good coding style? Those are harder to quantify. But since LLMs are such good mimics, if you tell an LLM to add code to your existing project, then it's usually going to do a decent job of matching your coding style. The question of maintainability still rears its head; my personal experience is that the smaller the chunk of code the LLM generated for me, the more likely it is I can understand it fully and therefore maintain it in the future. An entire vibe-coded project? Good luck maintaining that. A 100-line class with five 20-line methods with names that make sense? Yeah, I can take that and maintain it in the future, even if the LLM tool goes away.

But mostly, the difference between the acceptance of LLM text or of LLM code, IMHO, boils down to "are there objective measures that I can use to accept this"? Code has to work, which is a much higher bar than text (which just has to have correct spelling and grammar), and therefore it's easier to get the LLM to spit out decent-ish code than decent-ish text.

It’s not ok to write stories with AI and post them for others to read claiming “I did that”

Get the difference?

That is not to say that I don't think there is a place for LLM assisted writing. Writing is an exercise for the author and translates to oral argumentation so I don't think that fades away. But LLMs could do some "audience normalization". What if I could make a blogpost that is dynamic based on experience of the reader? There are lots of opportunities, but versus other fields where optimizing for laziness can be beneficial I don't think lazy writing currently has much value add.

It's your turn to host supper club this week and you serve frozen diners on your nice tableware.

Add to that the fact that AI is significantly worse for you than TV dinners, and has an outsized harm to humanity, and the conclusion is plain:

AI should not be imposed on others.

Same thing with LLM-assisted writing. It’s one thing to do it for yourself, quite another to share it with other folks without warning.

Most food is literally just fuel. I think that is fine. And at some level, eating 3$ burritos for a quick lunch on the daily really makes you appreciate the depths of a home made risotto even more. :]

right. he is talking about the first group. where did you get the 'shame' part?

I like your analogy. Microwaved empty calories perfectly describes llm output. You are going to pay the price sooner or later.

You can love to cook and enjoy every second of it, but sometimes you just need fuel and that $3 microwave burrito is sufficient. Just the same, 99% of your means could be frozen TV dinners, but that doesn't mean cooking a romantic meal for Valentine's Day is off the table.

There's a weird level of fear mongering occurring around AI that it will make you incapable of independent thought if you use it. People will throw around very light research that kind of points in that direction, but they'll never highlight any of the nuances in such an argument. You can use AI and still "think" if you choose to, just like how you can avoid AI and still opt out of "thinking" if you choose to. It has less to do with AI and more to do with the context of the individual using AI.

* Every user run becomes a memo

* Days summarize, rolled to weeks, rolled to months - all written once

* Today's runs are shortened and placed into running context past the cache break

Then while reading it before publishing it, I thought it sounded super LLM-y. I didn't rewrite it because I jammed that stuff together before hitting publish because I was in a hurry but now that I go re-read it, it's somehow worse. Dear god. It's happening.

The classic thing is this business with inanimate objects verbing metaphors or becoming other objects. It's a strangely uncanny valley, but I suppose I read so much of this with Claude that it gets stuck in my head!

It is impossible to reconstruct the intent/spec from the code. More often than not the intent is not clear in the code, and one has to track down the original author to check their context and decision at the time.

As the saying goes, naming things well is a hard problem. It's kinda mystical in a way, where only by knowing an entity's true name will one understand it.

However if it's communicated between agents one may care even less.

I think the key difference is that code is a tool used for a given purpose. It can be slop, or structured better or worse, but at the end of the day it exists to be executed, and its execution is either correct or not. Additionally, good code that does a given thing can only be expressed so many reasonable ways, probably. So LLM-generated code can be perfectly fine.

By contrast, human writing is about exchanging thoughts. Often those thoughts are about infinitely complicated and poorly understood systems, like other people, or emotions, or politics, so it's not a matter of exactly encoding something but attempting to describe it imperfectly. All the tiny little details encoded in how someone writes something can work to this end and are also a product of the writer's thought patterns and overall personality; this all conveys useful information to the reader. In other words, there is a much greater degree of fine detail that affects the quality of writing than of code, and LLMs suck at those fine details for most subjects that are interesting. On top of that, they don't have original thoughts.

Not least because it’s getting worse rather than better. But also because as AI gets better at writing like humans, humans will change how they communicate to avoid being confused with it.

Which will get a lot easier once human writing is nearly extinct. Now that could happen in two years.

The reason people notice "AI voice" is not that the AI voice is itself remarkable. It's that it appears so frequently. Every literate person spent nearly their entire life reading text written by a variety of different individuals. Dozens of different voices in your morning paper. Hundreds of different voices on every web forum. You get the idea.

Now, imagine that all of those voices were replaced by one guy (or two guys, or three guys if you're a Gemini believer). Everything in the newspaper is written by one guy. Every comment on reddit, and HN, and the NYTimes comment section, all written by the same guy. Other people tell him what they're trying to say, but he's writing all of it. Some people might even ask the guy to write in a different voice, but the voice is just that guy doing an impression of someone else, it's not actually someone else.

That guy's voice would probably start to get pretty grating. You'd start to notice every tell, every frequent word choice, every formatting quirk. You'd start to wonder if that guy was actually a good writer, or if having him write everything was just more convenient for everyone. You might even start writing like him yourself. This would be true even if that guy was an absolute virtuoso of the language and his writing was unassailable in a vacuum.

If everyone is using the same (small set of) model(s), then it doesn't matter if the model is GPT-3 or Claude Legendarium 9. Unless the operator - which in theory doesn't need to be a human - is doing something to move it off of its default voice, the outputs are going to sound the same, and you're eventually going to figure out, and get annoyed by, its tells.

Because, for sure, some of the real bangers of comments I've posted on HN and similar forums—all my XCompose-configuration aside—have gotten relatively few upvotes!

Same thing with conference talks. The ones with half a million or a million views on YouTube… the real magical ones, I think, are the ones no one seemed to pay any attention to… e.g., https://www.youtube.com/@PyDataTV/search?query=james%20powel...

The problem with LLM voice is that it sends the signal that it's a likely waste of effort to try to figure out if there's any "technical insight" there, because more often than not, there isn't.

Maybe it's not fair that you're paying the price for others using LLMs to abuse their readers, but it's the reality that you have to deal with. It's also not fair to expect readers to wade through all that abusive LLM prose in order to find your (assumed) non-abusive post.

Once LLM writing becomes indistinguishable from human writing, it will be really sad. There will just be a collapse of human engagement, and you might as well just talk to your machine because then at least you know what you're getting and aren't being deceived.

Technical skills aren't worth shit and never were, except as far as they enable you to engage with other people in ways you and they find valuable. This used to be really far! It made it easy to believe that the technical skills on their own were what was valuable.

But the engagement was always the valuable thing, generating technical insight was a necessary prerequisite for it. Now anyone with a $20 claude sub can generate technical insight into niche problems, so what's left is everything else. Everything of value.

It's the modern equivalent of going to a thesaurus and replacing all your natural words with five-dollar ones.

If you're not confident in your English then just write your brilliant ideas in broken English. Or at most do a direct translation. But don't fluff it with AI's nonsense.

If the ideas are good they will be recognized.

I can’t distinguish between well meaning slop and blogspam slop. I might be able to if I’m well informed on the topic and can point out inaccuracies. Otherwise I just don’t know what is accurate and what is filler or outright wrong. LLM-isms are very distracting and exhausting to sort through. It just doesn’t make for a good reading experience.