I think the practitioner angle is what makes interesting. Too many BEAM advocacy posts are theoretical.
I would push back on the "shared state with locks vs isolated state with message passing" framing. Both approaches model concurrency as execution that needs coordination. Switching from locks to mailboxes changes the syntax of failure, not the structure. A mailbox is still a shared mutable queue between sender and receiver, and actors still deadlock through circular messages.
IsTom 15 hours ago [-]
> actors still deadlock through circular messages
I've rarely seen naked sends/receives in Erlang, you mostly go through OTP behaviors. And if you happen to use them and get stuck (without "after" clause), the fact you can just attach a console to a running system and inspect processes' states makes it much easier to debug.
stingraycharles 15 hours ago [-]
Stateless vs stateful concurrency management is very different, though; I can roll back / replay a mail box, while this isn’t possible with shared locks. It’s a much cleaner architecture in general if you want to scale out, but it has more overhead.
cess11 11 hours ago [-]
With OTP you can trivially decide whether you want your sender to block or not, and how you do your decoupling if you decide it shouldn't.
In practice you'll likely push stuff through Oban, Phoenix PubSub or some other convenience library that gives you some opinions and best practices. It really lowers the bar for building concurrent systems.
epicepicurean 10 hours ago [-]
A rewrite of a stateful application written in python with postgres would be more illustrative of how you're solving the same problems but better. Do BEAM applications not use an actual databse? How is crash tolerance guaranteed?
In a typical application I'd write crash tolerance would be handled by the DB. So would transactionality. Without it, one would have to persist each message to disk and be forced to make every action idempotent. The former sounds like a lot of performance overhead, the latter like a lot of programming effort overhead.
I assume these problems are solved, but the article doesn't demonstrate the solutions.
linkdd 8 hours ago [-]
You would have a process handling the calls to the postgres.
That process has as local state the database connection and receive messages that are translated to SQL queries, here 2 scenarios are possible:
1) The query is invalid (you are trying to inert a row with a missing foreign key, or wrong data type).
In that case, you send the error back to the caller.
2) There is a network problem between your application and the database (might be temporary).
You just let the process crash (local state is lost), the supervisor restarts it, the restarted process tries to connect back to the database (new local state). If it still fails it will crash again and the supervisor might decide to notify other parts of the application of the problem. If the network issue was temporary, the restart succeeds.
Before crashing, you notified the caller that there was a problem and he should retry.
Now, for the caller. You could start a transient process in a dynamic supervisor for every query. That would handle the retry mechanism. The "querier process" would quit only on success and send the result back as a message. When receiving an error, it would crash and then be restarted by the supervisor for the retry.
There are plenty of other solutions, and in Elixir you have "ecto" that handles all of this for you. "ecto" is not an ORM, but rather a data-mapper: https://github.com/elixir-ecto/ecto
baud9600 16 hours ago [-]
Very interesting. Reading this made me think of occam on the transputer: concurrent lightweight processes, message passing, dedicated memory! I spent some happy years in that world. Perhaps I should look at BEAM and see what work comes along?
karmakaze 10 hours ago [-]
Likewise. You should read about the Cerebras WSE configurable colour channel mesh.
mrngm 15 hours ago [-]
Related thread from 11 days ago: https://news.ycombinator.com/item?id=47067395 "What years of production-grade concurrency teaches us about building AI agents", 144 points, 51 comments.
vipulbhj 7 hours ago [-]
Author of the post and founder of Variant System here, so cool to finally find out where we been getting all this traffic from.
So many threads I wanna jump in to, interesting discussions.
dzonga 9 hours ago [-]
inverse thinking is needed here - instead of having a solution trying to find a problem.
what would it look like if you didn't need concurrency at all - would simply having a step by step process enough e.g using DAGs
what would it look like if by not letting it crash - you can simply redo the process like a Traditional RDBMS does i.e ACID
they're domains where OTP / BEAM are useful - but for majority of business cases NO
OkayPhysicist 6 hours ago [-]
If you don't need concurrency, then you simply don't need to define any concurrency segmentation. But the real world is wildly concurrent, and most programs will eventually benefit from some degree of concurrency (especially when you can leverage that concurrency into parallelism), so it's beneficial to work in an environment where that improvement can be incremental rather than "we need do a complete rearchitecture to support n=2".
"letting it crash" in BEAM terms often means "simply redo the process". The difference is you end up defining your "transaction" (to borrow database terminology) by concurrency lines. What makes it so pleasant in practice is that you take a bunch of potential failure modes and lump them into a single, unified "this task cannot be completed" failure mode, which includes ~impossible to anticipate failure states, and then only have to expressly deal with the failure modes that do have meaningful resolutions within a task.
With that understanding in mind, I'd argue that nearly all business cases benefit from the BEAM. It's mostly one-off scripts and throwaway tools that don't.
Jtsummers 7 hours ago [-]
> what would it look like if you didn't need concurrency at all - would simply having a step by step process enough e.g using DAGs
What business systems don't use concurrency in some form? I can only think of the simplest data processing tasks written for batch processing. But even every embedded system I've ever developed or worked on used concurrency. Though for older systems this was often hand rolled, and as error prone as you might expect. For newer systems (developed this century), it was often done using a task system baked into the embedded RTOS.
nnevatie 9 hours ago [-]
BEAM/OTP are great, but do impose an exotic language onto the user. Most programs and solutions of today aren't Erlang-based.
OkayPhysicist 6 hours ago [-]
Any software developer worth hiring should be able to pick up a new language (especially one with as great learning materials as Elixir) and become productive in it in so little time that it's rounding error compared to the time to integrate into a new codebase and a new team.
This fear of better languages being some massive hurdle is either unfounded, or the big tech companies paying top dollar for talent aren't getting their money's worth.
Jtsummers 7 hours ago [-]
Erlang is a pretty simple language, it's hardly "exotic". Any competent programmer should be able to pick it up in a short period of time, days to weeks. Now, how long to master the concurrency model, "let it crash" mindset, small processes, and supervisors? Maybe a bit longer.
nnevatie 7 hours ago [-]
Yeah, I meant this from the perspective that for example, I'd love to have that environment and approach in my sleeve, but utilizing C++ and all I have built using it.
How closely is BEAM/OTP related to the foundational work on CSP (and the implementation in Occam/Transputer way back when…)?
lou1306 14 hours ago [-]
Good question! It's a bit of a stretch. BEAM has mailboxes, non-blocking sends, and asynchronous handling of messages, whereas the original CSP is based on blocking sends and symmetric channels. Symmetric means you have no real difference between sends and receives: two processes synchrnoise when they are willing to send the same data on the same channel. (A "receive" is just a nondeterministic action where you are willing to send anything on a channel).
Occam added types to channels and distinguished sends/receives, which is the design also inherited by Go.
In principle you can emulate a mailbox/message queue in CSP by a sequence of processes, one per queue slot, but accounting for BEAM's weak-ish ordering guarantees might be complicated (I suppose you should allow queue slots to swap messages under specific conditions).
loloquwowndueo 11 hours ago [-]
I really tried reading through this but couldn’t - it’s AI-written so it’s like trying to chew cardboard. I gave up after like 3 paragraphs.
emperorz0 14 hours ago [-]
Zero-sharing message passing is known. But what about shared state? Given the majority of systems manage shared access to arbitrarily constrained shared state or shared resources, I'd be interested to see how this should be handled without just saying "database". Maybe another article?
sriram_malhar 13 hours ago [-]
One process is made the logical goto for all operations on that data. The process identity is logically the identity of the shared state.
In other words, it is exactly a database, albeit an in-memory one.
rapsey 17 hours ago [-]
> Backpressure is built in. If a process receives messages faster than it can handle them, the mailbox grows. This is visible and monitorable. You can inspect any process’s mailbox length, set up alerts, and make architectural decisions about it. Contrast this with thread-based systems where overload manifests as increasing latency, deadlocks, or OOM crashes — symptoms that are harder to diagnose and attribute.
Sorry but this is wrong. This is no kind of backpressure as any experienced erlang developer will tell you: properly doing backpressure is a massive pain in erlang. By default your system is almost guaranteed to break in random places under pressure that you are surprised by.
Twisol 17 hours ago [-]
Yes, this is missing the "pressure" part of "backpressure", where the recipient is able to signal to the producer that they should slow down or stop producing messages. Observability is useful, sure, but it's not the same as backpressure.
IsTom 15 hours ago [-]
Sending message to a process has a cost (for purposes of preemption) relative to the current size of receiver's mailbox, so the sender will get preempted earlier. This isn't perfect, but it is something.
matthiasl 8 hours ago [-]
It took me a while to realise that you were responding to the article, not a comment here.
You're right in correcting the article, but I'd like to add that for probably around a decade, Erlang had 'sender punishment', which is what 'IsTom' who replied to you is probably talking about.
Ulf Wiger referred to sender_punishment as "a form of backpressure" (Erlang-questions mailing list, January 2011). 'sender punishment' was removed around 2018, in ad72a944c/OTP14667. I haven't read the whole discussion carefully, but it seems to be roughly "it wasn't clear that sender punishment solved more problems than it caused, and now that most machines are multi-core, that balance is tipped even more in favour of not having 'sender punishment'".
librasteve 16 hours ago [-]
Occam (1982 ish) shared most of BEAMs ideas, but strongly enforced synchronous message passing on both channel output and input … so back pressure was just there in all code. The advantage was that most deadlock conditions were placed in the category of “if it can lock, then it will lock” which meant that debugging done at small scale would preemptively resolve issues before scaling up process / processor count.
baud9600 16 hours ago [-]
Once you were familiar with occam you could see deadlocks in code very quickly. It was a productive way to build scaled concurrent systems. At the time we laughed at the idea of using C for the same task
librasteve 15 hours ago [-]
I spreadsheeted out how many T424 die per Apple M2 (TSMC 3nm process) - that's 400,000 CPUs (about a 600x600 grid) at say 1GIPs each - so 400 PIPS per M2 die size. Thats for 32 bit integer math - Inmos also had a 16 bit datapath, but these days you would probably up the RAM per CPU (8k, 16k?) and stick with 32-bit datapath, but add 8-,16-bit FP support. Happy to help with any VC pitches!
EdNutting 14 hours ago [-]
David May and his various PhD students over the years have retried this pitch repeatedly. And Graphcore had a related architecture. Unfortunately, while it’s great in theory, in practice the performance overall is miles off existing systems running existing code. There is no commercially feasible way that we’ve yet found to build a software ecosystem where all-new code has to be written just for this special theoretically-better processor. As a result, the business proposal dies before it even gets off the ground.
(I was one of David’s students; and I’ve founded/run a processor design startup raised £4m in 2023 and went bust last year based on a different idea with a much stronger software story.)
librasteve 12 hours ago [-]
Yes David is the man and afaict has made a decent fist of Xmos (from afar). My current wild-assed hope for this to come to some kind of fruition would be on NVidia realising this opportunity (threat?), making a set of CUDA libraries and the CUDA boys going to town with Occam-like abstractions at the system level and just their regular AI workloads as the application. No doubt he has tried to pitch this to Jensen and Keller.
mnsc 15 hours ago [-]
I wonder how much the roots of erlang is showing now? Telephone calls had a very specific "natural" profile. High but bounded concurrency (number of persons alive), long process lifetime (1 min - hours), few state changes/messages per process (I know nothing of the actual protocol). I could imagine that the agentic scenario matches this somewhat where other scenarios, eg HFT, would be have a totally different profile making beam a bad choice. But then again, that's just the typical right-tool-for-the-job challenge.
gethly 16 hours ago [-]
Go is good enough.
rustyhancock 14 hours ago [-]
I love the idea of Erlang (and by association Elixir), OTP, BEAM...
In practice? Urgh.
The live is all so cerebral and theoretical and I'm certain the right people know how to implement it for the right tasks in the right way and it screams along.
But as yet no one has been able to give me an incling of how it would work well for me.
I read learn you some Erlang for great good quite a while back and loved the idea. But it just never comes together for me in practice. Perhaps I'm simply in the wrong domain for it.
What I really needed was a mentor and existing project to contribute to at work. But it's impossible to get hold of either in the areas I'm in.
vipulbhj 7 hours ago [-]
I personally really really enjoy writing Elixir. It is a really intuitive way to write programs. Phoenix is a great web framework, and I think all of it is quite approachable. We just had a go programmer start at our org recently and they were contributing to one of our Phoenix bases SaaS apps within weeks
rustyhancock 6 hours ago [-]
It's the converse that's an issue. If your org doesn't use any Erlang.
You're not going to be able to add it.
I don't find that to be true of many other ecosystems.
We could and do have a few Rust tools and webapps.
There is a few older Python/Flask internal applications.
If I went to an org with established tools from the ecosystem then that is not a problem!
vipulbhj 5 hours ago [-]
I would try to build some small utilities etc, nothing big, maybe something in your personal workflow, or that throw away script your team needs.
Erlang is weird, it helps if you have some Lisp and Prolog background, but for a while it might get in the way of learning how OTP works.
socketcluster 16 hours ago [-]
The Node.js community had figured this out long before BEAM or even Elixir existed.
People tried to introduce threads to Node.js but there was push-back for the very reasons mentioned in this article and so we never got threads.
The JavaScript languages communities watch, nod, and go back to work.
pentacent_hq 16 hours ago [-]
> The Node.js community had figured this out long before BEAM or even Elixir existed.
Work on the BEAM started in the 1990s, over ten years before the first release of Node in 2009.
masklinn 16 hours ago [-]
And BEAM was the reimplementation of the Erlang runtime, the actual model is part of the language semantics which was pretty stable by the late 80s, just with a Prolog runtime way too slow for production use.
hlieberman 15 hours ago [-]
Forget Node.js; _Javascript_ hadn't even been invented yet when Erlang and BEAM first debuted.
leoc 16 hours ago [-]
You may be thinking of some recent round of publicity for BEAM, but BEAM is a bit older than JavaScript.
socketcluster 14 hours ago [-]
Haha. I guess the BEAM people can nod down at me with contempt and I nod down at the Elixir folks.
seanclayton 12 hours ago [-]
at least you're doubling down on your ignorance!
14 hours ago [-]
xtoilette 16 hours ago [-]
BEAM predates node js
worthless-trash 15 hours ago [-]
I think the author is trying to be clever to parody what was written in tfa.
16 hours ago [-]
Rendered at 00:05:16 GMT+0000 (Coordinated Universal Time) with Vercel.
I would push back on the "shared state with locks vs isolated state with message passing" framing. Both approaches model concurrency as execution that needs coordination. Switching from locks to mailboxes changes the syntax of failure, not the structure. A mailbox is still a shared mutable queue between sender and receiver, and actors still deadlock through circular messages.
I've rarely seen naked sends/receives in Erlang, you mostly go through OTP behaviors. And if you happen to use them and get stuck (without "after" clause), the fact you can just attach a console to a running system and inspect processes' states makes it much easier to debug.
In practice you'll likely push stuff through Oban, Phoenix PubSub or some other convenience library that gives you some opinions and best practices. It really lowers the bar for building concurrent systems.
That process has as local state the database connection and receive messages that are translated to SQL queries, here 2 scenarios are possible:
1) The query is invalid (you are trying to inert a row with a missing foreign key, or wrong data type). In that case, you send the error back to the caller.
2) There is a network problem between your application and the database (might be temporary).
You just let the process crash (local state is lost), the supervisor restarts it, the restarted process tries to connect back to the database (new local state). If it still fails it will crash again and the supervisor might decide to notify other parts of the application of the problem. If the network issue was temporary, the restart succeeds.
Before crashing, you notified the caller that there was a problem and he should retry.
Now, for the caller. You could start a transient process in a dynamic supervisor for every query. That would handle the retry mechanism. The "querier process" would quit only on success and send the result back as a message. When receiving an error, it would crash and then be restarted by the supervisor for the retry.
There are plenty of other solutions, and in Elixir you have "ecto" that handles all of this for you. "ecto" is not an ORM, but rather a data-mapper: https://github.com/elixir-ecto/ecto
So many threads I wanna jump in to, interesting discussions.
what would it look like if you didn't need concurrency at all - would simply having a step by step process enough e.g using DAGs
what would it look like if by not letting it crash - you can simply redo the process like a Traditional RDBMS does i.e ACID
they're domains where OTP / BEAM are useful - but for majority of business cases NO
"letting it crash" in BEAM terms often means "simply redo the process". The difference is you end up defining your "transaction" (to borrow database terminology) by concurrency lines. What makes it so pleasant in practice is that you take a bunch of potential failure modes and lump them into a single, unified "this task cannot be completed" failure mode, which includes ~impossible to anticipate failure states, and then only have to expressly deal with the failure modes that do have meaningful resolutions within a task.
With that understanding in mind, I'd argue that nearly all business cases benefit from the BEAM. It's mostly one-off scripts and throwaway tools that don't.
What business systems don't use concurrency in some form? I can only think of the simplest data processing tasks written for batch processing. But even every embedded system I've ever developed or worked on used concurrency. Though for older systems this was often hand rolled, and as error prone as you might expect. For newer systems (developed this century), it was often done using a task system baked into the embedded RTOS.
This fear of better languages being some massive hurdle is either unfounded, or the big tech companies paying top dollar for talent aren't getting their money's worth.
Occam added types to channels and distinguished sends/receives, which is the design also inherited by Go.
In principle you can emulate a mailbox/message queue in CSP by a sequence of processes, one per queue slot, but accounting for BEAM's weak-ish ordering guarantees might be complicated (I suppose you should allow queue slots to swap messages under specific conditions).
In other words, it is exactly a database, albeit an in-memory one.
Sorry but this is wrong. This is no kind of backpressure as any experienced erlang developer will tell you: properly doing backpressure is a massive pain in erlang. By default your system is almost guaranteed to break in random places under pressure that you are surprised by.
You're right in correcting the article, but I'd like to add that for probably around a decade, Erlang had 'sender punishment', which is what 'IsTom' who replied to you is probably talking about.
Ulf Wiger referred to sender_punishment as "a form of backpressure" (Erlang-questions mailing list, January 2011). 'sender punishment' was removed around 2018, in ad72a944c/OTP14667. I haven't read the whole discussion carefully, but it seems to be roughly "it wasn't clear that sender punishment solved more problems than it caused, and now that most machines are multi-core, that balance is tipped even more in favour of not having 'sender punishment'".
(I was one of David’s students; and I’ve founded/run a processor design startup raised £4m in 2023 and went bust last year based on a different idea with a much stronger software story.)
In practice? Urgh.
The live is all so cerebral and theoretical and I'm certain the right people know how to implement it for the right tasks in the right way and it screams along.
But as yet no one has been able to give me an incling of how it would work well for me.
I read learn you some Erlang for great good quite a while back and loved the idea. But it just never comes together for me in practice. Perhaps I'm simply in the wrong domain for it.
What I really needed was a mentor and existing project to contribute to at work. But it's impossible to get hold of either in the areas I'm in.
You're not going to be able to add it.
I don't find that to be true of many other ecosystems.
We could and do have a few Rust tools and webapps.
There is a few older Python/Flask internal applications.
If I went to an org with established tools from the ecosystem then that is not a problem!
You can still enjoy the language
Erlang is weird, it helps if you have some Lisp and Prolog background, but for a while it might get in the way of learning how OTP works.
People tried to introduce threads to Node.js but there was push-back for the very reasons mentioned in this article and so we never got threads.
The JavaScript languages communities watch, nod, and go back to work.
Work on the BEAM started in the 1990s, over ten years before the first release of Node in 2009.