Cloud providers have put a lot of time and effort into making you believe every web app needs 99.9999% availability. Making you pay for auto scaled compute, load balancers, shared storage, HA databases, etc, etc.

All of this just adds so much extra complexity. If I'm running Amazon.com then sure, but your average app is just fine on a single VM.

> Then before you know it, the devops folks have decided that they need to put a gazillion other services and an entire software-defined networking layer on top of it.

I'm not familiar with kubernetes, but doesn't it already do SDN out of the box?

>Incidents have also doubled or tripled, as has downtime

>And yes, a single VM really is fine

Same experience here, we have both setups (VM and k8s) across 2 projects. The project with a single VM is much, much more stable. Downtimes are extremely rare. Hotfixes are applied in under a minute after the first bug reports (because it's also just the good old PHP - just edit a file and boom it's done!). The project that runs on k8s (to support the microservice-oriented architecture) has downtimes like every week. Too many moving parts and harder to debug, which microservice calls which. Changes are harder to roll out. K8s and the microservice architecture were introduced by the former CTO who bought into the hype in the late 2010's. Another fun thing is that many B2B clients now want to run the product on-prem and they're forced to install k8s clusters on their hardware too. And obviously it's also pretty unstable there too, because not every client can afford a whole DevOps team just to manage some random helpful product. The possibility of selling an on-prem version is something to consider when choosing whether to design for a single VM or for k8s. Lots of money there at this moment, well at least in the region where we sell the most (dunno about the US).

And those devops folks just let your single debian VM be? It sounds like you have, like many of us, an organizational/people problem, not a k8s problem.

Maybe those devops folks only pay attention to k8s clusters and you're flying under their radar with your single debian VM + Kamal. But the same thinking that results in an overtly complex, impossible to debug, expensive to run k8s cluster can absolutely result in the same using regular VMs unless, again, you are just left to your own devices because their policies don't apply to VMs, yet.

The problem usually is you're one mistake away from someone shoving their nose in it. "What are you doing again? What about HA and redundancy? slow rollout and rollback? You must have at least 3 VMs (ideally 5) and can't expose all VMs to the internet of course. You must define a virtual network with policies that we can control and no wireguard isn't approved. You must split the internet facing load balancer from the backend resources and assign different identities with proper scoping to them. Install these 4 different security scanners, these 2 log processors, this watchdog and this network monitor. Are you doing mtls between the VMs on the private network? what if there is an attacker that gains access to your network? What if your proxy is compromised? do you have visibility into all traffic on the network? everything must flow throw this appliance"

If you spin up Kubernetes for "a couple of containers to run your web app", I think you're doing something wrong in the first place, also coupled with your comment about adding SDN to Kubernetes.

People use Kubernetes for way too small things, and it sounds like you don't have the scale for actually running Kubernetes.

I thought we collectively learned this with stack overflows engineering blog years ago.

Scale vertically until you can't because you're unlikely to hit a limit and if you do you'll have enough money to pay someone else to solve it.

Docker is development tooling. Not production infrastructure.

Clearly, Kubernetes wasn’t the right solution for your case, and I also agree that using it for smaller architectures is overkill. That said, it’s the standard for large-scale production platforms that need reproducibility and high availability. As of today I don’t see many *truly* viable alternatives and honestly I haven't even seen them.

DevOps lost the plot with the Operator model. When it was being widely introduced as THE pattern I was dismayed. These operators abstract entirely complex services like databases behind yaml and custom go services. When going to kubecon i had one guy tell me he collects operators like candy. Answers on Lifecycle management, and inevitable large architectural changes in an ever changing operator landscape was handwaved away with series of staging and development clusters. This adds so much cost.. Fundamentally the issue is the abstractions being too much and entirely on the DevOps side of the "shared responsibility model". Taking an RDBMS from AWS of Azure is so vastly superior to taking all that responsibility yourself in the cluster.. Meanwhile (being a bit of an infrastructure snob) I run Nixos with systemd oci containers at home. With AI this is the easiest to maintain ever.

First time I’ve heard of Kamal. Looks ideal!

Do you pair it with some orchestration (to spin up the necessary VM)?

I don't get it, I think that k8s is the best software written since win95. It redefines computing in the same way IMHO. I have some experience in working with k8s on prod and I loved every moment of it. I'm definitely missing something.

If you replaced k8s with a single app on a single VM then you’ve taken a hype fuelled circuitous route to where you should have been anyway.

I think this comment and replies capture the problem with Kubernetes. Nobody gets fired for choosing Kubernetes now.

It's obvious to you, me and the other 2 presumably techie people who've responded within 15 mins that you shouldn't have been using Kubernetes. But you probably work in a company of full of techie people, who ended up using Kubernetes.

We have HN, an environment full of techie people here who immediately recognise not to use k8s in 99% of cases, yet in actually paid professional environments, in 99% of cases, the same techie people will tolerate, support and converge on the idea they should use k8s.

I feel like there's an element of the emperors new clothes here.

i was just curious so i tested this actually.

Using fio

Hetzner (cx23, 2vCPU, 4 GB) ~3900 IOPS (read/write) ~15.3 MB/s avg latency ~2.1 ms 99.9th percentile ≈ ~5 ms max ≈ ~7 ms

DigitalOcean (SFO1 / 2 GB RAM / 30 GB Disk) ~3900 IOPS (same!) ~15.7 MB/s (same!) avg latency ~2.1 ms (same!) 99.9th percentile ≈ ~18 ms max ≈ ~85 ms (!!)

using sequential dd

Hetzner: 1.9 GB/s DO: 850 MB/s

Using low end plan on both but this Hetzner is 4 euro and DO instance is $18.

>3000 IOPS

If that's true, I wonder if this is a deliberate decision by cloud providers to push users towards microservice architectures with proprietary cloud storage like S3, so you can't do on-machine dbs even for simple servers.

Many cloud vendors have you pay through the nose for IOPS and bandwidth.

Edit: I posted this before reading, and these two are the same he points out.

> Cloud vendor pricing often isn't based on cost.

Business 101 teaches us that pricing isn't based on cost. Call it top down vs bottom up pricing, but the first principles "it costs me $X to make a widget, so 1.y * $X = sell the product for $Y is not how pricing works in practice.

I think we, as engineers, are a bit stuck on what “software” has traditionally been. We think of systems that we carefully build, maintain, and update. Deterministic systems for interacting with computers. I think these “traditional” systems will still be around. But AI has already changed the way users interact with computers. This new interaction will give rise to another type of software. A more disposable type of software.

I believe right now we are still in the phase of “how can AI help engineers write better software”, but are slowly shifting to “how can engineers help AI write better software.” This will bring in a new herd of engineers with completely different views on what software is, and how to best go about building computer interactions.

Sometimes “better” means “customized for my specific use case.” I expect that there will be a lot of custom software that never appears in any app store.

> Let's hope the focus shifts from code generation to something else. There are many ways LLMs can assist in writing better code.

My view is actually the opposite. Software now belongs to cattle, not pet. We should use one-offs. We should use micro-scale snippets. Speaking language should be equivalent to programming. (I know, it's a bit of pipe dream)

In that sense, exe.dev (and tailscale) is a bit like pet-driven projects.

> I don't understand why we are so obsessed with cranking out even more... the obvious usecase for LLMs should be to write better software

I honestly think this is ideal. Video games aside, I think one day we'll look back and realize just how insane it was that we built software for millions or even billions of users to use. People can now finally build the software that does exactly what they've wanted their software to do without competing priorities and misaligned revenue models working against them. One could argue this kind of software, by definition, is higher quality.

Both will likely happen to some degree.

As for the average quality: it’s unclear.

My intuition is that agents lift up the floor to some degree, but at the same time will lead to more software being produced that’s of mediocre quality, with outliers of higher quality emerging at a higher rate than before.

Alas, we shifted from quality to quantity somewhere in the mid 19th century.

There will be only 1 Microsoft® Excel, 1 Google Sheets and 1 LibreOffice and the rest are billions of dead vibe-coded "Excel killers" that no-one uses.

I find it difficult to configure Tailscale for my use case because they seem to completely not support making ACL rules based on the identity of the device rather than a part of the address space. I'm not configuring a router here, I'm configuring a peer-to-peer networking layer... or at least I'm supposed to be...

Companies buy cloud services because they want to reduce in-house server management and operations, for them it's a trade-off with hiring the right people. But you are right, when you can find the right people doing it yourself can be a lot cheaper.

Agree, I used to always use Heroku or Render style platforms for my own software, but nowadays I just have a Linux server with Docker Compose and a Cron job. The cron job every minute runs docker pull (downloads latest image) and docker up -d (switches to new version only if there is a new version). And put caddy in front for the HTTPS. This has been very cheap and reliable for years now.

Especially these days you can SSH to a baremetal server and just tell Claude to set up Postgres. Job done. You don't need autoscaling because you can afford a server that's 5X faster from the start.

we've done both. Hetzner dedicated was genuinely fine, until a disk started throwing SMART warnings on a Sunday morning and we remembered why we pay 10x elsewhere for some things. probably less about the raw cost and more about which weekends you want back.

Honestly I like Hetzner a lot but lately it has been very unstable for us. https://status.hetzner.com/ this page always has couple of incidents happening at the same time. I really appreciate the services they provide but i wish they were more stable.

Because if I have a government service with millions of users, I don’t want the cheap shitter servers to crap out on me.

An employee is going to cost anywhere between 8k and 50k per month. Hiring an employee to save 200/month on servers by using a shitty VPS provider is not saving you any money.

[dead]

Investment is done by relationships, belief in a future vision and team, and growth metrics like number of paying customers.

The technology itself in its current form is not valuable

Me too. I already moved our products to it and it is getting fairly robust. Guess many smaller companies got tired with the big guys asking a lot of money for things that should be cheap.

Yeah i feel like it's getting cloudy

Don’t worry - that will certainly change in the future if they have any kind of success :)

That’ll be true for the 0.1% of project that were limited by the speed of their programming language. For the other 99.9% of projects their vibe coded rust can fly and their database, network, or raw computation will still be the bottleneck.

(Percentages cited above are tongue-in-cheek, actual numbers are probably different)

You choose a region. Then you pay for some compute size (vcpu and mem), and then you can create a lot of VMs using those limits. If some VM's don't consume all resources, others can consume it in burst.

VMs have a built-in gateway to cloud providers with a fixed url with no auth. You can top that in via the service itself. No need for your own keys.

So likely a good tool for managing AI agents. And "cloud" is a bit of a stretch, the service is very narrow.

The complete lack of more detailed description of the regions except city name makes it really only suitable for ephemeral/temporary deployments. We don't know what the datacenters are, what redundancy is in place, no backups or anything like that.

As I understand, a cloud provider where instead of paying for each VM (with a set of resources), you pay for the resources, and can get as many VMs as you can fit on these resources.

Their first location (PDX) is on Amazon I believe and not accepting new customers. They’ve said it’s much more expensive for them than the others. Their other locations are listed here:

https://exe.dev/docs/regions

Well yes, because they needed high availability and flexibility and tons of features…

Hey wait a minute!

"I am white labeling a cloud"

For me it’s so my coding agent keeps running when I close my laptop lid and it goes to sleep. VM in the cloud because I’m too lazy to set up a computer to be running as a server all the time.

I don't think SSH vs OpenTofu is the core issue here.

For agents, declarative plans are still valuable because they are reviewable. The interesting question is whether exe.dev changes the primitive: resource pools for many isolated VM-like processes, or just nicer VPS provisioning.

It's hard to see the scale of what he's doing. Could be:

- I'm building a server farm in my homelab.

- I'm doing a small startup to see if this idea works.

- We're taking on AWS by being more cost effective. Funding secured.

inflation