NHacker Next
  • new
  • past
  • show
  • ask
  • show
  • jobs
  • submit
▲Deterministic Simulation Testing Is for Everyone (outdata.net)
csbartus 4 hours ago [-]
Today this is the second post I see where the 'traditional' testing methods (unit, e2e, integration, perf, pen, etc) are re-categorized, are put in new perspectives.

The other one is https://alperenkeles.com/posts/vocab-for-testing/

I truly understand our (only) job is to become the oracles of the machine: https://www.osequi.com/manifesto

Mitchem 3 hours ago [-]
I'm betting on something adjacent but flipped. As agents are writing the code, humans are stopping (and getting worse at) reviewing, so the oracle needs to be mechanical. My personal bet: https://github.com/NMitchem/charpente/blob/main/MANIFESTO.md
cipehr 3 hours ago [-]
Not sure I’m following, are you implying that deterministic simulation testing is a just a new name for a “traditional” test method? I’m not clear because 1. Deterministic simulation testing is not a new concept. (Foundation DB is one of the poster boys for it) 2. All the test methodologies mentioned are a name for a specific pattern and practice of testing, and DST is the same thing.
drooby 3 hours ago [-]
This is the future, I'm convinced.

I’ve been building a framework of my own and it’s already found dozens of important and critical bugs on my team at work.

The next problem to solve after this, though, is formalized intent.

A bug is often a bug if the app is not behaving as intended. What we intend is not always so clear, and it will remain an area of the engineer's and product designer's responsibility, because software is complex, and what we want cannot be computed so simply. Defining and understanding intent will remain a "human bottleneck" for many companies.

getleakcheck 3 hours ago [-]
[flagged]
ngamboa 1 hours ago [-]
[dead]
TacticalCoder 1 hours ago [-]
Deterministic testing is one thing and it's badly needed.

But I'm totally convinced we're moving to deterministic execution: n various implementations, in different languages, running on different stacks, and a minimal (maybe even hardware) voter taking care of the n-modular redundancy to pick the correct "answer".

0-day Postgres CVE? No problem: one of the implementation gives bogus answers and is detected as broken.

0-day Java? Same thing.

This shall make 0-day in anything but the voters a thing of the past (and the voters shall be minimal and have a tiny attack surface).

The answer can be an API query but we can go much lower: network packets, deterministically created, by n systems and all going to the voter(s). Whatever the "answer" is, it's going to be deterministic and verified from the lowest possible level.

The implementations shall be written from specs, formally verified, etc. But the one thing that'll guarantee any fuck up is caught is going to be n independant implementations of the same specs.

And as soon as one of the implementation/stack return even just one network packet that doesn't match what the other systems do produce: something is fucked up and has to be fixed.

This shall require throwing a huge lot of the non-deterministic bullshit away but we'll see systems doing just that: n-modular redundancy and voters, at the lowest level possible.

I'm 100% sure we're going there (and AI is going to write most of that for us).

kulukrok 1 hours ago [-]
interesting idea. any references (long-form) on this?
haukebri 52 minutes ago [-]
[flagged]
felixyoung 4 hours ago [-]
[flagged]
Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact
Rendered at 20:05:41 GMT+0000 (Coordinated Universal Time) with Vercel.