I expect the trend of large machine learning models to go towards bits rather than operating on floats. There's a lot of inefficiency in floats because typically they're something like normally distributed, which makes the storage and computation with weights inefficient when most values are clustered in a small range. The foundation of neural networks may be rooted in real valued functions, which are simulated with floats, but float operations are just bitwise operations underneath. The only issue is that GPUs operate on floats and standard ML theory works over real numbers.

Doesn't Jevons paradox dictate larger 1-bit models?

How do I run this on Android?

Super interesting, building their llama cpp fork on my Jetson Orin Nano to test this out.

Is Bonsai 1 Bit or 1.58 Bit?

What is the value of a 1 bit? For those that do not kno

[dead]