Personal anecdote: I exhausted my Claude usage yesterday so I decided to spend 20$ to try Kimi while I was at it. Logged in, paid, downloaded Kimi Code, set it to use K3 and prompted something along the lines of: "Check this repository and find all the settings that can be passed as input related to hardware, I/O, thread control or networking. Produce a report".
It thought for about 12 minutes and then told me I had exhausted my daily quota. (The next day Fable did the same task in 3m)
If you want to buy a plan for K3 do NOT buy the 20$ one.
impossiblefork 49 minutes ago [-]
I think the Kimi thing is super cool, especially that they have so many RNN/linear attention layers (3x more than they have full attention). I haven't yet tried it though. It seems like it would be extremely reasonable for long context tasks and I guess this fits the times.
I suspect that the reason it has so many parameters is the same reason that compute optimal xLSTMs have some many parameters, and the success of this model makes me a bit unhappy that we haven't gotten an xLSTM-style model of huge size developed in Europe.
Obviously these guys are very pragmatic, they're probably not committed to anything other than what works on their internal evaluations, so they still have ordinary attention layers in the model and so on, and one can't be guaranteed that the people who come up with a good model then do the engineering in an ideal way, but I still think the success of Kimi shows that it could have been if we had enough big supercomputers for LLM training and made them available to the right people-- because this is basically Hochreiter's thing. It's RNNs, or well, mostly RNNs.
georgeburdell 34 minutes ago [-]
Wasn't one of the original intents of the transformer architecture to get rid of RNN because they cannot be parallelized? Not an expert, just read a few papers several years ago.
impossiblefork 25 minutes ago [-]
Yes, although then people came up with reasonable ways of parallelizing RNNs entirely or in part anyway. There's also the need for lookups in transformers. Queries still need to be multiplied with old keys, all of them, so RNNs don't have to be totally incompatible with parallelism. It's just that you can't have a matrix inside them like in the linear RNN h_t = Ah_t + Bx_t that you have to multiply together step after step. If A is an input-dependent scalar or something you can multiply together easily you're fine.
mLSTM layers are completely parallel (the update rule for the cell state is C_t = f_t C_{t-1} + i_t x_t where f_t and i_t are gates that can be computed from x_t alone, x_t is the input at time t and this means that you can compute F_s = \prod_{s<t} f_s as fast as a cumsum and an exponential and then compute C_t = \sum_{s<t} F_s i_t x_t, again as a cumsum, so it's as good as if though it were parallel). I think the sLSTM layers that are the other component of the xLSTM have something else like this and presumably there's some trick also to training the Kimi "delta attention" RNN.
I'm not sure whether this is to some degree hardware or optimization dependent to some degree, but I get the impression that good custom kernels are an important part of this kind of thing.
abalashov 40 minutes ago [-]
I've been using Kimi for coding tasks for close to six months now, and haven't looked back. I'll periodically try something on Claude to make sure I'm not missing anything, but I've been very happy.
I just do the OpenRouter thing. My use of LLMs is narrow enough that cost is a negligible consideration either way.
comandillos 32 minutes ago [-]
Oh, so they’ve decided to suspend subscriptions because they know they might not be able to guarantee a minimum quality of service to their customers?
Alifatisk 1 minutes ago [-]
Yes
himata4113 51 minutes ago [-]
I wonder if anthropic and openai will remain relevant simply due to the fact that they're the only ones that are able to handle this much demand for the forseeable future? My bet would be that companies would probably not be too happy with employee time being wasted on outages and other related issues when it already costs so much.
skeledrew 13 minutes ago [-]
Anthropic also had a demand problem just a couple months ago, which led them to this on- vs off-peak usage thing that was driving people crazy, along with frequent outages. Then they made a deal with xAI and all has been mostly well since. Moonshot is making deals right now as well for compute.
georgeburdell 30 minutes ago [-]
Yes, this is the story I see playing out at my employer. Software is cheap, but the know-how to host and be accountable for uptime still keeps the actual deployments consolidated among a few groups. But yes, there are a bunch of productivity helpers that people have vibe coded that live on their laptops.
vblanco 1 hours ago [-]
Ive been pleasantly surprised with the quality of this model. Its really good at code review and PR review. But right now due to them being so overloaded + this being a big model, its SO slow. It takes forever to do a relatively simple code review.
aresant 1 hours ago [-]
Curious if this kimi moment is net growth (jervons paradox - abundance = more consumption) or just people shifting to cheaper model
Is there a good source for total token consumption between the various labs / openrouter / etc?
skeledrew 19 minutes ago [-]
If this was about shifting to cheaper models I think people would be going to DeepSeek for the v4 Flash. Kimi's latest is more expensive than most other Chinese models so I'd say this is more a wagonist "hey this new model really rocks; y'all gotta check it out!". Question is how much it'll stand up under the scrutiny. And those US labs are probably getting pretty nervous.
tosh 1 hours ago [-]
another reason why it’s great to have open weights: inference providers and your own hardware can ensure that you get the tokens you need even if a lab itself hits roadblocks (and there have been many in quality and availability)
charcircuit 1 hours ago [-]
The rate limits on the $20 plan go so fast that it's easy to use my weekly budget in only a couple days. I can't imagine also trying to use K3 on the plan.
Back in January with K2.5 it felt like I could use it a lot more.
lardosaurusrex 51 minutes ago [-]
unfortunately that probably won't be solved anytime soon because the entire point is to make you blow through it all asap and buy more.
there's always a push to make the models smarter but everyone was (and still is) convinced that if we just throw more hardware at the problem and then just tell the ai to re-read its entire ''thought'' process several dozen times it'll somehow all work out and... yeah.
and it kinda works but they'll guzzle your funds very quickly now.
unless serious strides are made in efficency this problem's only gonna get worse and worse.
Rendered at 19:06:06 GMT+0000 (Coordinated Universal Time) with Vercel.
It thought for about 12 minutes and then told me I had exhausted my daily quota. (The next day Fable did the same task in 3m)
If you want to buy a plan for K3 do NOT buy the 20$ one.
I suspect that the reason it has so many parameters is the same reason that compute optimal xLSTMs have some many parameters, and the success of this model makes me a bit unhappy that we haven't gotten an xLSTM-style model of huge size developed in Europe.
Obviously these guys are very pragmatic, they're probably not committed to anything other than what works on their internal evaluations, so they still have ordinary attention layers in the model and so on, and one can't be guaranteed that the people who come up with a good model then do the engineering in an ideal way, but I still think the success of Kimi shows that it could have been if we had enough big supercomputers for LLM training and made them available to the right people-- because this is basically Hochreiter's thing. It's RNNs, or well, mostly RNNs.
mLSTM layers are completely parallel (the update rule for the cell state is C_t = f_t C_{t-1} + i_t x_t where f_t and i_t are gates that can be computed from x_t alone, x_t is the input at time t and this means that you can compute F_s = \prod_{s<t} f_s as fast as a cumsum and an exponential and then compute C_t = \sum_{s<t} F_s i_t x_t, again as a cumsum, so it's as good as if though it were parallel). I think the sLSTM layers that are the other component of the xLSTM have something else like this and presumably there's some trick also to training the Kimi "delta attention" RNN.
I'm not sure whether this is to some degree hardware or optimization dependent to some degree, but I get the impression that good custom kernels are an important part of this kind of thing.
I just do the OpenRouter thing. My use of LLMs is narrow enough that cost is a negligible consideration either way.
Is there a good source for total token consumption between the various labs / openrouter / etc?
Back in January with K2.5 it felt like I could use it a lot more.
there's always a push to make the models smarter but everyone was (and still is) convinced that if we just throw more hardware at the problem and then just tell the ai to re-read its entire ''thought'' process several dozen times it'll somehow all work out and... yeah.
and it kinda works but they'll guzzle your funds very quickly now.
unless serious strides are made in efficency this problem's only gonna get worse and worse.