One thing is not obvious to me is how ConvRot can be applicable beyond diffusion models. Especially for LLM decoding, as each ConvRot would be more expensive for a given decoding vector, and it is required now, so you cannot easily get the benefit for prefill only, while maintaining the same decoding performance.
augment_me 1 hours ago [-]
Alternative title: "Our invention makes INT8 better on older consumer cards without FP8 support"
peter_d_sherman 2 hours ago [-]
>"FP8, which has been used as the standard until now, was only supported by the RTX 40/50 series.
In contrast,
INT8 ConvRot
is attracting attention because it can benefit from hardware support even on the RTX 20/30 series."
Just read that before finding this. Makes me feel like we're all on the same brain-wavelength. Im actually in the middle of pulling and setting up H3 locally (int8 convrot)
Rendered at 18:08:41 GMT+0000 (Coordinated Universal Time) with Vercel.
INT8 ConvRot
is attracting attention because it can benefit from hardware support even on the RTX 20/30 series."
Related:
https://blog.comfy.org/p/minimax-h3-day-0-support-in-comfyui