BitFace, my submitted work on 1-bit and 1.58-bit ternary Vision Transformers, was meant to make inference cheaper. The analytical model promised a clear speed-up from bit-level arithmetic.
The stopwatch disagreed. On CPU, the ternary model ran 4.5 to 6× slower than its FP32 counterpart. The gains never materialised, because the kernels underneath were never written for that arithmetic.
I could have reported the FLOP reduction and moved on. Most compression papers do. But an efficiency claim that only holds in the model, and not on the hardware, is not an efficiency claim — it's a hypothesis.
This is why I think the interesting work in efficient AI is now on the measurement side: energy and latency profiling of real inference pipelines, and quality-per-joule metrics that survive contact with deployment. Especially for video, where the per-frame budget is unforgiving.
Under review.
لا توجد تعليقات بعد. كن أول من يعلّق!