The Rise of 4-Bit and 6-Bit Floating-Point, WTW (What the What)?
What happened
This is a technical analysis article discussing the rise of low-precision floating-point formats. Citing the paper 'The Rise of Low Precision Floating Point' by Guy Lemieux, a professor at the University of British Columbia, and Michael Overton, a professor at New York University, the author notes that although 32-bit and 64-bit floating-point formats have dominated numerical computing for decades, the rise of deep learning has encouraged progressively smaller alternatives — from 32 bits to 16 bits, then 8 bits, and now even 6-bit and 4-bit floating-point formats. The article explains that NVIDIA's TensorFloat-32 (TF32) format, introduced in 2020, has only 19 effective bits (1.8.10); in NVIDIA's NVFP4 implementation, each block of 16 four-bit values has an associated 8-bit scale factor, while an additional 32-bit scale factor applies across the tensor. In 2023, the Open Compute Project (OCP), with contributions from companies including AMD, Arm, Google, Intel, Meta, and NVIDIA, introduced specifications for 8-bit, 6-bit, and 4-bit formats; the IEEE P3109 Working Group, of which Guy and Michael are both members, is developing a more comprehensive framework for low-precision arithmetic, with formal standardization anticipated in 2027. The author notes that 16-bit and 8-bit formats are already well established in AI applications, while the extent to which 6-bit and 4-bit floating-point will become commonplace remains an open question.
Impact and signal
Affected components and suppliers
Source and time
Source: EE Journal · 11/18/2026 · View original source
Real-time pricing, supply signals and AI analysis for semiconductor traders