Krisp: One FP16 Flag Roughly Doubles Voice Isolation Calls per Server

What Krisp changed

The change turns on FP16, a number format that stores values in 16 bits instead of the usual 32. According to Krisp, it is not a different algorithm and not a smaller model. It is the same network with the same layers, storing and multiplying half as many bits.

The company points to two effects. First, weights and intermediate results are half the size, so each frame moves less data between RAM and the CPU. Krisp says these models are often limited by data movement rather than raw multiplication speed. Second, a CPU register of a given width holds twice as many FP16 numbers as float32 numbers. A 512-bit register on an Intel chip holds 16 float32 values or 32 FP16 values.

On a 16-vCPU Graviton3, single-stream p95 latency for Krisp VI Balanced dropped from 3.00 ms to 1.48 ms per 20 ms audio frame. In one 20 ms window, that means 13 frames fit with FP16 on, against 6 with it off.

How the capacity was measured

Krisp measured how many calls a server can hold at once rather than the speed of a single call, since that figure drives server cost. Streams were added one load level at a time until one of three limits broke, and the last count that held all three was reported.

• CPU utilization (p95) at or below 85%
• Real-time factor (p95) at or below 0.85
• Dropped frames at or below 1%

The company tested three models: Krisp VI Balanced and Krisp VI Default (the full models) and Krisp VI Lite. Each ran with FP16 off and on across AWS Graviton3, Intel Sapphire Rapids and Intel Ice Lake instances. The runs used Krisp Python SDK 1.12.0 in August 2026. The benchmark had a safety cap of 508 streams, so a result marked 508+ means the real ceiling is higher.

The results by CPU

For Krisp VI Balanced, FP16 raised capacity on both CPUs with native FP16 hardware. On Graviton3, the 4-vCPU instance went from 23 to 43 streams, the 16-vCPU instance from 76 to 137, and the 32-vCPU instance from 86 to 171. On Sapphire Rapids, the figures were 11 to 21, 44 to 77, and 79 to 144 for the same three sizes.

Krisp VI Default behaved almost identically, reaching 138 streams on a 16-vCPU Graviton3 and 146 on a 32-vCPU Sapphire Rapids machine. The lightweight Krisp VI Lite gained less in percentage terms but handled more streams overall. On a 16-vCPU Graviton3 it rose from 209 to 358 (+71%), and on a 32-vCPU Graviton3 it hit the 508-stream cap with FP16 on, up from 287. On Sapphire Rapids, the gains across all three models ranged from +61% to +91%.

Intel Ice Lake showed no change. It has no FP16 hardware, so Balanced stayed at 7 streams on 4 vCPUs and 38 on 16 vCPUs, and the other models were equally flat.

What it means for operators

The flag does not require changes to the model or the audio pipeline. When it is enabled, the SDK checks whether the CPU supports FP16 arithmetic. If it does, weights are loaded as float32, converted to FP16, and inference runs in FP16. If not, everything stays on the float32 path. Krisp states that FP16 does not degrade quality.

For teams running Krisp Voice Isolation at scale on Graviton3 or Sapphire Rapids, the practical effect, according to Krisp's benchmark, is roughly twice the live calls per server with the same model. Operators on older chips such as Ice Lake should expect no benefit. Krisp advises checking the documentation for enableLowPrecisionExecution in the latest SDK release for your language. At the time of writing, those were Python 1.13.3, Node.js 1.8.2 and C/C++ 9.22.0.

The figures come from Krisp's own tests on specific cloud instances, so results on other hardware or traffic patterns may differ.