// HACKER NEWS — CYBERSECURITY
Getting 50 GB/S Back from the Apple Neural Engine
An RTL performance erratum in the Apple M3 Neural Engine throttles DRAM weight streaming throughput down to 17–19 GB/s from the nominal 45–60 GB/s, whenever the total weight size is an integer multiple of 1 MiB, which currently affects 7 of ANEMLL’s 15 models. Avoiding the problematic path in the kernel DMA engine's speculative prefetch ring increased Llama 3.2 1B token throughput from 10.0 to 24.3 tokens/s (DRAM usage from 24.7 to 60.0 GB/s), and Qwen3-8B from 1.36 to 2.97 tokens/s (DRAM usage from 22.4 to 48.7 GB/s).
I was profiling the neural engine's DRAM weight streaming throughput (GB/s) for single token decode:
At \(N=4096\), I noticed that \(D=1536\) ran nearly 3× faster than \(D=2048\), the default used in Llama 3.2.
STATIC (pure KernelDMA) median µs per replica, N=4096:
Sweeping the D around the neighborhood of D = 2048:
At D=2048, throughput was 16.93 GB/s. At D=2016, throughput was 44.5 GB/s, meaning
A 27.57 GB/s drop, from 44.5 down to 16.93 GB/s. Note that the sweep data was collected on an M3 Air, repeated across 40 runs, under the same thermal/load conditions in a single run.
I also ensured that the ANE register file's DMA size and address were the only variables being changed:
That was a good idea, because I'm seeing a resonance at D = 2048. Never thought I’d do an FFT of throughput (GB/s) against tensor dimension (D), but here it is:
Apparently the memory controller's throughput has a dominant harmonic with wavelength 2048 in tensor-dimension space. And sadly, it's a dip :(.
All multiples of D = 2048 are similarly capped at a fixed bandwidth floor of 17-19 GB/s: