pctablet505
Work / Performance and inference

3x faster models on the Snapdragon NPU and DSP

Made deep-learning models run 3x faster on the Snapdragon NPU and DSP, and cut memory bandwidth 75% with quantization-aware training.

  • Performance and inference
  • Qualcomm
  • Sep 2024 – Jan 2025
  • Built
  • short entry
3x
faster inference on the NPU and DSP
○ reported, from my LinkedIn
75%
less memory bandwidth for image-processing models
○ reported, from my LinkedIn
CPUGPUNPUDSPRAMINT8
Illustration: work moves from the CPU to the NPU and DSP, quantized so less data crosses the memory bus.

Models tuned for the phone's NPU and DSP ran 3x faster, and quantization-aware training cut the memory bandwidth of high-fidelity image-processing models by 75%.

More in performance and inference

  • Keras on PyTorch: compiled generation 11.6x fasterJul 2026 – Oct 2026

    Traced why Keras on PyTorch ran about 9x slower than plain PyTorch, and merged the fixes upstream, one root cause each. · Google · Merged

  • GLM-5.3-Flash on one GPU2 Sep 2026

    Served a 181 GiB mixture-of-experts LLM from one 96 GiB GPU and took decoding from 10.8 to 15.0 tokens/s shipped, 16.5 at peak. · Independent · Published

  • One call from a Keras model to a phone: LiteRT exportDec 2025 – Jul 2026

    Built the LiteRT export path in Keras and KerasHub: a trained model to a phone-ready file in one call, on TensorFlow and PyTorch. · Google · Merged

  • End-to-end UI tests for XProfAug 2026 – now

    Wrote the end-to-end UI test suite for XProf, the open-source ML profiler, with 107 browser tests and a side-by-side check of two builds. · Google · Shipped