// HACKER NEWS — CYBERSECURITY
Nvidia Nemotron 3.5 Lightning
How to use nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4 with Transformers:
How to use nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4 with vLLM:
How to use nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4 with SGLang:
How to use nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4 with Docker Model Runner:
NVIDIA Nemotron™ is a family of open models with open weights, training data, and recipes, delivering leading efficiency and accuracy for building specialized AI agents.
NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4 is a large language model (LLM) trained by NVIDIA.
The model employs a hybrid Mixture-of-Experts architecture, utilizing interleaved Mamba-2 and MoE layers, along with select Attention layers. The Lightning 3.5 model is released alongside a number of speculative decoding methods for faster text generation. The model has 3B active parameters and 30B parameters in total.
To get quickly started on DGX Spark (GB10) you can use the following command.
Run it with vLLM — this recipe uses DSpark speculative decoding, tuned for DGX Spark. (vLLM Nightly: vllm/vllm-openai:v0.27.1)
For more details on how to deploy and use the model — see the Quick Start Guide below!