// TOM'S HARDWARE US — HARDWARE & GADGET
Lawyer builds private AI server with eight used Nvidia V100 GPUs
The builder estimates 50% to 60% of their AI work can now skip frontier models, running on two office circuits.
When you purchase through links on our site, we may earn an affiliate commission. Here’s how it works.
A lawyer posted on Reddit about their private AI server built from eight used 2018-era Nvidia Tesla V100 data center GPUs so case files never have to leave the office. With two 4-card SXM2 NVLink boards, four PEX switches, and SlimSAS cables everywhere — plus one burnt power connector — the builder now runs the open DeepSeek V4.1 Flash model with every expert in GPU memory, at up to 215 tokens per second. The four-GPU boards cost about $2,400 apiece when the builder bought them.
The eight Tesla V100-SXM2-32GB cards, each with 32GB of HBM2 at 900 GB/s, plug into SXM2, Nvidia’s mezzanine socket for data center GPUs, rather than PCIe slots, and are split across two baseboards. Inside each board, all four cards are NVLinked — NVLink is Nvidia’s direct GPU-to-GPU link — for 128GB per board, but the two boards only communicate over PCIe. Each board connects to the host through two PEX (PLX) switch cards over SlimSAS cables, four switches in all, with each switch handling two GPUs at PCIe 3.0 x8 speeds over its own x16 uplink.
The given prices are already in the past, though: used V100 SXM2 32GB cards are now listed on eBay for about $900, more than double the roughly $400 apiece from the post, and listings suggest prices are still climbing. The system also already has some other necessary components: a Gigabyte Threadripper Pro motherboard with a 12-core 3945WX CPU to match, 8x 32GB DDR4 RDIMM, one NUMA node, and two PSUs, 2,800 W in total, are on separate regular office circuits. This is a real-world case for used data center hardware as a cheap route to getting lots of fast VRAM together in one place.
The server is currently in daily use for first-pass work on case files: it reads a folder of records, pulls out facts, dates, and numbers, summarizes them, and takes a first swing at routine drafting. “Nothing goes out the door without a real review,” which is a good human-in-the-loop policy in general. It also helps ensure privacy for cases where client files can’t be sent to a cloud or frontier model. RAG databases and tool calls are wired in; the operator can drive the system through a Discord-style interface over SSH, in this case from a Windows PC. For now, it handles only one request at a time.
DeepSeek V4.1 Flash is compressed to mostly 2 bits here, and its weights still total 248.8GB. The model’s “experts,” the parts that do most of the work for each word, live entirely in GPU memory, while about 110GB of lookup tables, called Engram tables, stay in the system’s regular RAM. Fetching from RAM costs about 0.12 milliseconds per step, which the builder says barely affects the speed numbers.
The first half of the model’s layers (0–19) runs on the first SXM2 board and the second half (20–39) on the second. Each board handles its heavy traffic over its own NVLink, so only one hand-off per step has to cross the slower PCIe link between the boards. In an earlier post, the builder found that splitting every layer across all eight cards made speeds “fall off a cliff” because of that PCIe hop.
Performance is surprisingly good for a model this size. Code comes out fastest, at up to 215 tokens per second, because a small draft model built into DeepSeek guesses five tokens ahead, and code is easy to guess. Legal writing, which is what the server is for, runs at up to 118 tokens per second.
Get Tom's Hardware's best news and in-depth reviews, straight to your inbox.