Running local LLMs on two P104-100 GPUs
Running local LLMs on two P104-100 GPUs EXPERIMENT NOTEBOOK · 2026-10-18 Running local LLMs on two P104-100 GPUs From environment setup to Qwen and Strata performance and coding tests 0. Summary Two 8GB P104-100 GPUs ran local LLMs successfully. Qwen Q2 led in generation speed; Strata Coder passed the most tasks among the local configurations in this coding evaluation. Median coding generation was 31.42 tok/s for Q2 and 11.45 tok/s for Coder. Q2 passed two of six custom coding tasks, while Coder passed four. Test Main result Interpretation Build and startup Two P104 GPUs; CUDA 12.4 / sm_61 build succeeded; Qwen and Strata returned responses Both engines ran on this hardware. Synthetic prompt / decode Q2 layer: 543.77 / 31.87 tok/s; Q4 layer: 9.89 / 20.68 tok/s Q2 was faster. Q4 included CPU offloading, so quantization was not the only variable. Context Q2 32K: 27.47 tok/s; Q4 4K: 20.54 tok/s (three decode runs each) Synthetic decoding was confirmed, not accura...
