Running local LLMs on two P104-100 GPUs
Running local LLMs on two P104-100 GPUs
From environment setup to Qwen and Strata performance and coding tests
0. Summary
Two 8GB P104-100 GPUs ran local LLMs successfully. Qwen Q2 led in generation speed; Strata Coder passed the most tasks among the local configurations in this coding evaluation. Median coding generation was 31.42 tok/s for Q2 and 11.45 tok/s for Coder. Q2 passed two of six custom coding tasks, while Coder passed four.
| Test | Main result | Interpretation |
|---|---|---|
| Build and startup | Two P104 GPUs; CUDA 12.4 / sm_61 build succeeded; Qwen and Strata returned responses | Both engines ran on this hardware. |
| Synthetic prompt / decode | Q2 layer: 543.77 / 31.87 tok/s; Q4 layer: 9.89 / 20.68 tok/s | Q2 was faster. Q4 included CPU offloading, so quantization was not the only variable. |
| Context | Q2 32K: 27.47 tok/s; Q4 4K: 20.54 tok/s (three decode runs each) | Synthetic decoding was confirmed, not accuracy on long natural-language documents. |
| Coding API | Median decode: Q2 31.42, Q4 19.22, IQ2_XS 14.40, Coder 11.45 tok/s | Q2 generated fastest on this request set. |
| Coding correctness | Q2 2/6; Q4 3/6; IQ2_XS 2/6; Coder 4/6; GPT baseline 6/6 | Coder led the local configurations on six custom tasks, not a general leaderboard. |
| Short Strata requests | Median: IQ2_XS 12.1, Coder IQ1_M 9.3 tok/s (three runs each) | Large-model responses were verified; long-input performance was not. |
| Storage | Median sequential read 555.45 / write 93.34 MiB/s; five profiles, three runs each | Measures the VM storage path, not isolated device performance or a proven inference bottleneck. |
| Remote API authentication | Unauthenticated lookup: 401; authenticated chat: success | LAN authentication was verified. Internet exposure and automatic startup were not configured. |
Choosing a configuration: Q2 layer was the candidate for responsive short conversations. Coder IQ1_M passed more coding specifications in this set, but its six requests took 624.83 seconds overall. Generation speed and answer correctness need separate judgments.
The sections below present final settings, commands and archived output from October 18, 2026. Metrics were aggregated from saved responses; remote experiments were not rerun during editing. Replace USER and SERVER_IP with values for your environment.
1. Test environment
The test system used two 8GB P104-100 GPUs. The VM configuration allocates 48GiB, whereas the guest reports 46GiB of RAM. Model weights, KV caches and compute buffers were distributed across the devices.
| Item | Recorded value |
|---|---|
| OS / kernel | Ubuntu 26.04 LTS / 7.0.0-38-generic |
| Virtualization / CPU | KVM / AMD EPYC-Rome, 8 vCPU |
| RAM / swap | 46 GiB / 8 GiB |
| GPU | 2 × NVIDIA P104-100, 8192 MiB each, compute capability 6.1 |
| PCIe | Gen1 ×1 observed; reported maximum Gen1 ×4 |
| Driver / toolkit | 580.178.04 / CUDA 12.4.131 |
| Compiler / CMake | GCC 13.4.0 / CMake 4.2.3 |
| llama.cpp fork | dc5c644c45f35167edfa041110f8c9bd75404b34 |
| Strata | 0.1.45 / 477ac13cbe8e582c20484871ce459d20d82df09d |
2. Preparing the driver and CUDA environment
Initially, nouveau was loaded and neither nvidia-smi nor nvcc was available. The administrator installed NVIDIA 580 and the CUDA 12.4 development tools from the Ubuntu packages available at the time, then rebooted. The following is the documented installation procedure. A complete apt transcript is not reproduced here, so the commands alone are not presented as proof of success.
sudo apt update
sudo apt install --no-install-recommends \
linux-modules-nvidia-580-$(uname -r) \
nvidia-driver-580
sudo apt install --no-install-recommends \
build-essential gcc-13 g++-13 cmake ninja-build pkg-config \
libssl-dev libcurl4-openssl-dev nvidia-cuda-toolkit python3-venv
Device recognition and the Toolkit were checked separately after rebooting. CUDA 13.0 in the nvidia-smi header is the driver's compatibility level; nvcc identifies the installed compiler as 12.4.131. The recorded verification commands and archived output follow.
nvidia-smi
nvidia-smi --query-gpu=name,memory.total,driver_version,compute_cap --format=csv
nvidia-smi --query-gpu=name,pcie.link.gen.current,pcie.link.gen.max,pcie.link.width.current,pcie.link.width.max --format=csv
nvidia-smi topo -m
nvcc --version
gcc-13 --version
name, memory.total [MiB], driver_version, compute_cap, pcie.link.gen.current, pcie.link.gen.max, pcie.link.width.current, pcie.link.width.max
NVIDIA P104-100, 8192 MiB, 580.178.04, 6.1, 1, 1, 1, 4
NVIDIA P104-100, 8192 MiB, 580.178.04, 6.1, 1, 1, 1, 4
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2024 NVIDIA Corporation
Built on Thu_Mar_28_02:18:24_PDT_2024
Cuda compilation tools, release 12.4, V12.4.131
Build cuda_12.4.r12.4/compiler.34097967_0
gcc-13 (Ubuntu 13.4.0-10ubuntu1) 13.4.0
3. Preparing model storage
Storage was addressed before model downloads. The VM already had a 300GiB disk, but its root LV was only 100GiB. The issue was unallocated space inside the volume group, not a missing disk expansion. These are the administrator's recorded steps, not commands rerun for this article. +100%FREE consumes all remaining space in the VG and should not be copied blindly into another environment.
sudo pvs
sudo vgs
sudo lvs
sudo lvextend -r -l +100%FREE /dev/ubuntu-vg/ubuntu-lv
df -hT /
lsblk
The follow-up record reports an approximately 298GiB LV and 293G in df for ext4. Free space changed as downloads progressed; a single free-space snapshot was not used to infer final installation size.
4. Building for sm_61 and pinning model files
build.sh is provided by the qr0n/p100-llama-cpp fork, not an upstream ggml-org/llama.cpp build script or a file written for this experiment. These preparation commands obtain the recorded commit in a fresh directory; they were not executed during editing.
git clone https://github.com/qr0n/p100-llama-cpp.git ~/p100-llama-cpp
cd ~/p100-llama-cpp
git checkout --detach dc5c644c45f35167edfa041110f8c9bd75404b34
Then run the repository’s build.sh with the following settings. The build used CUDA_ARCH=61 for the P104. The target CUDA architecture was sm_61. The archived build.sh exit code is 0. The follow-up documentation reports both CUDA0 and CUDA1 as P104 devices and llama-server as 0.3.0-dev, build 23, commit dc5c644.
cd ~/p100-llama-cpp
CUDA_ARCH=61 JOBS=8 ./build.sh
./llama.cpp/build/bin/llama-server --list-devices
0
The UI assets used the latest version available at installation time.
The model revision and hashes below were pinned, and the records report completed size and SHA-256 checks after download. The large GGUF files are not in this ZIP; their hashes were not freshly verified for this article.
For reproduction on P104, check the actual sm_61 CUDA target rather than the repository name. P100 in the command is only the archived working-directory name. Build exit code 0 was checked alongside the subsequent CUDA device enumeration.
bartowski/Qwen_Qwen3.6-35B-A3B-GGUF · revision 5c2410d71524f4f72b023ce8daf7a80528226d5f
| File | Bytes | SHA-256 |
|---|---|---|
| Qwen_Qwen3.6-35B-A3B-Q2_K_L.gguf | 14,003,418,752 | bcecb058c4b349b2df86353ae16538b0787a5704cafe412a79600696ea9234ef |
| Qwen_Qwen3.6-35B-A3B-Q4_K_M.gguf | 22,285,080,192 | b46fedd33e0bfb0cae308aa3c158d0a4b2c4a1d2185a1ed6f093cdaf39064772 |
5. Measuring prompt processing separately from decoding
Prompt processing (pp512) and decoding (tg128) were measured separately, with three runs per configuration after downloads finished. pp measures processing of 512 input tokens; tg measures generation of 128 tokens.
The Q2 layer benchmark command follows. The -ts 1/1 option used by this build is retained.
/home/USER/p100-llama-cpp/llama.cpp/build/bin/llama-bench -m /home/USER/models/qwen3.6-35b-a3b/Qwen_Qwen3.6-35B-A3B-Q2_K_L.gguf -ngl 999 -sm layer -ts 1/1 -t 8 -b 512 -ub 128 -fa on -ctk f16 -ctv f16 -r 3 -o json -v -p 512 -n 128
| Configuration | Mean pp512 tok/s | Mean tg128 tok/s |
|---|---|---|
| Q2 layer / GPU | 543.77 | 31.87 |
| Q2 tensor / GPU | 204.52 | 31.81 |
| Q4 layer / CPU offload | 9.89 | 20.68 |
Each value is a three-run mean. Inputs are pseudorandom token IDs, not natural-language documents. Q2 is GPU-resident while Q4 includes CPU offloading, so this does not isolate quantization. Q4 tensor plus auto-fit was unsupported and is excluded from the comparison.
6. What the 32K result does—and does not—show
The length includes the prefix plus 128 generated tokens. Prefix processing is outside the decode timer. Capacity exploration used one run per point; only the final Q2 32K and Q4 4K configurations were reconfirmed with three decode repetitions.
| Total length / condition | Q2 decode tok/s | Q4 decode tok/s |
|---|---|---|
| 2K | 31.67 | 20.09 |
| 4K | 31.29 | 20.28 |
| 8K | 30.65 | timeout (900 s) |
| 16K | 29.56 | — |
| 32K exploratory | 27.49 | — |
| 32K Q2 / 4K Q4 confirmed | 27.47 (3 runs) | 20.54 (3 runs) |
The complete Q2 32K confirmation command took approximately 14 minutes 44 seconds. That includes loading, prefix processing and state save/restore; it is not HTTP time to first token. The prefix was processed first, then later decode repetitions restored saved state. This does not establish three fresh, accurate readings of a 32K natural-language document.
Q2 at 64K and Q4 at 8K hit a 15-minute timeout, not a confirmed OOM. Strata's 32768 setting was verified at startup, but long-input performance was not tested.
7. Checking the storage path with fio
The VM's storage path was measured separately before interpreting Strata. The runs used fio 3.41, libaio, a 4GiB test file, direct=1, 30 seconds plus a 3-second ramp, and three repetitions per profile. Below is the archived command for the first sequential-read run. The suite also contains write tests, so actual execution requires a dedicated test file and sufficient free space.
/usr/bin/fio --name=sequential-read-r1 --filename=/home/USER/pascal-llm-project/artifacts/2026-10-18-disk-bench/fio-test-sfnz_nfr/test.bin --size=4G --rw=read --bs=1M --iodepth=1 --numjobs=1 --ioengine=libaio --direct=1 --invalidate=1 --refill_buffers=1 --buffer_compress_percentage=0 --dedupe_percentage=0 --group_reporting=1 --output-format=json --time_based=1 --runtime=30 --ramp_time=3
| Profile | Median MiB/s | Min–max MiB/s | Median IOPS |
|---|---|---|---|
| sequential-write | 93.34 | 87.77–94.28 | 93.34 |
| sequential-read | 555.45 | 530.22–560.48 | 555.45 |
| random-read-4k-qd1 | 18.35 | 17.31–18.53 | 4698.21 |
| random-read-64k-qd1 | 108.26 | 96.48–112.94 | 1732.21 |
| random-read-64k-qd32 | 443.59 | 381.63–512.40 | 7096.39 |
These measurements traverse the VM's ext4 path. Host caching and I/O from other VMs were not isolated, so this is not bare-device performance. Nor does fio directly measure Strata decoding speed. The records report completion and temporary-file cleanup at 14:45:29 KST.
8. Building Strata, first responses and the Coder follow-up
Here, setup.sh is the installer entry point supplied by the Strata project, not a custom experiment script. The following prepares the recorded source in a fresh directory and was not executed during editing. Obtain the complete repository, not just this one file.
git clone https://github.com/Niko1221/Strata.git /home/USER/strata
cd /home/USER/strata
git checkout --detach 477ac13cbe8e582c20484871ce459d20d82df09d
Installation downloads dependencies and models, builds the engine and generates configuration; check available storage and inspect the script first. After downloading the split IQ2_XS GGUF files, Strata was built through its CUDA 12 / sm_61 path. The installer also downloaded MTP weights and created the model pack. The following is the core execution section of the archived run.sh.
cd /home/USER/strata
export CC=/usr/bin/gcc-13 CXX=/usr/bin/g++-13 CUDAHOSTCXX=/usr/bin/g++-13 STRATA_NVCC=/usr/bin/nvcc
./setup.sh --yes --family qwen --model IQ2_XS --context 32768 --vision no --gpus 0,1 --cuda 12 --build --no-start --no-browser --host 127.0.0.1 --port 8082 --data-dir /home/USER/Strata-data
The installation exit code was 0. The first health response follows. api_key=false belongs to the initial localhost-only stage and must not be confused with the later authenticated LAN configuration.
{
"status": "ok",
"max_context": 32768,
"model": "qwen3.8-flash-next-iq2_xs",
"images": false,
"api_key": false,
"image_generation": false,
"loaded": true,
"service": "strata"
}
Initial smoke requests returned a greeting/arithmetic answer and English Python tips. The 128-token request took 13.83 seconds overall, with 11.079 seconds in engine-reported generation. Subsequent short IQ2_XS requests produced 10.6 / 12.1 / 12.8 tok/s, with a median of 12.1.
The Coder IQ1_M follow-up downloaded a first file of approximately 29.61GB and reused the second GGUF through a hard link. It used Strata 0.1.45, GPU 0/1 layer splitting, context 32768, images disabled and MTP. Three short 128-token runs produced 8.5 / 9.7 / 9.3 tok/s, with a median of 9.3. The first Coder request followed startup, so warm-up conditions were not identical to IQ2_XS.
The engine's approximately 0.2GB/s host-to-device probe and PLE I/O warning are diagnostic clues. They do not isolate disk or PCIe as the sole cause of all latency.
The experiment wrapper
The three installation lines above were expanded from a custom run.sh; that wrapper is not a Strata project file. Its remaining code records the exit status and completion time. The full archived wrapper follows, with the account name replaced by USER. Create the recording directory before running it.
#!/bin/bash
set -o pipefail
cd /home/USER/strata
export CC=/usr/bin/gcc-13 CXX=/usr/bin/g++-13 CUDAHOSTCXX=/usr/bin/g++-13 STRATA_NVCC=/usr/bin/nvcc
./setup.sh --yes --family qwen --model IQ2_XS --context 32768 --vision no --gpus 0,1 --cuda 12 --build --no-start --no-browser --host 127.0.0.1 --port 8082 --data-dir /home/USER/Strata-data
result=$?
echo "$result" > /home/USER/pascal-llm-project/artifacts/2026-10-18-strata-build/exit-code.txt
date -Is > /home/USER/pascal-llm-project/artifacts/2026-10-18-strata-build/ended.txt
exit "$result"
9. Speed on actual coding API requests
Synthetic benchmarks and actual coding requests need separate interpretations. This table aggregates six coding responses per model. Inputs were approximately 182–245 tokens, but answer lengths varied. Total request time includes input processing and generation; it is not the same metric as decoding throughput.
| Model | Prompt tok/s range | Median decode tok/s | Decode range tok/s | Six requests, seconds |
|---|---|---|---|---|
| Qwen Q2_K_L | 100.79–134.27 | 31.42 | 31.38–31.50 | 74.02 |
| Qwen Q4_K_M | 7.30–10.47 | 19.22 | 18.57–20.26 | 262.29 |
| Strata IQ2_XS | 3.00–3.70 | 14.40 | 13.00–16.00 | 593.84 |
| Strata Coder IQ1_M | 2.60–3.30 | 11.45 | 10.80–12.10 | 624.83 |
Generation speed is the median of six responses per model; total time is the sum of the six complete request durations. Values are shown to two decimal places.
Local requests used temperature=0, max_tokens=2048, stream=false and concurrency 1. Qwen used reasoning off; Strata used reasoning_effort none and MTP spec 4. Q2/Q4 reused some shared prefixes of 50–70 tokens, while Strata reported cached_tokens=0. Part of the Q4 collection overlapped with downloads. Every archived result has finish_reason=stop. TTFT was not measured because collection was non-streaming.
Archived output example: the first Q4 interval-merging request.
{
"start": "2026-10-18T06:05:04.811298+00:00",
"prompt_sha256": "996ea492b141479e03d441d3a3e98a6c38a616dfca3692a26f501a684d6e639c",
"returncode": 0,
"finish_reason": "stop",
"usage": {
"completion_tokens": 335,
"prompt_tokens": 182,
"total_tokens": 517,
"prompt_tokens_details": {
"cached_tokens": 0
}
},
"timings": {
"cache_n": 0,
"prompt_n": 182,
"prompt_ms": 24940.415,
"prompt_per_token_ms": 137.03524725274727,
"prompt_per_second": 7.297392605536035,
"predicted_n": 335,
"predicted_ms": 16482.051,
"predicted_per_token_ms": 49.347458083832336,
"predicted_per_second": 20.2644682994853
},
"wall_seconds": 41.437647333,
"end": "2026-10-18T06:05:46.249187+00:00"
}
Recomputed from six saved responses per model. GPT speed was not measured.
10. Final coding results
pascal-code-v1 is a custom six-task evaluation written in Korean. Each model's first answer was graded without code repair or regeneration. The score counts fully passing tasks, not individual test groups. It is neither a HumanEval score nor a general coding leaderboard.
| Task | Q2 | Q4 | IQ2_XS | Coder | GPT baseline |
|---|---|---|---|---|---|
| Integer interval merging | Fail | Pass | Pass | Pass | Pass |
| TTL + LRU cache | Fail | Fail | Fail | Pass | Pass |
| Stable topological sort | Pass | Fail | Fail | Fail | Pass |
| Chunked UTF-8 JSONL | Fail | Pass | Fail | Fail | Pass |
| Concurrency limits and cancellation | Fail | Fail | Fail | Pass | Pass |
| Idempotent ledger | Pass | Pass | Pass | Pass | Pass |
| Final | 2/6 | 3/6 | 2/6 | 4/6 | 6/6 |
These final judgments apply common edge cases to all answers, including empty graphs, invalid edges and U+2028 inside JSON strings. Coder passed 4/6 tasks, the most among the local configurations, and was the only local model to pass both the TTL cache and asynchronous concurrency tasks.
Coder still failed early-return validation in topological sorting and U+2028 handling through splitlines(). One first answer per task and a six-task suite cannot establish general model superiority. The original manual review was an internal second pass by the responsible assistant, not an independent external review.
Answers were not repaired after generation. The table shows only final task-level outcomes after automatic checks and specification review.
What was tested
Each request asked for code using only the Python 3.14 standard library, without mutating inputs or relying on external files, networking or the current time. A task passed only when its specification and common edge-case checks were satisfied.
| Task | Specification and judgment criteria |
|---|---|
| Intervals | Half-open and adjacent ranges; ignore empty ranges; reject bool, noninteger and reversed endpoints |
| TTL + LRU | Remove expired entries first; get updates recency without extending TTL; allow None values |
| Topological order | Lexicographic priority; deduplicate edges; reject invalid edges even with empty nodes; detect cycles |
| UTF-8 JSONL | Split characters, LF/CRLF and final lines; reject malformed UTF-8 or JSON |
| Async execution | Bound concurrency and preserve result order; cancel and await remaining tasks on failure or cancellation |
| Idempotent ledger | Ignore identical repeated events; reject conflicting IDs, invalid types and negative balances |
The name pascal-code-v1 is an internal identifier for this experiment. These external references explain evaluation methods and API semantics; they are not sources for these six tasks or evidence for the measured results.
- Original HumanEval paper: Evaluating Large Language Models Trained on Code · Authors’ evaluation code: a separate public evaluation, not the score reported here.
- Python 3.14: Task cancellation: reference for cancellation behavior.
- JSON Lines format: reference for UTF-8 and line-delimited JSON. Additional conditions such as allowing whitespace-only lines follow this experiment’s own specification.
Reevaluate model-generated code only in an isolated environment without access to networking, secrets or personal files.
Six custom tasks, first answers only. Harnesses and execution conditions differ.
11. The GPT baseline and limits of the comparison
GPT-6.1-sol default is the model identifier recorded in the original experiment. Its client was documented as MacBookPro17,1 / Apple M1 / 8GiB / macOS 26.5.1 with Codex CLI 0.163.0. The Mac submitted cloud requests and ran grading; it did not run GPT inference locally. That machine is also distinct from the machine used to edit this article.
The prompts requested the Python 3.14 standard library, but the common grading interpreter was Python 3.9.6. This mismatch is disclosed. The records attribute the score corrections to specification violations rather than syntax-version failures, but a normalized follow-up should also run the requested interpreter version.
GPT used Codex's default harness, system instructions, budget and caching; the local models used simple chat API requests. The comparison answers what these workflows did on the same user tasks, not what models do under identical budgets and inference environments. GPT speed was not measured and is not plotted as zero.
12. Operational commands to keep for reproduction
These Q2 32K and Q4 4K commands come from the final operations guide. Run one engine at a time to avoid competing GPU workloads. The context limit includes both input and output. Change example log names before repeating runs so the original evidence is not overwritten.
Q2 uses fixed GPU placement with fit disabled.
ssh USER@SERVER_IP
cd ~/p100-llama-cpp
mkdir -p ~/pascal-llm-project/logs
./llama.cpp/build/bin/llama-server \
-m ~/models/qwen3.6-35b-a3b/Qwen_Qwen3.6-35B-A3B-Q2_K_L.gguf \
-ngl 999 -sm layer -ts 1,1 --fit off \
-c 32768 -np 1 -t 8 -b 512 -ub 128 \
-fa on -ctk f16 -ctv f16 \
--jinja --reasoning off --cache-ram 0 \
--host 127.0.0.1 --port 8081 --metrics \
2>&1 | tee ~/pascal-llm-project/logs/qwen-q2-server.log
Q4 leaves -ts unset so auto-fit can choose CPU/GPU placement. Its context limit is 4096.
cd ~/p100-llama-cpp
./llama.cpp/build/bin/llama-server \
-m ~/models/qwen3.6-35b-a3b/Qwen_Qwen3.6-35B-A3B-Q4_K_M.gguf \
-sm layer --fit on -fitt 512 -fitc 4096 \
-c 4096 -np 1 -t 8 -b 512 -ub 128 \
-fa on -ctk f16 -ctv f16 \
--jinja --reasoning off --cache-ram 0 \
--host 127.0.0.1 --port 8081 --metrics \
2>&1 | tee ~/pascal-llm-project/logs/qwen-q4-server.log
After startup, inspect /health and /v1/models and use the actual model ID. Startup checks follow. Neither command was remotely executed during this editorial pass.
curl -fsS http://127.0.0.1:8081/health
curl -fsS http://127.0.0.1:8081/v1/models
Strata was subsequently bound to the LAN with API-key authentication. Archived verification shows 401 for an unauthenticated model query and success for an authenticated chat. Actual secrets and commands that print secrets are excluded from the blog. No systemd auto-start or router port forwarding was configured. The final Coder stage replaced the model on the same port, so the earlier IQ2_XS health response is not a claim about the current model.
The memory error that blocked Q4 startup
In this build, fixing the GPU split with -ts 1,1 prevented auto-fit from adjusting placement. These two lines are excerpts from the actual startup log.
0.00.781.449 W common_fit_params: failed to fit params to free device memory: model_params::tensor_split already set by user, abort
0.02.064.775 E ggml_backend_cuda_buffer_type_alloc_buffer: allocating 10316.83 MiB on device 0: cudaMalloc failed: out of memory
The working command removes -ts and allows automatic placement with --fit on -fitt 512 -fitc 4096. Increasing the memory reserve alone did not fix the failure. Six coding requests were collected after the correction. This was a startup failure before answer generation, not a failed coding answer.
13. What the results support on this hardware
For short interactions where responsiveness matters most, Q2 layer was the fastest operating candidate: 31.42 tok/s median coding generation, but only 2/6 tasks passed. Q4 reached 3/6, at the cost of slow prompt processing under CPU offloading. Coder IQ1_M passed the most tasks among the local configurations in this set, 4/6, but its median generation speed was 11.45 tok/s and six requests took approximately 625 seconds.
Showing that Strata runs on this hardware is not the same as establishing a fast, accurate long-context coding server. The archived logs support the former; the latter was not tested.
The source also contains scenarios of 12.6–14.5 tok/s for two to three GPUs, 13.2–16.1 for two to four, and 14.2–22.0 for Gen1 x1 to x4. These are assumption-based sensitivity calculations anchored to the short IQ2_XS median of 12.1, not measurements, guarantees or confidence intervals. GPU-count and link-width effects must not be multiplied together as a prediction. Physical expansion, an x4 link, Strata long inputs, natural-language long-context accuracy and streaming TTFT remain untested.
댓글
댓글 쓰기