Independent benchmarks will determine whether NVIDIA Nemotron 3.5 Lightning 30B-A3B delivers a practically useful quality-throughput tradeoff for local sparse-MoE inference.

state: expiredheat: lowuncertainty: mediumknownscott: lowopen-models local-inferenceNVIDIA

What is this?

The case concerns a purported NVIDIA open model, Nemotron 3.5 Lightning 30B-A3B, and the claim that independent benchmarking is needed to establish its real quality-versus-throughput tradeoff for local sparse-MoE inference. The supplied search results are unrelated results about the newspaper The Independent; they do not verify the model’s architecture, release details, benchmark performance, licensing, hardware requirements, or practical local-inference utility. Beyond the evidence titles naming NVIDIA and a Hugging Face repository, the event is therefore not substantively grounded by these snippets.

Why it matters to Scott

Scott already holds the relevant position in Capability Audit and Hardware-aware local inference: model utility must be established through representative, hardware-specific evaluation rather than vendor claims. The model could eventually matter to his gamepc local-model stack, but the supplied evidence does not verify its architecture, requirements, performance, or even substantive release details, so it currently adds only an ungrounded benchmark candidate.

ip:concept.capability-auditdev:concept.hardware-aware-local-inferencedev:project.gamepcradar:person.nvidiaradar:concept.mixture-of-expertsradar:concept.local-inferenceradar:concept.model-evaluation

queries asked of Scott's wikis

sparse MoE local inference economics
quality-throughput benchmarks for local models
active-parameter count versus deployment cost
open-weight model evaluation framework
NVIDIA models in local inference stacks
hardware constraints for mixture-of-experts inference

Measured heat

no measured readings yet — the hourly heat pass fills this in

How the heat travelled

no chain yet — the hourly chain pass fills this in

Evidence (8) — ⭐ canonical anchor

source	object	author	score	comments
🟠 reddit ⭐	nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16 · Hugging Face LocalLLaMA	coder543	572	170
🟧 hn	Nvidia Nemotron 3.5 Lightning	beklein	121	32
🟠 reddit	nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16 LocalLLaMA	TheRealMasonMac	95	33
🟧 hn	Nvidia Nemotron 3.5 Lightning and NeMo Switchyard	droidjj	261	136
🟠 reddit	MoE task time comparison LocalLLaMA	parepeg	2	6
🟠 reddit	Tested Nemotron 3.5 Lightning locally on coding, Hermes Agent and agentic work LocalLLaMA	curiousily_	15	16
🟧 hn	Nemotron-3.5-lightning-30B-a3B Model by Nvidia for use on 1 GPU	janandonly	1	0
🟠 reddit	Nemotron 3.5 Lightning 30B-A3B: W4A16 vs IQ4_XS on the same RTX 3090: near-parity at B1, ~4.5× throughput by B16 LocalLLaMA	mitchins-au	3	3

Interpretation history

2026-08-16T07:34:32Z
The release episode has faded without a matched comparative benchmark: existing independent tests consistently indicate fast, 24GB-class deployment but uneven quality, while the latest activity is only minor engagement and repetitive commentary.
2026-08-14T07:26:36Z
The refreshed discussion adds only an unsupported preference for competing models, not a new measurement or compatibility result. Existing tests still support fast, 24GB-class local deployment with uneven quality, while the practical model-selection verdict awaits matched comparative evaluation.
2026-08-14T02:28:02Z
The same-GPU RTX 3090 test strengthens the practical deployment case by showing that a vLLM-compatible W4A16 quant can fit in 24GB and substantially improve high-batch throughput. It reinforces Nemotron’s speed advantage but does not establish end-to-end utility because quality testing is narrow and the compared quantization/runtime stacks differ.
2026-08-14T02:22:27Z
evidence attached: reddit.post.1vnu219 — An independent same-GPU benchmark compares Nemotron quantization formats and reports a substantial throughput tradeoff relevant to local serving.
2026-08-14T01:25:40Z
Refreshed discussion adds no new reproducible benchmark or compatibility result beyond the existing informal pattern of fast local inference with uneven task quality. The case remains open but has shifted from release monitoring to awaiting substantive comparative evaluation.
2026-08-13T01:23:19Z
Refreshed comments add no new measurement, reproducible benchmark, or compatibility finding beyond the established informal pattern of fast local inference with uneven quality. The case remains unresolved, but repetitive release discussion no longer merits close monitoring.
2026-08-12T16:51:56Z
The refreshed discussion adds no measured result or independent benchmark beyond the existing informal pattern of high throughput and uneven task quality. The practical local-inference tradeoff remains unresolved and repetitive commentary does not justify closer monitoring.
2026-08-12T15:42:38Z
Refreshed comments add no new measured result or independent benchmark; they repeat the established directional pattern of strong local throughput with uneven coding quality. The practical quality-throughput verdict remains unresolved and no longer warrants release-day monitoring.
2026-08-12T12:26:12Z
The added NVIDIA listing only reconfirms artifact availability and does not strengthen the emerging independent pattern of high throughput with uneven task quality. With no reproducible comparative benchmark or compatibility finding, repeated release coverage is no longer time-sensitive.
2026-08-12T12:22:59Z
evidence attached: hn.story.49270736 — NVIDIA’s first-party model listing independently corroborates the released Nemotron 3.5 Lightning local-inference episode.
2026-08-12T11:40:55Z
Refreshed comments add speculative explanations and repeated comparisons, not a new measured or reproducible result. Existing informal tests still suggest high throughput with uneven task quality, but the practical local-inference verdict remains unresolved.
2026-08-12T10:31:52Z
A second hardware-specific local run joins earlier independent Mac and llama.cpp tests in converging on high local throughput but uneven, task-dependent quality. That corroborates the broad speed-versus-capability pattern, though informal workloads and unmatched configurations still prevent a reliable practical verdict.
2026-08-12T10:22:25Z
evidence attached: reddit.post.1vma27w — A small independent local run supports the case’s quality-throughput question, showing about 65 tok/s and strong tool calling but weak coding quality.
2026-08-12T09:24:49Z
The refreshed discussion adds no reproducible benchmark or independent evidence line; it remains repetitive amplification around the same weak hands-on anecdotes. Nemotron is still a credible local test candidate, but its quality-throughput tradeoff remains unresolved.
2026-08-12T08:35:19Z
Refreshed comments and negligible engagement movement add no reproducible benchmark or independent evidence line. The only directional local tests remain methodologically weak, so the practical quality-throughput tradeoff is still unresolved.
2026-08-12T06:34:20Z
Refreshed comments and engagement add only repetitive release comparisons and anecdotes; no second independent, reproducible benchmark advances the quality-throughput case. Nemotron remains a credible local-inference test candidate, but its practical advantage is still unvalidated.
2026-08-12T05:34:00Z
The llama.cpp comparison supplies a first hardware-specific directional result—faster task completion than Qwen on simpler prompts but materially weaker quality—yet its mismatched configurations and limited methodology prevent independent validation. Refreshed discussion adds amplification rather than a second reproducible evidence line.
2026-08-12T05:22:20Z
evidence attached: reddit.post.1vm4qrs — Anecdotal llama.cpp measurements provide low-confidence independent context on Nemotron Lightning’s speed-quality tradeoff versus Qwen for local inference.
2026-08-12T03:24:14Z
A Cloudflare engineer’s coding-task comparison adds another hands-on implementation anecdote, but the refreshed discussion still lacks reproducible hardware, runtime, memory, throughput, and task-quality measurements. This is weak directional evidence rather than independent validation of the model’s practical tradeoff.
2026-08-12T01:26:03Z
A Mac user now reports roughly 100 tokens/second but poor task quality, offering a weak directional glimpse of the tradeoff rather than a reproducible benchmark. Without hardware details, memory figures, runtime configuration, or broader task evaluation, the case remains unvalidated.
2026-08-12T00:24:06Z
The refreshed comments remain speculative release comparisons and isolated anecdotes, adding no reproducible local quality, throughput, memory, or compatibility results. The model remains a credible benchmark candidate, but repeated amplification has not advanced the validation case.
2026-08-11T23:27:53Z
Discussion continues to cycle release-day model-card comparisons (vs Qwen3.6, Meta Muse) and anecdotal use-cases; no independent quality-throughput-memory-runtime measurements have appeared across five refreshes. Repetitive amplification with no new information — case remains a credible but unvalidated benchmark candidate.
2026-08-11T22:25:46Z
Refreshed comments remain release comparisons, speculation, and isolated anecdotes rather than reproducible local measurements. The model is still a credible benchmark candidate, but its practical quality-throughput, memory, and runtime tradeoff remains unvalidated.
2026-08-11T22:08:11Z
Discussion continues to be release-day model-card comparisons (vs Qwen3.6, Meta) and anecdotes; no independent quality-throughput-memory-runtime measurements have appeared across four evidence items and multiple refreshes. The case remains a credible but unvalidated benchmark candidate — repetitive amplification, not new information.
2026-08-11T20:23:12Z
evidence attached: hn.story.49263340 — NVIDIA's first-party announcement is meaningful corroboration of Nemotron 3.5 Lightning and also introduces the adjacent NeMo Switchyard serving-router artifact.
2026-08-11T19:34:26Z
The refreshed discussion still consists of release comparisons, expectations, and isolated use-case anecdotes rather than reproducible local measurements. The model remains a credible benchmark candidate, but its quality-throughput, memory, and runtime tradeoff is still unvalidated.
2026-08-11T17:45:54Z
Refreshed discussion remains release-day comparison and speculation, not independent local inference validation. Without measured quality, throughput, memory use, or runtime compatibility, the case’s meaning has not advanced.
2026-08-11T16:50:25Z
The official-weight repost and refreshed comments confirm availability but add only expectations, model-card comparisons, and isolated use-case anecdotes. No independent quality, throughput, memory, or runtime-compatibility measurements advance the practical local-inference tradeoff.
2026-08-11T16:24:13Z
evidence attached: reddit.post.1vllhz2 — The official Nemotron 3.5 Lightning weights provide a confirmed artifact for the open model's local-inference validation.
2026-08-11T16:03:39Z
Refreshed discussion remains repetitive release-day reaction and model-card comparison, with no independent measurements of quality, throughput, memory use, or local-runtime compatibility. The artifact remains testable, but the validation case has not advanced.
2026-08-11T14:59:52Z
grounded: known/low — Scott already holds the relevant position in Capability Audit and Hardware-aware local inference: model utility must be established through representative, hard
2026-08-11T14:57:32Z
The exact Nemotron 3.5 Lightning artifact and local GGUF path are now credible, but the added HN discussion supplies only model-card comparisons and anecdotal impressions rather than independent quality-throughput measurements. The case is testable, not yet validated.
2026-08-11T14:28:21Z
evidence attached: hn.story.49257947 — Links NVIDIA's released Nemotron 3.5 Lightning artifact, directly enabling the open local-inference validation.
2026-08-11T13:54:29Z
A linked ggml-org GGUF conversion makes the named model more plausibly real and locally testable, moving the case beyond a bare release pointer. No independent quality, throughput, memory, or compatibility results have arrived, so the core tradeoff remains unresolved.
2026-08-11T13:28:58Z
grounded: known/medium — Scott already holds the core position in Capability Audit and Evaluation-Driven Development: vendor benchmark claims require representative, independent validat
2026-08-11T13:25:56Z
case created — NVIDIA has released a concrete open-model artifact whose sparse activation profile is directly testable on local hardware.

Decision trace

08-16 17:34expireThe release episode has faded without a matched comparative benchmark: existing independent tests consistently indicate fast, 24GB-class deployment but uneven quality, while the latest activity is onl
08-16 17:34alert_silentNo new measurement, implementation, or compatibility finding has arrived; the small engagement change does not alter the established directional evidence, so this can wait unless a controlled quality-
08-16 17:34alert_routeNo new measurement, implementation, or compatibility finding has arrived; the small engagement change does not alter the established directional evidence, so this can wait unless a controlled quality-
08-14 17:26repriceThe refreshed discussion adds only an unsupported preference for competing models, not a new measurement or compatibility result. Existing tests still support fast, 24GB-class local deployment with un
08-14 17:26alert_silentNo consequential new fact has arrived beyond commentary on the existing RTX 3090 result, so waiting for a controlled quality comparison or matched-runtime benchmark carries little attention regret.
08-14 17:26alert_routeNo consequential new fact has arrived beyond commentary on the existing RTX 3090 result, so waiting for a controlled quality comparison or matched-runtime benchmark carries little attention regret.
08-14 16:21sensor_dirtycomment_update
08-14 12:28repriceThe same-GPU RTX 3090 test strengthens the practical deployment case by showing that a vLLM-compatible W4A16 quant can fit in 24GB and substantially improve high-batch throughput. It reinforces Nemotr
08-14 12:28alert_silentThis is a useful implementation result, but not a time-sensitive verdict: a controlled quality comparison or matched runtime benchmark is still needed before it changes a local-model decision.
08-14 12:28alert_routeThis is a useful implementation result, but not a time-sensitive verdict: a controlled quality comparison or matched runtime benchmark is still needed before it changes a local-model decision.
08-14 12:22alert_silentA concrete single-RTX-3090 result suggests W4A16 under vLLM can greatly improve high-batch throughput while fitting in 24GB, but it compares different quantization/runtime stacks and provides only nar
08-14 12:22surface_candidateA concrete single-RTX-3090 result suggests W4A16 under vLLM can greatly improve high-batch throughput while fitting in 24GB, but it compares different quantization/runtime stacks and provides only nar
08-14 12:22alert_routeA concrete single-RTX-3090 result suggests W4A16 under vLLM can greatly improve high-batch throughput while fitting in 24GB, but it compares different quantization/runtime stacks and provides only nar
08-14 12:22attachAn independent same-GPU benchmark compares Nemotron quantization formats and reports a substantial throughput tradeoff relevant to local serving.
08-14 12:21propose_attachAn independent same-GPU benchmark compares Nemotron quantization formats and reports a substantial throughput tradeoff relevant to local serving.
08-14 11:25repriceRefreshed discussion adds no new reproducible benchmark or compatibility result beyond the existing informal pattern of fast local inference with uneven task quality. The case remains open but has shi
08-14 11:25alert_silentThe delta is repetitive discussion and engagement movement without a new measured performance, quality, memory, or runtime finding, so it can wait for routine review.
08-14 11:25alert_routeThe delta is repetitive discussion and engagement movement without a new measured performance, quality, memory, or runtime finding, so it can wait for routine review.
08-13 16:21sensor_dirtyengagement_update
08-13 15:21sensor_dirtyengagement_update
08-13 12:21sensor_dirtyengagement_update
08-13 11:23repriceRefreshed comments add no new measurement, reproducible benchmark, or compatibility finding beyond the established informal pattern of fast local inference with uneven quality. The case remains unreso
08-13 11:23alert_silentThe new delta is only refreshed discussion without a consequential performance, memory, runtime, access, or reproducibility result, so it can wait for the normal briefing cadence.
08-13 11:23alert_routeThe new delta is only refreshed discussion without a consequential performance, memory, runtime, access, or reproducibility result, so it can wait for the normal briefing cadence.
08-13 11:21sensor_dirtycomment_update
08-13 11:21sensor_dirtycomment_update
08-13 09:21sensor_dirtyengagement_update
08-13 08:21sensor_dirtyengagement_update
08-13 06:21sensor_dirtyengagement_update
08-13 04:21sensor_dirtyengagement_update
08-13 02:51repriceThe refreshed discussion adds no measured result or independent benchmark beyond the existing informal pattern of high throughput and uneven task quality. The practical local-inference tradeoff remain
08-13 02:51alert_silentThis is only a comment refresh with no new performance, memory, compatibility, access, or reproducibility finding; it can wait for normal briefing cadence.
08-13 02:51alert_routeThis is only a comment refresh with no new performance, memory, compatibility, access, or reproducibility finding; it can wait for normal briefing cadence.
08-13 02:21sensor_dirtycomment_update
08-13 01:42repriceRefreshed comments add no new measured result or independent benchmark; they repeat the established directional pattern of strong local throughput with uneven coding quality. The practical quality-thr
08-13 01:42alert_silentThe delta is only refreshed discussion without a consequential new performance, compatibility, memory, or access fact, so it can wait for normal briefing cadence.
08-13 01:42alert_routeThe delta is only refreshed discussion without a consequential new performance, compatibility, memory, or access fact, so it can wait for normal briefing cadence.
08-13 00:22sensor_dirtycomment_update
08-13 00:22sensor_dirtycomment_update
08-13 00:22sensor_dirtycomment_update