Hot Chips 37 / 2025 / Recording Q&A
Fabrics, optics and the cost of recovery
Fabric behaviour, memory access and recovery can determine the value of faster compute.
Updated 2 October 2026 · Analyst interpretation of historical disclosures
Where audience questions concentrated
131 eligible audience asks in the reviewed caption material. Networking led by question volume, followed by optics. Fabric questions recurred across vendors, while optical packaging drew repeated follow-ups from a smaller specialist group. Model questions concentrated in the keynote.
Original Executive Brief figure. Darker cells indicate more audience questions in reviewed session blocks; both axes run from hottest to coolest. Moderator prompts are excluded. Q&A time is not normalized.
Counts, coverage and session key
The figure is reproduced from the existing Executive Brief, without rebuilding its session matrix. Exact annual counts for the twelve common cross-year themes are available in the ten-year data; they do not cover the full annual taxonomy.
Conference coverage is partial. Moderator prompts are excluded from audience-question rankings. Caption errors and uneven session coverage limit comparisons; these counts describe reviewed evidence, not a complete conference census.
Based on the existing executive brief and its review of YouTube automatic captions. Conference coverage is partial; moderator prompts are excluded from audience-question rankings. Caption errors and uneven session coverage limit comparisons. Rankings describe the reviewed material, not the full audience or industry.
Two-page Executive Brief · Exact ten-year theme counts (CSV)
Patterns in the reviewed Q&A
Fabric mechanisms mattered beyond bandwidth.
Questions probed retries, buffers, programmable controls and mixed-vendor operation. Interoperability did not establish equal performance.
Optics exposed packaging and repair tradeoffs.
Thermal paths, fiber connections and driver placement drew scrutiny. External laser replacement left internal optical-engine repair unresolved.
Metric boundaries changed the interpretation.
Marvell excluded DRAM latency from its expander figure. dMatrix reported a core efficiency metric; NVIDIA clarified the traffic used in its noise test.
Memory expansion needed an access model.
Questions tested host paths, capacity and coherence. Marvell scoped coherence to the expander; IBM discussed added memory-path latency and capacity tradeoffs.
Reliability required system-level recovery.
Stacked-memory mitigations lacked field failure statistics. Huawei’s recovery depended on checkpoints and spare racks, not just a redundant chip.
Implications and concrete vendor asks
Analyst inference: these present-day acceptance tests follow from historical uncertainties. Historical statements and roadmap promises do not establish later delivery.
| Decision and timing | Vendor ask and decision test |
|---|---|
| Scale-up and scale-out Before design freeze | NVIDIA, AMD, Intel and Broadcom: run matched mixed-vendor tests with incast, background traffic and link faults. Report loaded tail latency, retry limits and recovery. Choose on delivered job performance and operating cost. |
| KV-cache offload Before architecture lock | Memory and fabric suppliers: measure the complete GPU-to-memory path, including DRAM, queues, copies and synchronization. Compare local memory, RDMA and coherent options on long-context serving. Offload must lower cost while meeting TTFT and TPOT targets. |
| Head nodes Before shortlist | AMD and Intel: compare Venice and DMR with identical accelerator I/O, memory population and host workloads. Include cross-domain shared-data access and contention. Component latency claims cannot substitute for the full serving path. |
| MI400 Before prototype | AMD: provide workload-matched serving results with explicit precision, quality, memory residency and whole-server energy. Test concurrent traffic and recovery. Advance only if the gain persists beyond accelerator-core measurements. |
| Optical reliability Before qualification | NVIDIA, Ayar Labs, Lightmatter and other optics suppliers: disclose internal engine repair, spare requirements and workload interruption. Compare ring and alternative modulation under matched thermal transients and complete power boundaries. |
Supporting architectural interpretation
Recurring themes within the reviewed questions and answers. The implications below are architectural interpretation, not a ranking of industry demand.
1. The fabric is judged by behaviour under load
Retries, buffers, programmable controls and mixed-vendor operation recur in the networking review; compatibility did not establish equivalent performance.
Architectural implication: Compare delivered communication and tail latency during contention and failure, not aggregate link bandwidth alone.
2. A component metric can omit the dominant cost
Memory-expander latency excluded DRAM access, core efficiency did not establish whole-card efficiency, and traffic conditions changed noise-test interpretation.
Architectural implication: Define the path, traffic and energy boundary before using a vendor number in a system comparison.
3. Reliability extends to the recovery and repair path
Optical-engine repair remained unresolved beyond replaceable external lasers. Stacked-memory discussion lacked field statistics, while system recovery depended on checkpoints and spare racks.
Architectural implication: Account for the replaceable unit, interruption, spare capacity and recovery procedure—not only device redundancy.
These are present-day evaluation recommendations drawn from the historical evidence, rather than claims that later products were discussed in this conference year.
Fabric performance under stress
Run mixed-vendor tests with incast, background traffic and link faults. Report loaded tail latency, retry limits and recovery; compatibility alone does not establish equivalent performance.
Complete memory paths
Include DRAM, queues, copies and synchronization in GPU-to-memory measurements. Compare local memory, RDMA and coherent paths under the same long-context serving targets.
Repair and recovery
Identify the replaceable unit, spare requirements and workload interruption for optical failures. Qualify thermal transients and full power boundaries, as well as checkpoint and spare-system dependencies.
Sources and scope
Based on the existing executive brief and its review of YouTube automatic captions. Conference coverage is partial; moderator prompts are excluded from audience-question rankings. Caption errors and uneven session coverage limit comparisons. Rankings describe the reviewed material, not the full audience or industry.
Research source: Hot Chips 2025 Executive Brief. Public edition omits raw question logs, private correspondence, personal attribution and attendee-only links.
Official Hot Chips 2025 site ↗ · Ten-year comparison →
Annual heatmap reproduced from Hot Chips 2025 Executive Brief. The format update preserves its source population and does not add new transcript or recording verification.