Hot Chips 34 / 2022 / Recording Q&A
The system around the accelerator.
Data placement, communication, software and power accounting determine how much accelerator capability becomes useful work.
Updated 2 October 2026 · Analyst interpretation of historical disclosures
Where audience questions concentrated
186 caption-attributed audience asks form the ten-year comparison subset. The original session heatmap below retains all 251 topical asks, including chair prompts.
251 coded topical asks across 11 recordings. Software and compilers lead (44), followed by scale-up/scale-out fabrics (32) and memory bandwidth/capacity (26). Machine learning (42) and GPU/HPC (41) contain the most asks, followed by mobile/edge (30).
Updated 28 September 2026. This replaces the earlier presentation-emphasis heatmap. Each recoverable topical ask has one primary theme; darker cells indicate more asks on a common raw-count scale. Themes and sessions are ordered by total count. Chair prompts and audience questions read aloud are included; Q&A duration is not normalized.
Counts, coverage and session key
Reviewed timestamped English automatic captions from the official recordings, without independent audio verification. Distinct requested issues and substantive follow-ups can count separately; repeats and clarification restatements do not. This is a recoverable-question inventory, not a certified complete census. No Slack content was accessed.
The 2023–2026 comparison excludes moderator prompts and uses within-year shares. The cross-year heatmap now uses a filtered subset of 186 caption-attributed audience questions, excluding 52 moderator questions and 13 unclear-origin questions. Themes are mapped and counts normalized to within-year shares. This annual session chart retains all 251 asks. Coverage and question-boundary differences still limit comparison.
Source: Hot Chips 2022 Independent Report, version 2, and Hot Chips 2022 Q&A Calculation, 28 September 2026.
Patterns in the reviewed Q&A
Bandwidth boundaries reach the software model
AMD explained MI250X’s two-device exposure through inter-die bandwidth relative to HBM. Intel confirmed coherent cross-stack access on Ponte Vecchio while warning of NUMA effects and offering a two-sub-device configuration. Coherence does not remove placement-sensitive performance.
Software readiness changes the value of hardware
NVIDIA described Transformer Engine as hardware plus software; AMD described framework integration as ongoing. Intel separated demonstrated throughput from a newer specification calculation, and Biren distinguished standard benchmark settings from customer-specific models. Evaluation needs the exact software release, workload and measurement basis.
Scale depends on mapping and coordination
NVIDIA described Ethernet or InfiniBand beyond the 256-GPU NVLink domain and the need to map work to bandwidth boundaries. Cerebras placed static-route planning and deadlock avoidance in software. CXL discussion identified directory growth and invalidation traffic as coherence scaling costs.
Dynamic workloads test static designs
Groq’s answers addressed predication, resident-code dispatch and nondeterministic I/O. Tesla explained that its memory ratio is configurable and that LLMs would likely need more DRAM around a tile. Workload balance must be tested rather than inferred from topology.
Access latency and resilience require complete paths
Grace Q&A emphasized translation throughput while declining a remote-memory latency figure. Tesla discussed ECC, retries, redundancy and limited rerouting; Juniper connected retransmission to residual errors across many links and challenged inconsistent power-reporting conditions. Test translation, remote access, degraded-mode throughput and recovery together.
Implications and concrete vendor asks
Analyst inference: these present-day acceptance tests follow from historical uncertainties. Historical statements and roadmap promises do not establish later delivery.
| Decision and timing | Vendor ask and decision test |
|---|---|
| Head-node selection: Venice versus DMR Before the associated selection or procurement decision | Ask: Matched workload, accelerator/NIC placement, memory capacity and software; host CPU time, local/remote paths, contention and tail latency. Decision test: A measured host or data-path bottleneck is removed at lower system cost while service targets hold. |
| MI400 engagement Before the associated selection or procurement decision | Ask: Reproducible prefill, decode and MoE at matched quality, context, batch and precision; host/NIC use, software gaps and whole-server energy. Decision test: Useful throughput and cost per token improve after host, network and software costs, within latency targets. |
| Scale-up and scale-out Before the associated selection or procurement decision | Ask: Useful AllReduce, All2All and point-to-point performance under mixed traffic, incast and failure; address, coherence and interoperability boundaries. Decision test: Workload communication and recovery improve. A collective-specific gain is not generalized to all traffic. |
| Disaggregated inference and KV offload Before the associated selection or procurement decision | Ask: HBM versus DDR, CXL and RDMA-backed tiers, including reuse, placement, transfer, synchronization, concurrent updates and recovery. Decision test: Capacity savings exceed movement and coordination costs while preserving time-to-first-token and per-token latency. |
Supporting architectural interpretation
These are 2022 speaker statements recovered from automatic captions. They do not establish subsequent delivery or independently verified performance. Source report references GPU/HPC S1, machine learning S4, networking S5, ADAS/Grace S6, Tesla keynote K2 and CXL T1.
What the presentation review adds
Memory and movement recur together across GPU, machine-learning and CXL presentations. The mechanisms differ: coherent host links, PCIe nodes, switched accelerator networks and pooled memory create different software and failure boundaries.
The material mixes architecture descriptions, measured examples, projections and simulations. It supplies a stronger basis for qualification tests, rather than a present-day product winner.
Patterns across the year
Recurring themes within the reviewed presentations. The implications below are architectural interpretation, not a ranking of industry demand.
1. Compute designs are also data-movement designs
Reuse and memory hierarchy recur in NVIDIA, AMD, Intel and Biren material; Groq and Cerebras extend locality into distributed execution.
Architectural implication: Treat placement and communication as part of compute efficiency, not an overhead added after selecting the accelerator.
2. Integration creates different software contracts
CXL pooling, coherent CPU–GPU paths, PCIe nodes and explicitly mapped remote addresses offer different operating models. MLIR and distributed execution designs expose software responsibilities.
Architectural implication: Specify ownership, synchronization and placement for the exact configuration; integration does not make these obligations disappear.
3. The comparison boundary changes the result
Pre-silicon estimates, simulations and normalized performance ratios coexist with measured examples. Cooling, laser power and serviceability sit at different accounting boundaries.
Architectural implication: Compare reproducible systems with complete energy and maintenance costs. Presentation emphasis is not evidence of audience demand.
Supporting presentation findings
The vendor examples below summarize the reviewed presentation evidence. The evaluation recommendations are architectural interpretation.
1. Locality is part of the compute design
NVIDIA, AMD, Intel and Biren describe reuse and memory hierarchy alongside arithmetic. Groq and Cerebras extend locality into distributed execution. Measure time spent moving and waiting for data before buying more peak compute.
2. The node and fabric are part of the product
AMD’s MI200 material distinguishes coherent HPC nodes from PCIe-based configurations. NVIDIA’s NVLink Network uses explicit remote-address mapping; Tesla separates data-loading resources from ML compute. Qualify the exact host, accelerator and network topology together.
3. Pooling, sharing and coherence are different contracts
The CXL tutorials separate resource allocation from coherent multi-host sharing. Grace and MI200 describe particular CPU–GPU memory paths. Specify which agents may cache each region, who owns updates and what synchronization costs. A shared address alone does not settle these questions.
4. Software controls placement and communication
Groq, Tesla and Cerebras expose placement and communication responsibilities. Arm’s MLIR material makes operator and quantization contracts explicit. Include porting, scheduling and debugging effort in the architecture comparison, and test model coverage directly.
5. Physical integration needs complete power accounting
Tesla presents tile-level power and cooling alongside computation. Lightmatter highlights packaging and serviceability; Ranovus separates engine and laser power. Compare complete links and maintainable systems, including cooling and repair, rather than isolated component efficiency.
6. Performance claims require maturity and boundary checks
Grace results include pre-silicon estimates; Untether and AMD SmartNIC results include simulations. AMD’s DGEMM comparison uses per-compute-unit normalization across differing systems. Require a reproducible workload, configuration and power boundary before accepting an economic advantage.
Poster signal: serving concerns were already visible
Separately from the main-session evidence, KAIST’s DFX presentation addresses transformer generation and key/value buffering; SK hynix discusses GPT-3 inference with memory-side compute. These are early implementation precedents. They do not prove that a current disaggregated KV tier meets its latency or cost target.
A common measurement contract
Request measured versus projected status; exact hardware and software; workload, precision and quality; absolute results alongside ratios; and the full energy boundary. Resolve mismatched bandwidth or configuration figures before modelling total cost.
Use the same workload and measurement contract across CPU, accelerator, memory and fabric discussions. Ask each vendor which bottleneck its proposal removes, which one it creates, and what measurement would disprove its advantage.
Sources and scope
Adapted from Hot Chips 2022 — Independent assessment with recording Q&A heatmap, version 2, updated 28 September 2026.
The latest recording assessment codes 251 topical asks across eleven official recordings using automatic English captions. It replaces the earlier presentation-only Q&A coverage limitation; presentation findings remain supporting context.
The companion Hot Chips 2022 to 2026: Patterns and decisions for AI infrastructure supplies the longitudinal context. The updated cross-year page now incorporates this later independent 2022 assessment while keeping its evidence type separate from the 2023–2026 question reviews.