Meta AI Infrastructure Architecture Evolution

2015-2026 | 2026-09-28

01 Scope and executive synthesis

This report covers Meta’s AI infrastructure from 2015 to 28 September 2026: MTIA accelerators, their integrated AI networking, system packaging and software, plus the related MSVP video-processing branch. Merchant CPUs, GPUs and switch ASICs are distinguished from Meta-designed silicon. Earlier years establish infrastructure context rather than an invented annual sequence of custom chips.

MTIA evolves from low-power recommendation inference into an HBM-based training platform and then an inference-oriented generative-AI roadmap. The decisive transition is not merely more arithmetic: it adds network chiplets, near-memory reduction and rack-scale communication while preserving a PyTorch-facing software contract. Meta’s workload ownership allows narrower optimization than a merchant GPU, but portability, reliability and model evolution remain constraints.

02 MTIA100 and MTIA200: recommendation inference

MTIA100, originally MTIA v1, was designed around recommendation inference and disclosed at ISCA 2023. Its 7 nm implementation combines an 8×8 processing-element grid, local memories and a larger shared SRAM tier with LPDDR5. The key architectural balance is enough arithmetic for dense portions of the model without sacrificing the storage and access behavior needed by sparse features. This is a different objective from maximizing large-batch transformer training FLOPS.

MTIA200, originally MTIA2i, retains the 64-PE organization while moving to 5 nm, higher frequency, more local storage and 256 MB shared SRAM. Dense FP16/BF16 rises from 51.2 to 177 TFLOP/s, while LPDDR5 bandwidth increases only from 176 to 204.8 GB/s. The design therefore relies increasingly on on-chip reuse and sparse-workload scheduling. The 2024 platform fits 72 accelerators in a rack, but this is not the same as the 72-device dedicated scale-up domain introduced with MTIA400.

03 MTIA300: training changes the data path

The ISCA 2026 paper describes one compute chiplet, two network chiplets and six HBM3e stacks. A 12×6 active PE array combines RISC-V vector control, dot-product, special-function, reduction and DMA engines. The memory system provides 216 GB and 6.1 TB/s. Redundant PE resources address yield and faults. Moving from LPDDR to HBM changes both packaging and the balance between dense layers, embeddings and collective traffic.

Each network chiplet integrates six 800 Gb/s RDMA controllers. The aggregate endpoint rate is 1.2 TB/s in each direction. Message engines and near-memory reduction let collectives progress without occupying the principal compute engines or passing the payload through the host PCIe link. The paper’s deployed scale-up configuration uses 800 GB/s, with support up to 1,000 GB/s; scale-out is 200 GB/s. The marketing table uses the supported maximum. Both configurations are retained rather than silently merged.

A one-to-one host-CPU/accelerator relationship supports optimizer offload. Large HBM capacity can permit larger local batches and fewer communicating trainers, but the benefit depends on optimizer state, precision and convergence constraints. HCCL can fuse communication and computation; the reported collective results are workload- and topology-specific. Integrated NICs reduce one bottleneck, not every synchronization or network bottleneck.

04 MTIA400, 450 and 500: the inference roadmap

The March 2026 roadmap identifies MTIA300 as in production, MTIA400 as laboratory-tested and moving toward deployment, and MTIA450/500 as planned for 2027. MTIA400 uses two compute chiplets and a 72-device scale-up system. MTIA450 prioritizes memory bandwidth and low-precision inference. MTIA500 moves to four compute chiplets plus separate network and host-I/O functions. These are disclosure statuses, not assumed public-cloud or merchant availability.

Architectural interpretation: reusing chassis and network infrastructure can shorten deployment cycles, but it fixes parts of the system bandwidth budget while compute and HBM continue to grow. From 400 to 500, scale-up remains 1.2 TB/s and scale-out 100 GB/s per device, while HBM bandwidth triples. The compiler must consequently preserve more locality or change parallelization. A 25× comparison from MTIA300 MX8 to MTIA500 MX4 crosses numerical formats and is not a like-for-like arithmetic speedup.

05 Generation specifications and disclosure reconciliation

MTIA reference specifications

The March roadmap lists MTIA300 at 0.6 PFLOP/s BF16, 1.2 PFLOP/s FP8/MX8 and 800 W module TDP. The ISCA paper lists 560/1,120 TFLOP/s and 912 W TDP with 667 W typical power. These represent different published configurations or measurement bases; no undocumented reconciliation is imposed. For derived ratios, the paper’s matched values are used. Likewise, MTIA200’s launch states 90 W while the later comparison table states 85 W. Sparse peaks are kept separate from dense values.

Host CPU boundary

06 Integrated NICs, networking and MSVP

MTIA network chiplets

MSVP is a dedicated video-transcoding ASIC disclosed in 2023. It belongs in the infrastructure portfolio because video processing consumes substantial fleet resources, but it is not an MTIA generation or a general neural-network accelerator. Likewise, Meta’s OCP server and networking designs, including switch software and rack standards, should not be confused with ownership of every merchant ASIC they contain.

Interconnect evolution

07 Rack architecture and software

2023-2024 stack

flowchart TD
  cpu["External host CPU"]
  gpu["MTIA PE array"]
  mem["LPDDR5 / SRAM"]
  sw["PyTorch / compiler / HCCL"]
  net["PCIe fabric / optional RDMA"]
  rack["Rack / cooling / operations"]
  cpu ---|"PCIe"| gpu
  gpu ---|"memory"| mem
  gpu ---|"data"| net
  net ---|"integration"| rack

Later diagram includes announced products; silicon and rack availability are tracked separately.

2026 roadmap stack

flowchart TD
  cpu["External host CPU"]
  gpu["MTIA PE array"]
  mem["HBM3e / SRAM"]
  sw["PyTorch / compiler / HCCL"]
  net["Integrated NICs / backplane"]
  rack["Rack / cooling / operations"]
  cpu ---|"PCIe"| gpu
  gpu ---|"memory"| mem
  gpu ---|"data"| net
  net ---|"integration"| rack

Later diagram includes announced products; silicon and rack availability are tracked separately.

System generations

Software stack

08 Normalized trends and engineering tradeoffs

Memory and communication balance

The large increase in bytes per BF16 FLOP at MTIA300 reflects a workload choice: recommendation training needs embedding capacity and traffic handling. Subsequent inference products raise both HBM and low-precision compute but retain a fixed rack network budget. Their success depends on model placement, expert routing, KV-cache behavior and compiler utilization. Compare at matched precision, quality, batch size and service-level latency, with module and rack power kept separate.

09 Conference and product crosswalk

Disclosure map

No product-specific ISSCC, HPCA, MICRO or ASPLOS circuit/architecture chain is established here for MTIA. ISCA provides the strongest direct product evidence. The public roadmap is broader than the available implementation papers: process, die details and dense/sparse arithmetic basis remain incomplete for 400/450/500. The report retains those gaps in the tables rather than extrapolating earlier silicon details.

Naming and ownership

ISSCC background

Source map

01 Scope and executive synthesis — C01, E01, E03, E04, P03, P04

02 MTIA100 and MTIA200: recommendation inference — C02, C03, P03, P04

03 MTIA300: training changes the data path — C01, C04, E01, P01

04 MTIA400, 450 and 500: the inference roadmap — E01

05 Generation specifications and disclosure reconciliation — C01, E01, E03, P03, P04

06 Integrated NICs, networking and MSVP — C01, E01, E03, E04, P01, P04

07 Rack architecture and software — C01, C04, E01, P01, P04

08 Normalized trends and engineering tradeoffs — C01, E01, P04

09 Conference and product crosswalk — C01, C02, C03, C04, C05, E01, E04, P02, P03, P04

References

Product and technical documentation

[P01] MTIA300 built-in NICs and collective offload. https://engineering.fb.com/2026/08/24/networking-traffic/mtia-300-meta-training-chip-built-in-nics/. Primary source; accessed by 2026-09-28

[P02] Inside MTIA hardware co-design, 2024. https://engineering.fb.com/2024/08/22/ml-applications/meta-mtia-hardware-co-design/. Primary source; accessed by 2026-09-28

[P03] First generation MTIA architecture. https://ai.meta.com/blog/meta-training-inference-accelerator-AI-MTIA/. Primary source; accessed by 2026-09-28

[P04] Second generation MTIA architecture. https://ai.meta.com/blog/next-generation-meta-training-inference-accelerator-AI-MTIA/. Primary source; accessed by 2026-09-28

Conference records and implementation disclosures

[C01] MTIA300 ISCA2026 paper. https://aisystemcodesign.github.io/papers/MTIA300_ISCA2026.pdf. Primary source; accessed by 2026-09-28

[C02] MTIA v1, ISCA2023. https://dl.acm.org/doi/10.1145/3579371.3589348. Official indexed record or related announcement; full document download unavailable.

[C03] Second generation MTIA, ISCA2025. https://dl.acm.org/doi/10.1145/3695053.3731409. Official indexed record or related announcement; full document download unavailable.

[C04] HCCL collective communications, 2026 preprint. https://arxiv.org/pdf/2608.00358. Primary source; accessed by 2026-09-28

[C05] ISSCC2019 LeCun plenary record. https://www.isscc.org/isscc-plenary-videos. Indexed record or searchable official page; product details cross-checked with the associated technical document.

Launches, ecosystem and deployment

[E01] Four MTIA chips in two years, March2026. https://ai.meta.com/blog/meta-mtia-scale-ai-chips-for-billions/. Primary source; accessed by 2026-09-28

[E03] Infrastructure evolution, 2025. https://engineering.fb.com/2025/09/29/data-infrastructure/metas-infrastructure-evolution-and-the-advent-of-ai/. Primary source; accessed by 2026-09-28

[E04] MSVP first video transcoding ASIC. https://ai.meta.com/blog/meta-scalable-video-processor-MSVP/. Indexed record or searchable official page; product details cross-checked with the associated technical document.