Where Does the Next $100 Billion of AI Capex Actually Go?

1.1 Hyperscalers (hyperscale cloud providers)

The small group of technology giants that build and operate giant data centres themselves and then rent that compute on demand to the rest of the world. Their combined annual spending on buildings, equipment and chips — what the industry calls capex, or capital expenditure, meaning the large one-off outlays a company makes for future business — runs into the hundreds of billions of dollars, and constitutes the single largest order book in the entire AI supply chain. Think of them as super-landlords who have leased an entire industrial park and then sublet the sheds: their budgets determine how much work everyone downstream gets.

1.2 Frontier Model Labs (model labs)

Teams dedicated to training the most advanced large AI models. The bigger and more capable the model, the more outlandish the compute required, which makes these labs the most concentrated and most insatiable pocket of compute demand anywhere. Most are not independently listed but have been funded or absorbed by the giants: Microsoft stands behind OpenAI, Amazon has invested in Anthropic, and Google has DeepMind in-house.

1.3 Sovereign AI (state-level compute programmes)

AI compute infrastructure funded by a state or government and kept under national control. Worried about having their compute supply choked off by someone else, governments everywhere are moving to build AI infrastructure of “their own,” and that has spawned a distinct class of government-facing compute orders. It is much like every country wanting to build its own power station rather than buying electricity from the neighbours in perpetuity.

1.4 Enterprise Private AI (enterprise on-prem deployment)

Large enterprises that decline to hand their data to the public cloud and instead build an AI stack inside their own machine rooms. For reasons of data confidentiality, compliance or cost, banks, hospitals and manufacturers are willing to buy and run the servers themselves. “On-prem” simply means “on the customer's own premises.”

1.5 Neocloud Demand (GPU-cloud customers)

The customer base that rents AI compute rather than building machine rooms of its own, taking capacity by the hour or under contract from the “neoclouds” — companies whose business is renting out GPU compute. How active this cohort is provides one of the most direct read-throughs on whether AI demand is running hot or cold.

1.6 Vertical and Scientific Computing Demand (vertical AI / scientific HPC)

Demand from specialist fields using AI and supercomputing to crack problems specific to their industry. HPC stands for high-performance computing — running extremely complex calculations on vast amounts of compute. Pharma groups hunting for new drug molecules with AI, brokerages running AI risk control, defence agencies processing intelligence, factories running AI simulation: all are hard-cash buyers of compute.

1.7 Inference Token Opex (per-query inference spending)

This is not a category of company but a way of spending money that runs through the entire industry. “Inference” means using a trained model to answer a question — every prompt you send an AI triggers one inference — and a “token” is the smallest billable unit in which AI processes text. Every use of AI costs a little money, and the more it is used the more it costs; that spending accrues month after month like a utility bill, and it is the recurring engine underpinning long-run growth in compute demand. Building the factory is the one-off outlay; but once the factory is running, the meter never stops turning, and inference spending is that meter.

1.8 Reserved Capacity and Compute Offtake (multi-year take-or-pay contracts)

Customers signing long-term contracts years in advance to lock in future compute, with an undertaking to pay whether or not they use it — what the industry calls take-or-pay. Compute is scarce enough that large customers, fearful of being shut out later, simply book it up front. It is the equivalent of block-booking an entire tier of the arena before the concert has been announced: whether or not the seats are filled, the money is owed.

1.9 Vendor-Financed Demand (circular, vendor-funded demand)

Chip and equipment suppliers turning round and putting up the money — as equity or as credit — to help customers afford their own products. It can amplify near-term demand, but it also plants a hazard: money circles around inside the supply chain, and demand that looks robust is partly demand the supplier has funded itself. This is one of the more important warning signals for judging how genuine AI demand really is.

2.1 Public Cloud IaaS / PaaS (cloud services rented on demand)

Cloud services that rent out compute, storage and networking on demand. IaaS is “infrastructure as a service” (you rent the raw compute and storage and install your own software); PaaS is “platform as a service” (the development platform comes ready-made and you only write the application). IaaS is like renting an unfinished shell of a flat; PaaS is like renting a fully fitted-out office.

2.2 Hyperscaler Self-build (giants building their own facilities)

Compute that hyperscalers do not rent from anyone else but site, build and operate themselves. They are large enough that self-building is the cheaper option, so they account for a great deal of greenfield construction (building from nothing on empty land) and expansion of existing shells.

2.3 Neocloud GPUaaS (new-model clouds renting GPUs)

A new generation of cloud companies whose business is packaging up large blocks of GPUs (AI chips) and renting them out. “GPUaaS” means “GPU as a service.” They lock in revenue with long-term take-or-pay contracts and then borrow against those contracts to expand. They are the compute landlords of the AI era — except the building was put up with borrowed money.

2.4 Crypto-to-AI Conversion (mining sites repurposed for AI compute)

Sites originally built to mine Bitcoin, converted to host AI compute. Mining and AI are both extraordinarily power-hungry, and these operators already hold the scarce commodity — electricity, and land already energised — so they are redirecting existing power resources to serve AI. The most valuable thing on their balance sheet is not the mining rigs; it is the parcel of land that is already connected to the grid and can be powered up at will.

2.5 Wholesale Colocation (whole-building leases)

Leasing an entire data hall — routinely several megawatts of power — to a single large customer on a long-term basis. “Colocation” means placing a customer's equipment inside a professionally run facility, with the operator supplying power, cooling and security. “Wholesale” refers to large contiguous areas on long contracts. It is the equivalent of letting an entire floor to one large corporate on a lease running well over a decade.

2.6 Retail Colocation (rack-by-rack leasing)

Leasing facility space piecemeal — by the rack or by the cage (a fenced-off compartment) — to small and mid-sized customers, with power and cooling provided. Against the whole-hall wholesale model, retail serves many more and much smaller customers: unit pricing is higher but management is far more fragmented.

2.7 Interconnection and Carrier Hotels (network exchange hubs)

The traffic interchanges where networks and clouds converge and connect to one another. A “carrier hotel” is a building in which multiple carriers place equipment in order to interconnect; a “cross-connect” is the single cable that directly joins two of them. Customers pay a premium for the ability to reach every major network and cloud from close range. It works like an airport hub: the more routes it carries the more valuable it becomes, and a new airport finds it very hard to prise the route network away from an established hub.

2.8 Edge and Micro Data Centres (small facilities at the edge)

Small facilities built close to city centres and close to users, so that data processing sits nearer to the end user and latency — the round-trip time for data — falls. Video streaming, gaming and autonomous driving all suffer badly from lag, and moving the facility into your neighbourhood makes responses faster.

2.9 Data Centre REITs (real-estate investment trusts)

Real-estate investment trusts that exist to own and lease data centres. A REIT is a listed vehicle that packages property, distributes rental income as dividends, and trades on the stock market; it is obliged to pay out most of its earnings to shareholders. Buying one makes you a small shareholder in a data-centre landlord, taking a share of the rent each year.

2.10 Data Centre Infrastructure Finance (project finance and debt)

The asset-management giants putting up the hard cash that gets these multi-billion-dollar facilities built. An AI campus routinely costs billions to tens of billions, far more than an operator can fund from its own pocket, so the money comes through project finance (debt raised for one specific project), GPU-backed lending (borrowing against the chips as collateral), and ABS/CMBS (packaging rents or assets into bonds sold to investors). They are the true paymasters behind this construction wave, and what they earn is lending spread and equity returns.

2.11 Land Banking and Powered Shells (permitted land and energised empty buildings)

Sites secured in advance with permits already granted and power already connected, or buildings erected but not yet fitted with equipment. In the AI era the scarcest input is not the chip; it is a parcel of land that can be energised quickly, and whoever holds that land holds hard currency. A “powered shell” is a bare facility with power and water already in place, waiting only for a customer to move equipment in.

2.12 Speed-to-Power and Site Origination (siting for the fastest energisation)

The specific capability of finding and locking up sites that can be energised soonest. As noted earlier, joining the main grid means queuing for three to five years, so whoever can secure behind-the-meter generation (a campus generating and consuming its own power without waiting on the grid), or a favourable position in the interconnection queue, or the existing power feed of a converted industrial building, is the one who gets the customer switched on first. In a world where everybody is short of electricity, speed to power is itself the most valuable skill on offer.

3.1 Site Selection and Entitlement (finding land and securing permits)

Before ground is broken: selecting the site on the owner's behalf, assessing whether power and fibre are close at hand, running environmental and geotechnical surveys, and working zoning and approvals through to completion. “Entitlement” means assembling every permission needed to make the parcel legally buildable as a data centre. This is the reconnaissance and paperwork stage — and if the permits do not come through, everything downstream is wasted.

3.2 Engineering and MEP Design (mechanical, electrical and plumbing design)

Designing the building, mechanical, electrical, piping and control systems for the whole facility. “MEP” covers the three principal disciplines — mechanical (M, cooling and ventilation), electrical (E, power supply and distribution) and plumbing (P, water and drainage) — and is the drawing-board foundation of whether a facility will run reliably. This is the factory's full set of blueprints: how power is routed, how heat is rejected, how water flows, all settled at this step.

3.3 Structured Cabling Design (integrated cabling design)

Designing how the thousands upon thousands of network cables and fibres inside a facility are routed and organised. “Structured cabling” means planning otherwise chaotic cable runs into a manageable, extensible system to a defined standard — outside plant fibre, backbone runs, in-hall connections, cable tray (the overhead channels that carry cables), and so on. Without this design discipline, the cabling inside a facility becomes an unmaintainable tangle.

3.4 EPC General Contracting (engineering, procurement and construction)

A single firm taking on design, procurement and construction end to end and handing over the keys. “EPC” abbreviates engineering, procurement and construction; “design-build” means the same firm does both design and construction, removing the back-and-forth and building faster. The owner simply hands over the requirements and receives a facility that can be switched on; everything in between belongs to the contractor. In an AI build-out where schedule is everything, being able to build fast is itself the core competitive advantage.

3.5 Site Civil and Shell & Core (earthworks and building structure)

The hard civil work: foundations, site grading, buried utilities, and raising the building's primary structure. A data centre has to carry extremely heavy liquid-cooled racks, so foundations and structure must be substantially stronger than in an ordinary building. This is the factory's skeleton and footings — invisible, and indispensable.

3.6 Mechanical and Electrical Fit-out (on-site M&E installation)

The site work of physically installing, wiring and positioning transformers, switchgear, cooling pipework and control systems. “Fit-out” means getting the equipment on the drawings actually installed in the building and working. “White space” is the data hall area where the racks ultimately sit. Civil work produces an empty building; the M&E fit-out is what brings power, water and cooling into each hall.

3.7 Modular and Prefabricated Skids (factory-built modules)

Assembling complete power and cooling systems in a factory as standard modules or “skids” (equipment sets mounted on a base and shipped as a single unit), then craning and bolting them together on site, eliminating a great deal of field construction. It is Lego logic: rather than laying brick by brick on a building site, build the whole block in the factory, ship it, and clip it into place — faster and more consistent. In a schedule-driven AI build-out, that speed is extraordinarily valuable.

3.8 Containment and Raised Floor (aisle containment and raised access flooring)

The containment systems that separate cold air from hot, and the raised floors that create an underfloor void for cabling and airflow. “Containment” uses panels and doors to keep supply air and exhaust air from mixing, which raises cooling efficiency; a “raised floor” lifts the floor slab so the space beneath can carry cold air and cabling. Containment is effectively a dedicated lane for the cold air, stopping it from being wasted by mixing with the hot.

3.9 Project Management and Commissioning (owner's representation and acceptance testing)

Tracking schedule and controlling cost on the owner's behalf throughout, then testing and accepting every system in successive stages before handover. “Commissioning” runs from the individual device up to the whole campus, level by level (the industry labels them L0 through L6), confirming that power, cooling and backup all hold up under real load before anyone dares hand over. A “load bank” is the equipment used to simulate full-load heat output and prove that power and cooling can take it. This is the factory's full medical examination before start-up — and no keys change hands until it passes.

3.10 Fire and Life Safety Systems (fire protection and life safety)

Fire detection, suppression and life-safety systems for the data hall. A facility is full of chips worth hundreds of thousands of dollars apiece, and conventional water sprinklers would destroy the equipment, so operators use “clean-agent suppression” (discharging an inert gas that extinguishes without harming equipment) or “pre-action sprinklers” (pipes normally kept dry, charged with water only once a fire is confirmed). “VESDA” very-early-smoke detection raises the alarm before visible smoke appears. This is the factory's fire brigade and security detail at once: it must put the fire out without destroying the very expensive equipment in the process.

3.11 Structural Steel and Pre-engineered Buildings (steel frames and prefabricated structures)

The steel beams and frames that hold the building up, and factory-prefabricated metal buildings. Liquid-cooled racks are extremely heavy, so floor slabs and steelwork need purpose-designed load reinforcement. A “pre-engineered building” is one whose steel members are manufactured to standard in a factory and shipped for rapid on-site assembly. This is the factory's steel skeleton, built to carry racks far heavier than an ordinary building would ever see.

3.12 Roofing, Envelope and Waterproofing (roof, facade and damp-proofing)

Delivering a roof, facade and waterproofing layer that do not leak. A data hall is filled with electronics that cannot tolerate water, and any ingress through roof or wall is a major incident, so the envelope and its waterproofing must be absolutely watertight. The “envelope” is the building's outer shell — roof plus walls. This is the factory's raincoat, keeping the priceless equipment inside entirely separated from the weather outside.

3.13 Seismic Bracing and Restraint (earthquake reinforcement)

Securing heavy equipment, racks and pipework to seismic code so that nothing sways or topples in an earthquake. The kit inside a data hall is both heavy and expensive, and in seismic zones it requires purpose-made bracing, restraints and vibration isolators. This is a seat belt for very expensive equipment: when the ground moves, the kit does not tip over and destroy itself.

4.1 Regulated Utility Generation

The familiar electric utility: it generates power and sells it to customers over a government-regulated grid. Tariffs and allowed returns are set by the regulator, so revenue is as steady as an annuity on a long-term power contract — and equally incapable of growing quickly. It is a landlord collecting rent under government-controlled leases: assured in fair weather and foul, but never the route to an overnight fortune.

4.2 Independent and Merchant Generation (IPP / merchant generation)

IPP stands for Independent Power Producer. These generators are not subject to retail tariff regulation and sell their output at market prices or under long-term agreements directly to large customers — AI data centres among them. The essential difference from a regulated utility is that the price floats with the market, which is why this is also called merchant generation. If the regulated utility is the landlord with a controlled rent, this is the landlord quoting at whatever the market will bear — and earning considerably more when conditions are good.

4.3 Gas Turbines and Reciprocating Gensets

A gas turbine is the large machine that burns natural gas on site and drives blades with the resulting hot gas stream to generate electricity; reciprocating gensets (pistons moving back and forth, on the same principle as a car engine) suit more flexible small- and mid-scale duty and standby generation. This is the most realistic near-term answer to the large-power shortfall. Sub-types include: heavy-duty frame gas turbines — the largest and most efficient, used for baseload plant; industrial mid-range gas turbines — moderate output with better flexibility; aeroderivative turbines — fast start-stop, suited to peaking; reciprocating gas engine sets — modular and easy to maintain; dual-fuel reciprocating engines — able to switch fuel; CHP / cogeneration sets — higher overall energy efficiency; and mobile / temporary power units — trailer-mounted, for immediate relief.

4.4 Large Light-Water Nuclear Reactors (nuclear LWR)

LWR stands for Light-Water Reactor, the dominant commercial nuclear design in service today (ordinary water serves as both coolant and moderator). This component covers three distinct things: uprating and extending the life of existing ageing units, restarting units that have been shut down, and newbuild large pressurised-water (PWR) or boiling-water (BWR) reactors. The sub-segments include: uprate / life extension — getting more output and more years out of existing plant; restart / repower — bringing shut-down units back to generation, as with Microsoft's Three Mile Island restart and Amazon's purchase of Susquehanna capacity; large newbuild PWR / BWR — building a large nuclear station from scratch; the nuclear turbine island — the large steam-turbine set that converts steam into electricity; NSSS / RPV (nuclear steam supply system and reactor pressure vessel) — the core steel vessel that holds the fuel and raises steam; and nuclear fuel services and LEU (low-enriched uranium) — fuelling the reactor.

4.5 Small Modular Reactors (nuclear SMR)

SMR stands for Small Modular Reactor: nuclear plant built as standardised small modules, factory-produced and assembled on site, in theory cheaper, faster and more flexible. It is the favourite subject of headlines and the sexiest story in the layer — but the overwhelming majority cannot supply power at scale before 2030 and contribute nothing to the shortfall over the next year or two. The technology routes include: light-water SMRs — miniaturisation of mature light-water technology, closest to deployment; liquid-metal fast SMRs (LMFR) — the route taken by Oklo and NANO Nuclear; molten-salt reactors (MSR) — theoretically safer, and further out; high-temperature gas-cooled reactors (HTGR) — gas-cooled designs capable of high outlet temperatures; microreactors (<20 MWe) — units small enough to move by truck; SMR power island modules — the generating module of an SMR; and HALEU fuel services (High-Assay Low-Enriched Uranium) — the specialist fuel many SMR designs require.

4.6 Fuel Cells and Distributed Generation

A fuel cell does not burn anything; it generates electricity by reacting fuel (usually natural gas or hydrogen) directly, which makes it quieter and lower-emitting. “Distributed generation” means generating right on the campus, immediately adjacent to the load (behind-the-meter), without queuing for the main grid. Solid-oxide fuel cells are among the few options across the six generation routes that can be installed and generating now at scale (reciprocating gas sets and small gas turbines can also be deployed reasonably quickly on suitable sites). The sub-types: solid oxide fuel cells (SOFC) — Bloom Energy's core product, able to run directly on natural gas and installable today; proton exchange membrane fuel cells (PEM) — fast start-stop, used more often for backup; molten carbonate fuel cells (MCFC) — high-temperature, suited to cogeneration; hydrogen backup power — emergency reserve power on hydrogen; power conditioning — converting the fuel cell's DC output into usable AC; and reformer / desulfurisation skids — pre-treating natural gas into fuel clean enough for the cell.

4.7 Renewables: Solar, Wind and Geothermal

The collective term for clean generation. Solar PV and wind are cheap but intermittent, dependent on the weather; geothermal (generating from subsurface heat) is steady but geographically constrained. The new generation of “enhanced geothermal systems” (EGS, drilling to create an artificial heat reservoir) aims to break that geographic constraint. Sub-segments: central and string inverters — the core equipment converting panel DC output into AC; rooftop and carport PV — panels on a campus's own roofs and car parks; utility-scale solar — large ground-mounted solar farms; onshore and offshore wind — large turbines on land and at sea; and EGS and conventional binary geothermal — generation from subsurface heat.

4.8 Grid-Scale Storage and Long-Duration Storage (grid-scale BESS / LDES)

BESS stands for Battery Energy Storage System: storing power in large batteries and releasing it when needed, shaving peaks and filling troughs for intermittent sources like wind and solar. LDES stands for Long-Duration Energy Storage, capable of discharging continuously for far longer (hours to days) and addressing the problem of successive cloudy, windless days. It is the grid's power bank: charging when solar output is abundant during the day, discharging at night when it is not. Technology types: LFP container batteries and NMC container batteries — mainstream lithium storage; sodium-ion grid batteries — sodium in place of lithium, better on cost and resource availability; pumped hydro — the original large-scale long-duration store; gravity / mechanical, compressed air and thermal LDES — the various long-duration approaches; vanadium flow, iron-zinc and iron-air batteries — suited to very long discharge; and battery management systems (BMS) and container integration — monitoring and systems integration.

4.9 High-Voltage Transmission and HVDC

The high-voltage network that hauls power over long distances from remote generation to the campus. HVDC stands for High-Voltage Direct Current, which loses less over long distances and subsea crossings than conventional AC — at the price of expensive converter stations at each end to shuttle between DC and AC. AC is the ordinary road; DC is the motorway, and on long journeys the motorway saves more. Sub-segments: HVDC converter stations, LCC and VSC types — the core stations at each end that convert between DC and AC; HVDC converter transformers and valve stacks — the critical heavy plant inside those stations; HV overhead lines and underground HV cable systems — the circuits that actually carry the power; protection relaying — millisecond-scale isolation on fault; and transmission FACTS/STATCOM and series compensation — equipment that holds grid voltage steady and raises transfer capability.

4.10 Utility Substations and Switchyards

A substation is the grid's hub for stepping voltage up and down and for aggregating and distributing power; a switchyard is the open-air area within it filled with switches and busbars. This is where transmission-level extra-high voltage is stepped down and sent onward, with fault isolation, protection and metering handled along the way. It is also the slowest-delivering and most chokepointed stretch of the entire layer. Sub-segments: EHV power transformers and generator step-up units (GSU) — ordering now means waiting 36 months or more, the single tightest link in the chain; gas-insulated switchgear (GIS) and air-insulated switchyards (AIS); HV circuit breakers and disconnect / earthing switches; instrument transformers (CT / VT) — scaling large currents and voltages down to something an instrument can read; revenue metering and synchrophasors — precise energy measurement and grid phase monitoring; and substation RTU/SCADA hardware and protection & control panels.

4.11 Interconnection Queues and Grid Services

Before a large campus can draw power it must apply to the utility for interconnection, then queue while a grid impact study is run (assessing whether a load of that size will overwhelm the local network) — that is the interconnection queue. “Grid services” covers things like demand response (voluntarily shedding load at peak in exchange for compensation) that help keep the network stable. This is the step money cannot accelerate: physical construction plus regulatory approval means a median wait of roughly five years.

4.12 Power Purchase Agreements, Tariffs and Gas Supply (PPAs / tariffs / gas supply)

A PPA is a Power Purchase Agreement, the long-term contract under which a data centre buys power from a generator (usually locking in both price and volume — a commercial contract, not a piece of equipment). “Gas supply” covers the entire upstream that fuels an on-site gas plant: extraction, pipelines, storage and transport. This link is fundamentally commercial and contractual, and it determines from whom, at what price, and in what quantity the power is bought.

4.13 Gas Turbine Balance of Plant

A gas-fired plant is a great deal more than the turbine itself: “balance of plant” (BOP) is the full set of ancillary equipment outside the turbine that makes the station run. This is the supporting beneficiary set that most people overlook. Sub-segments: heat recovery steam generators (HRSG) — recapturing waste heat and lifting efficiency substantially; bottoming steam turbines and air-cooled condensers — condensing steam with air where water is scarce; feedwater pump skids and lube-oil skids; inlet air filtration and chilling — cleaning and cooling turbine intake air to raise output; demineralised water treatment — producing the pure water the boiler requires; and exhaust stacks and SCR (Selective Catalytic Reduction) — cutting NOx emissions.

4.14 Fuel Handling and Gas Infrastructure

The complete set of facilities that fuel on-site generation and store that fuel safely: gas pressure regulation and metering, compression and filtration, diesel tankage for backup, and eventually hydrogen and ammonia storage and handling. Without this link, the gas plant and the standby sets have nothing to burn. Sub-segments: onsite gas pipeline laterals — the spur that brings the trunk gas line onto the campus; metering and regulating stations and gas conditioning / filtration skids; gas compression skids and fuel transfer pumping; backup diesel storage tanks; and hydrogen storage and compression and ammonia carrier storage — for the clean fuels of the future.

4.16 Utility-Scale BESS PCS and Grid-Forming Inverters

Storage batteries hold direct current while the grid runs on alternating current, and the PCS (Power Conversion System) is the interpreter between them, converting bidirectionally. Grid-forming inverters go a step further: they actively establish voltage and frequency, holding up the framework of the network the way a conventional generator does — which matters more and more as inverter-based renewables take a larger share of the grid. An ordinary inverter dances to the grid's beat; a grid-forming inverter is the drummer setting the beat itself. Sub-segments: MV PCS skids and bidirectional inverter-transformer skids; grid-following and grid-forming inverters; battery plant controllers and storage EMS/SCADA gateway hardware; and black-start and grid-support control — using storage to restore supply first when the network goes dark.

4.17 Microgrid Controllers and Islanding (microgrid controller / BTM EMS / islanding)

When a campus simultaneously holds its own gas engines, fuel cells, storage and a grid connection, something has to conduct them — deciding when to generate, when to store, and when to switch to standalone operation. That is the microgrid controller. “Islanding” is the ability to disconnect from the main grid and run the campus on its own generation; “BTM” means behind-the-meter, on the campus side. It works like an orchestra conductor, keeping several in-house power sources and the grid itself playing in time rather than over one another. Sub-segments: microgrid controllers — coordinating multiple sources; islanding / reconnection controllers — safe transition off the grid and back on; and protection and synchronisation controllers — aligning phase at the instant of reconnection to prevent a shock to the system.

5.1 Medium-Voltage Switchgear (MV switchgear)

Switchgear is the circuit's combined gate valve and fuse assembly: it distributes current and, on fault, isolates within milliseconds to protect everything downstream. Medium-voltage (MV, typically a few thousand to a few tens of thousands of volts) switchgear splits the incoming MV supply out to individual buildings and halls, and a campus holds a great many of them. Sub-segments: air-insulated MV switchgear (AIS, insulated by air, physically large) and SF6-free gas-insulated switchgear (SF6-free GIS, insulated with environmentally benign gas, far more compact); metal-clad switchgear (metal compartments, high safety); MV vacuum circuit breakers (extinguishing the arc in vacuum to break the current); reclosers and sectionalizers (automatic re-close attempts after a fault, isolating the faulted section); protection relay panels (detecting anomalies and commanding the trip); and arc-resistant gear (containing an internal arc flash so it cannot injure personnel).

5.2 MV / LV Transformers

Transformers step voltage up and down. Facility-level transformers take medium voltage down to the low voltage (LV) the data hall can use. They come in two broad families: liquid-filled (cooled by insulating oil) and dry-type (oil-free, better suited to indoor fire safety). This is one of the longest-lead-time links in the whole distribution chain. Sub-segments: liquid-filled distribution transformers (high capacity, mostly outdoors); dry-type cast-resin and VPI transformers (epoxy or vacuum-pressure-impregnated insulation, safe indoors); K-rated harmonic-tolerant transformers (built to absorb the harmonics that a data centre's non-linear loads generate without overheating); isolation transformers (electrical isolation and interference suppression) and rectifier transformers (matched to rectifier equipment); and transformer monitoring sensors (real-time oil temperature, loading and health).

5.3 Unit Substations

Transformers, switchgear and associated components pre-assembled into a modular miniature substation, built complete in a factory and shipped to site, removing the time cost of installing each piece in the field — which is extraordinarily valuable to a schedule-driven AI campus. They are prefabricated substation Lego bricks: deliver, connect, energise. Sub-segments: prefabricated unit substations and skid-mounted power centres (complete power modules on a skid); e-house power modules (an entire prefabricated electrical room); and containerised substation pods (shipped in a standard container).

5.4 Facility Backup Gensets

The on-site generator sets that pick up the load the instant the grid fails (a “genset” being a generator set). Unlike the primary generation of Layer 4, these are purely emergency reserve: idle in normal operation, starting the moment the supply drops to hold up critical load. Sub-segments: standby diesel and standby gas gensets (the two mainstream emergency sources); HVO- and biodiesel-ready sets (running on cleaner renewable diesel); radiator cooling packages and emissions aftertreatment (bringing emissions within limits); and day-tank fuel systems (fuel stored close by so the set can always start).

5.5 Paralleling and Transfer Switching (paralleling / ATS / STS)

A campus with both a utility feed and several backup sets needs equipment to switch between sources and to parallel multiple sets. An ATS is an Automatic Transfer Switch (switching automatically from utility to generator on an outage); an STS is a Static Transfer Switch (using power electronics for seamless millisecond transfer, for sensitive loads); and paralleling switchgear coordinates multiple generators into a single virtual power station. Sub-segments: ATS and STS; generator paralleling switchgear (synchronised parallel operation of multiple sets); closed-transition transfer switches (briefly paralleling both sources so the load never drops) and bypass-isolation switches (allowing maintenance without an outage); and load-bank connection switchgear (connecting a dummy load to test the generators).

5.6 UPS Systems

UPS stands for Uninterruptible Power Supply, the battery-backed system that takes over the instant utility power fails. A chip cannot lose power for even a second — every running job would be lost and hardware may be damaged — and the UPS holds the supply through the few tens of seconds between the utility dropping and the backup generators coming up to speed. It is the airbag of the electrical system: inconspicuous in normal operation, and life-saving in a single critical moment. Sub-segments: online double-conversion UPS (power is converted to DC and back to AC, isolating the load from every grid disturbance — the cleanest output) is the data-centre mainstream; line-interactive UPS (for smaller applications) and delta-conversion UPS (a higher-efficiency architecture); transformerless modular UPS (dispensing with the heavy transformer and scalable on demand); diesel rotary UPS (DRUPS, using a flywheel plus diesel engine in place of batteries); lithium-ready UPS modules (moving from traditional lead-acid to lithium to save floor space); and UPS bypass switchgear (feeding the load around the unit during maintenance).

5.7 UPS Batteries, Lead-Acid and Lithium (VRLA / Li)

The cells that actually hold the charge inside a UPS. The traditional choice is VRLA (Valve-Regulated Lead-Acid: cheap and durable, but bulky and heavy), increasingly displaced by lithium (higher energy density, smaller footprint, longer life). Sub-segments: VRLA lead-acid (established and low-cost); LFP UPS batteries (safer) and NMC UPS batteries (higher energy density), the two lithium routes; sodium-ion UPS batteries (a newer route with friendlier resource economics) and nickel-zinc thin-plate cells; and UPS battery management systems (BMS, monitoring the safety of every cell) and battery cabinets and racks (support and cabling).

5.8 Flywheels and Facility-Level Storage UPS (flywheel / facility BESS)

Batteries are not the only way to hold instantaneous reserve. A flywheel UPS stores kinetic energy in a heavy wheel spinning at high speed and converts that rotational inertia back into electricity for seconds to tens of seconds when the supply drops (no chemical cells, long life, small footprint); supercapacitors can charge and discharge at very high power over very short intervals, riding through momentary voltage sags. The flywheel is a fast-spinning top: at the moment of failure, its momentum is turned back into electricity. Sub-segments: kinetic flywheel UPS and short-duration facility BESS; supercapacitor ride-through banks (absorbing momentary voltage sags); and UPS-BESS hybrid cabinets (combining UPS and storage in one enclosure).

5.9 LV Switchgear and Busway

Once power has been stepped down to low voltage, LV switchgear and busway (a metal conductor bar carrying high current — effectively a tappable power motorway) carry the distribution onward to each hall and each row of racks. Busway is easier to tap and more flexible to expand than cable. Sub-segments: LV switchgear and switchboards, and panelboards; facility busway / bus duct (the high-current trunk) and tap-off boxes (drawing power at any point along the run); air circuit breakers (ACB, high-current main breakers) and moulded-case circuit breakers (MCCB, protecting smaller circuits); and power bus monitoring modules (real-time busway current and temperature).

5.10 PDUs, Remote Power Panels and Rack Power (PDU / RPP / rack power)

A PDU is a Power Distribution Unit, dividing power out to each row of racks and then to each level within the rack. They come as floor-mount units (room-scale cabinets) and rack PDUs (the power strip inside the rack). An RPP is a Remote Power Panel, an extension distribution panel downstream of a floor-mount PDU. Rack PDUs are further graded by intelligence. Sub-segments: floor-mount PDUs and remote power panels; rack PDUs in basic, metered and switched variants (from supplying power only, to measuring it, to switching each outlet remotely); rack-level ATS (automatic transfer between dual feeds inside the rack) and branch-circuit monitoring PDUs (metering every circuit individually).

5.11 In-Rack Power Shelves and Battery Backup Units (power shelf / BBU)

Modern AI racks package power as “power shelves” slotted into the rack itself, centrally rectifying AC into the DC the servers need (48 V, for example) at higher efficiency and with easier maintenance. A BBU is a Battery Backup Unit, a small battery mounted in the rack that picks up locally the instant power drops, reacting faster than a centralised UPS. This is the stretch dominated by Taiwanese power ODMs, competing on thin margins and cost. Sub-segments: AC-DC power shelves, and 48 V / ORV3 power shelves (ORV3 being the Open Compute Project's Open Rack v3 standard); battery backup units and power-shelf rectifier modules (the core AC-to-DC stage); and power-shelf controllers (coordinating multiple power modules) and rack power busbar assemblies (the copper bars that distribute power within the rack).

5.12 HVDC Rack Power Conversion

This is the stretch of Layer 5 undergoing the most violent change, and the one most worth watching. HVDC stands for High-Voltage Direct Current. Because a 120 kW rack fed at traditional low voltage (12 V) loses far too much to copper resistance — the busbars would be too thick to physically fit — the power architecture is being forced to migrate from 12 V to 48 V to ±400 V and even 800 V DC. That has turned “rack power” into a new business that was a niche only a few years ago, and it has pulled power semiconductors into the story alongside it (high-voltage conversion calls for wide-bandgap, high-efficiency power devices such as gallium nitride and silicon carbide). Sub-segments: MVAC-to-LVDC power nodes (converting medium-voltage AC directly to low-voltage DC in a single step); rack HVDC rectifier systems and solid-state transformer (SST) power nodes (power electronics in place of the conventional iron-core transformer, far smaller and more flexible); HVDC-to-48 V DC-DC shelves (stepping the high-voltage DC back down to 48 V); and HVDC power sidecars (a standalone power unit hung alongside the rack) and DC distribution controllers.

5.13 Surge Protection, Grounding and Power Monitoring (SPD / grounding / power monitoring)

This stretch handles protection, earthing and metering. An SPD is a Surge Protective Device (blocking the transient voltage spikes produced by lightning or switching, protecting equipment downstream); grounding carries fault current safely to earth, protecting both people and equipment; and power monitoring meters each circuit and watches power quality. EPMS refers to the field hardware of the Electrical Power Monitoring System. Sub-segments: SPDs and grounding / bonding systems (safely discharging fault current); branch-circuit power meters (per-circuit metering) and power quality meters (monitoring harmonics and voltage variation); and insulation monitoring devices (detecting degrading insulation and leakage) and EPMS gateway / RTU hardware (the field acquisition layer of the monitoring system).

5.14 Facility Power Cables and Accessories

The bloodstream connecting everything above: MV and LV power cable, together with the terminations, tray and lugs that go with it. It looks unremarkable, but an AI campus consumes an enormous quantity of copper and holds cable to demanding fire and service-life requirements. Sub-segments: MV and LV power cable; termination and splice kits (insulating and sealing the joints) and cable lugs and connectors; cable tray, ladder and raceway (supporting and protecting bundled cable); and flexible power whips (the flexible final drop at the rack) and fire-rated cable systems (maintaining critical supply through a fire).

5.15 Power Quality and Reactive Compensation

The many power supplies and variable-frequency drives in a data centre generate “harmonics” (distortion of the current waveform, which heats equipment and erodes efficiency) and “reactive power” (the component that does no useful work yet occupies network capacity). This stretch is the equipment that treats those power-quality problems: filters strip out harmonics, while capacitors and STATCOMs compensate reactive power and hold voltage steady. It is noise cancellation and voltage regulation fitted to a noisy circuit. Sub-segments: active harmonic filters (actively cancelling harmonics) and passive harmonic filters (fixed components doing the filtering); power factor capacitor banks (compensating reactive power and lifting power factor) and reactor banks (current limiting and filtering); facility-scale STATCOMs (static synchronous compensators for fast voltage support) and static var compensators (SVC); and power factor correction controllers.

5.16 HVDC DC Protection Gear

As racks move to high-voltage DC, a complete set of equipment is needed to protect DC systems specifically. DC differs fundamentally from AC: alternating current crosses zero a hundred times a second, so the arc is easy to extinguish and the current easy to break, whereas direct current never crosses zero and the arc is extremely hard to quench. That makes DC breakers and DC fuses both harder to build and more critical. Solid-state DC breakers (interrupting instantaneously with power semiconductors) are the new frontier. Sub-segments: DC circuit breakers, solid-state DC breakers (semiconductor-speed interruption) and hybrid DC breakers (mechanical and solid-state combined); DC fuse switches and DC contactors / disconnects (making, breaking and isolating DC); DC busbar switchgear and precharge / discharge units (buffering inrush at energisation); and HV DC connectors (interconnects purpose-built for high-voltage DC).

6.1 Room-Level Air Conditioning (Air CRAC / CRAH, precision data-hall cooling)

This is the most traditional approach to cooling: large industrial air conditioning installed for the data hall. A CRAC (Computer Room Air Conditioner) carries its own compressor and refrigerant circuit, working on the same principle as the unit in your home, moving heat with refrigerant. A CRAH (Computer Room Air Handler) does not generate cooling itself; it uses chilled water piped in from elsewhere (chilled water — water cooled down by a chiller plant) to cool the air. Put simply: a CRAC is an air conditioner with its own built-in refrigerator, a CRAH is a fan unit that only moves air, with the cold source somewhere else. Sub-segments include dry (direct-expansion refrigerant) units, glycol-based units (antifreeze as the intermediate medium), fan-wall air handlers (an entire wall of fans), underfloor supply units, and humidification / dehumidification modules. In the AI era, pure air cooling can no longer hold down high-power racks and is being displaced by liquid cooling — but it remains heavily deployed in low- and mid-density halls and throughout the transition.

6.2 In-row Cooling & Containment

As a row of racks runs hotter and hotter, cooling the whole hall makes less sense than dropping the air conditioning directly between the racks (in-row) and cooling at close range, which is far more efficient. Containment means using doors, roof panels and blanking to physically separate cold air from hot air so the two do not mix randomly across the hall. Consider: if the cold air blown out at the front of a rack and the hot air exhausted at the back churn together in the room, you are effectively running the heating and the air conditioning at the same time, for nothing. Containment builds a wall around each aisle, so the cold stays cold and the hot stays hot. Sub-segments include in-row coolers (DX and chilled-water), hot-aisle containment, cold-aisle containment, rack air dams and blanking panels (plugging the empty rack positions so cold air cannot leak through), and containment door-and-ceiling systems.

6.3 Economizers & Evaporative Cooling (free cooling and evaporative cooling)

This is the family of free-cooling tricks, built on one idea: when it is cold outside, why spend electricity making cold? Just bring in the outdoor cold air or cold water. An economizer is the apparatus that admits that outdoor cold source, either air-side (drawing outdoor air in directly) or water-side (using cold outdoor water). Evaporative cooling exploits the physics of water absorbing heat as it evaporates — the same reason hosing down a patio in summer makes it feel cooler. Sub-segments include air-side economizers, water-side economizers, indirect evaporative coolers, direct evaporative coolers, adiabatic assist coolers (misting to add humidity and boost cooling), and free-cooling heat exchangers. This stretch bears directly on a data centre's electricity bill and water draw, and is the key lever on both.

6.4 Direct-to-chip Liquid Cooling (cold-plate liquid cooling)

This is the main battleground of AI cooling. A cold plate is a metal plate with fluid channels running through it, pressed directly onto the top of a hot chip — a GPU or CPU — so coolant flowing through the plate carries the heat away at source, far more efficiently than blowing air across an intervening gap. Consider: cooling a chip with air is like fanning a bowl of hot soup; a cold plate is setting the bowl itself into a basin of cold water. Much faster. It divides into single-phase (the fluid stays liquid throughout, carrying heat by flow) and two-phase (the fluid boils and vaporises inside the plate, absorbing far more heat but much harder to control). Sub-segments include single-phase cold plates, two-phase cold plates, GPU/CPU cold-plate assemblies, memory and power-module cold plates (even the memory and power components beside the chip now need their own plates), cold-plate manifold assemblies, and server liquid-cooling kits. This stretch is the physical precondition for an AI rack running at all.

6.5 Immersion Cooling (single-phase / two-phase immersion)

A more radical approach than the cold plate: submerge the entire server board in a tank of non-conductive fluid (a dielectric fluid — insulating, so it will not short a circuit board on contact) and let the liquid wrap around every heat-generating component. Put simply: a cold plate lays a cold towel on the chip; immersion gives the whole machine a cold bath. It splits into single-phase immersion (the fluid does not boil) and two-phase immersion (the fluid boils at the chip surface, condenses against a cold surface at the top and falls back, circulating on its own). Sub-segments include single-phase tanks, two-phase tanks, immersion blade-chassis kits, immersion condenser coils, immersion pump-and-filter skids, immersion lifting and service systems, and the dielectric coolant itself. Immersion remains relatively early-stage; the hard parts are servicing, sealing and the cost of the fluid.

6.6 Rear-door Heat Exchangers (rear-door HX)

An intelligent compromise: build a heat exchanger (a device that transfers heat from one medium to another) full of water channels and hang it on the rack as the rear door. Hot air exhausting from the back of the rack passes first through that grid of chilled-water tubes, the heat is captured on the spot, and the data hall barely feels the exhaust. Consider it a water curtain fitted over the rack's exhaust: the hot air is doused the moment it emerges. Variants include passive doors (no fan, relying on the rack's own fans to push the airflow), active fan-assisted doors (with integral fans for extra capacity), chilled-water doors, and rack-integrated designs. The advantage is that nothing inside the server has to change, which makes it the smooth bridge from air cooling to liquid.

6.7 Coolant Distribution Units (CDU)

The CDU is the heart and central exchange of the entire liquid-cooling loop. Its job is to take coolant from the data centre's primary loop, pass it through an internal heat exchanger, and hand that cooling capacity over to the secondary loop that feeds the cold plates inside the rack — while pumping, regulating temperature, filtering, and monitoring flow. Consider the district heat-exchange substation: the city's main network does not run into your apartment; it reaches a substation first, which then distributes the heat (here, the cooling) to each building. A CDU's rated capacity depends heavily on operating conditions: one Eaton CDU is rated at 2.3 megawatts of cooling at an 8°C delta-T, but only 1.15 megawatts at 4°C. Nameplate numbers mean nothing once you leave the conditions they were validated under, which is why the binding constraint in this link is certified capacity. Variants include liquid-to-liquid (L2L, rejecting heat to facility water), liquid-to-air (L2A, rejecting heat directly to air), in-row, in-rack and facility-scale units, plus pump-control and filtration assemblies.

6.8 Manifolds, Quick Disconnects & Hoses (distribution piping and quick couplings)

The vessels and fittings of the liquid-cooling loop. A manifold is a header with many branch ports, distributing coolant evenly to a row of racks or to every level within one rack — the same idea as the header in a bathroom that splits one supply into several shower heads. The quick disconnect (QD; UQD is the industry-common version) is the least conspicuous and most consequential part in the loop: it seals automatically on connection and does not leak on disconnection, and blind-mate versions (no alignment needed — push it in and it engages) connect without the operator even having to look. Consider a petrol nozzle that seals as it goes in and does not drip as it comes out. A high-density rack holds dozens of them or more, and a single leak can take an entire rack offline — or destroy several million dollars of silicon inside it. This stretch also covers rack secondary-loop manifolds, row-level distribution manifolds, drip-protected blind-mate couplings, quick disconnects, flexible coolant hose, stainless-steel and polymer coolant piping, and balancing valve assemblies.

6.9 Chillers

The chiller is the data centre's high-capacity refrigeration plant, dedicated to driving water down to a low temperature (producing chilled water) and sending it on to the CRAHs, rear-door exchangers and CDUs described above. It is, in essence, an oversized refrigeration machine. Variants include air-cooled units (rejecting heat to outdoor air, roof-mounted), water-cooled centrifugal units (rejecting heat via cooling towers, the most efficient, suited to large campuses), magnetic-bearing oil-free units (the compressor shaft levitates on a magnetic field with no lubricating oil — more efficient and more reliable), screw chillers, absorption chillers (driven by waste heat rather than electricity), modular units, and units with integrated free cooling. These are among the highest-priced and heaviest items in the whole facility-level cooling plant.

6.10 Cooling Towers & Dry Coolers

The last stop on the whole cooling chain: all the heat moved out by every stage upstream is finally rejected to the atmosphere here. A cooling tower carries heat away through water evaporation (and therefore consumes water); a dry cooler relies purely on air blown across metal fins (no water consumption, but lower efficiency). Consider the difference between cooling by sweating — effective, but you lose water — and cooling by fanning yourself: no fluid lost, weaker effect. Variants include open-circuit cooling towers, closed-circuit fluid coolers, dry air coolers, adiabatic hybrid coolers (dry most of the time, spraying water when conditions turn hot, balancing water draw against efficiency), condensate pump skids, and heat-rejection fan arrays. In water-stressed regions, choosing between a tower and a dry cooler is a decision about water rights.

6.11 Coolants, Dielectric Fluids & Refrigerants (working fluids)

The actual bloodstream circulating through the cooling loop. Several categories: water-glycol (the common antifreeze coolant), inhibited water (dosed to stop the piping corroding), single-phase dielectric fluids (non-conductive, for liquid cooling in direct contact with circuitry), two-phase fluorocarbons (specialty fluids that boil and vaporise — but many contain PFAS “forever chemicals” and are being squeezed by environmental policy), low-GWP refrigerants (low global-warming-potential alternatives), and an array of additives and testing services. As noted above, this is an annuity business: the hardware sells once, but coolant has to be tested, topped up and replaced on a schedule, and that money is collected year after year by chemical companies that never touch a cold plate.

6.12 Water Treatment, Reuse & WUE

Data centres are heavy water users, and this stretch is about managing that water, using less of it, and recirculating it. Makeup water treatment purifies incoming raw water to usable standards; blowdown recovery recaptures part of the water that a cooling tower would otherwise discharge once impurities have concentrated; reverse osmosis and deionisation (RO/DI) take water to extreme purity; and alongside these sit side-stream filtration, leak detection, greywater reuse (treating waste water for non-potable duty), and water-quality monitoring. WUE (Water Usage Effectiveness) measures how much water a data centre consumes per unit of electricity used. Consider: in a water-stressed region, how well a site handles water determines outright whether the campus gets permitted and whether it can keep operating.

6.13 Cooling Controls & BMS (building management)

This is the brain and nervous system of the cooling plant. The BMS (Building Management System) is the central controller governing a building's power, water and cooling; the hardware covered here includes cooling PLC controllers (PLC, programmable logic controller — the small industrial control computer on the plant floor), chiller-plant controllers, pump variable-frequency control panels (VFDs letting pumps modulate speed on demand and save power), thermal ride-through controllers (the life-saving function that maintains cooling through the seconds-to-minutes of a power interruption so the chips do not spike instantly), and control gateway / IO panels. Consider: once cooling stops on a 120-kilowatt rack, temperature can climb into the danger band within tens of seconds, which makes this system's ride-through capability the real safety floor. Note: this covers the control hardware; the pure software layer belongs to Layer 17.

6.14 Facility Sensors & Instrumentation

The sense organs of the cooling system: probes distributed throughout the hall, continuously measuring temperature, humidity, differential pressure, coolant flow, energy consumption, leaks and vibration, then passing the readings to the control system through IO gateways (the interface boxes that aggregate and upload field signals). Specifically: temperature probes, humidity probes, differential-pressure sensors, coolant flow meters, BTU energy meters (the heat meter that quantifies exactly how much heat was removed), leak-detection cable (laid under the floor, alarming and locating the moment a leak occurs), and vibration sensors. Consider leak-detection cable as an alarm rope running the length of the pipework: any seepage is identified immediately, down to the metre. For a rack worth several million dollars, that one second of warning is the difference.

6.17 Coolant Filtration & Fluid Conditioning (coolant purification and maintenance)

Coolant in a liquid loop gets dirty over time, drops out contaminants, and entrains air. This stretch is the purification and maintenance that keeps the fluid clean, in specification and free of blockage. Specifically: side-stream particulate filters (diverting a small flow off the main loop, filtering it and returning it, without disturbing primary flow), magnetic filter separators (capturing metallic debris), ion-exchange resin cartridges (stripping ions to hold high purity, preventing corrosion and conductivity), vacuum degassers (pulling out dissolved air, since bubbles degrade both heat transfer and pump performance), chemical dosing skids, dielectric-fluid reclamation units, and inline water-quality analysers. Consider it the equivalent of changing a car's oil and filters on schedule: skip it, and the loop will fail eventually.

6.18 Heat Reuse & Energy Recovery

Rather than throwing away the waste heat a data centre rejects, sell it or use it again. This stretch is the machinery for turning that waste into value: heat-recovery exchangers (extracting the waste heat), district heating interface skids (feeding waste heat into a municipal heating network — many Nordic data centres heat residential blocks this way), heat-pump upgrade systems (pumping low-grade waste heat up to a higher, more useful temperature), thermal storage tanks, organic Rankine cycle generation (ORC, recovering a little electricity from the waste-heat temperature differential), and greenhouse heating. Consider: one data centre's waste heat is enough to warm tens of thousands of nearby homes, and in Europe this is shifting from an environmental gesture into a contracted, revenue-generating business.

6.20 Water Sourcing & Wastewater

The far upstream and far downstream of water: where a data centre draws it from, how it discharges what it has used, and the municipal-facing infrastructure that handles both. Specifically: raw-water intake pump stations (drawing from rivers, lakes or wells), municipal supply connections, on-site wastewater pre-treatment (bringing effluent into compliance before it enters the municipal network), reclaimed-water pipelines (cooling with treated water instead of potable supply, easing pressure on the resource), on-site reservoirs and storage tanks, zero liquid discharge systems (ZLD, recovering all wastewater with nothing discharged — the highest environmental bar), and water-rights metering stations. Consider: in a water-short state such as Arizona, whether a developer can secure water rights and afford reclaimed water determines outright whether a multi-billion-dollar campus gets built.

7.1 Merchant GPUs

GPU stands for graphics processing unit — originally the chip that drew video-game frames, until people discovered it was exceptionally good at working through a great many similar small calculations at once, which is precisely what AI requires. “Merchant” means it is sold openly to anyone, unlike chips that large firms design and keep behind their own walls. Put simply: the GPU is the general-purpose money printer of the AI era. Training large models and running AI inference both rest on it today.

7.2 Custom Training ASICs / XPUs

ASIC stands for application-specific integrated circuit — a chip tailored to one particular task. Where a merchant GPU does everything, an ASIC does one thing, but does it faster, at lower power and at lower cost. XPU is the umbrella term for this class of dedicated accelerator. Consider: a GPU is a Swiss Army knife that will cut anything; an ASIC is a purpose-built kitchen knife that cuts one ingredient quickly and cleanly. Large cloud providers have each begun designing these dedicated chips, both to avoid being held hostage by a single supplier in NVIDIA and to save money.

7.3 Inference Accelerators / NPUs

Two terms worth separating first. Training is the months-long process of teaching an AI model; inference is the single burst of computation performed each time that trained model answers your question. NPU stands for neural processing unit, a chip optimised specifically for that inference step. Because a model, once trained, is then used billions upon billions of times, inference is a steady, extremely high-volume business — well worth a dedicated low-power, low-cost chip of its own.

7.4 FPGAs as Accelerators (reconfigurable silicon)

FPGA stands for field-programmable gate array. An ordinary chip's function is frozen the moment it leaves the fab; an FPGA is different — after it ships you can still use software to rewire its internal circuitry into whatever shape you need. Put simply: an ordinary chip is a house already built; an FPGA is a box of Lego you can dismantle and rebuild. It is flexible and quick to change, but expensive per unit of compute relative to dedicated silicon, so in AI it is used mostly where the design has to keep changing.

7.5 Wafer-Scale Engines (one chip from an entire wafer)

An ordinary chip is a small piece cut out of a round silicon wafer. A wafer-scale engine inverts that: do not cut at all, and make the whole wafer into a single enormous chip. Data then never has to leave the die and come back, which makes it extraordinarily fast internally. Consider: the conventional approach slices a large cake into many small pieces to share; the wafer-scale engine brings the whole cake to the table as one course. It is a niche path, but a genuinely distinctive one.

7.6 Dataflow, In-Memory & Photonic Compute (non-traditional accelerator architectures)

Almost every chip built today follows the old von Neumann architecture, in which compute and memory are separate and data has to shuttle back and forth between them — and the shuttling itself burns both power and time. This category covers the attempts to replace that architecture. In-memory computing lets the data be computed where it already sits, with no movement at all. Photonic computing uses light rather than electricity to compute, which is in theory both faster and far more efficient. Most of this remains early-stage and pre-commercial: high on narrative, high on risk.

7.7 Hyperscaler In-House Silicon Programmes

Not a new kind of chip, but a separate lens on the in-house design effort mentioned above. Google's TPU, Amazon's Trainium, Microsoft's Maia and Meta's MTIA are all chips the hyperscale cloud providers built themselves, to save money and to reduce their dependence on NVIDIA. As a rule they are not sold externally; the developers consume them internally.

7.8 Advanced Packaging Content in the Accelerator

A modern AI chip is not one isolated die but several smaller dies (including the high-bandwidth memory discussed earlier) assembled like building blocks and then sealed together with an expensive process step known as advanced packaging. This line item is the value captured in that packaging step. It is currently one of the genuine bottlenecks on AI chip capacity: whoever controls this step has pricing power.

7.9 Accelerator Board and Module Silicon (the chips surrounding the main die)

On an AI accelerator card, beyond the main processor, a whole ring of supporting chips is soldered around it — power delivery, signal retiming, management and more — collectively the board-level support silicon. This line item is simply the separate lens on that surrounding content. It is not itself a new chip type; it is the reminder that alongside the main die, that supporting ring is real value too (and precisely which chips those are is the subject of Layer 8, next).

8.1 Host Server CPUs — x86

CPU stands for central processing unit, the computer's commanding brain. In an AI server, the expensive accelerator does the heavy arithmetic, but a CPU is still needed as the commander — dispatching work and running the machine as a whole. x86 is a CPU instruction-set architecture that has been in use for decades (think of it as a common language standard), and essentially every conventional server and PC runs on it.

8.2 Server CPUs — Arm

Arm is a competing CPU architecture standard, set against x86. It first took hold in mobile phones, where its defining characteristic is power efficiency, and it is increasingly being deployed in data centres — because AI halls consume so much electricity that saving power is saving money. Consider: x86 is the powerful but thirsty large-displacement engine, Arm the fuel-efficient one. Over a long haul — and a data centre runs 24/7 — the fuel saved is considerable.

8.3 Logic ASICs / ASSPs

As covered above, an ASIC is a chip built for a specific task. ASSP stands for application-specific standard product — a chip designed for a class of application but sold openly to everyone. The distinction: an ASIC is customised exclusively for one customer; an ASSP is built once and anyone can buy it. Both handle the specialised odd jobs — networking, display, interfaces and the rest.

8.4 FPGAs as Logic (reconfigurable silicon as glue logic)

The same field-programmable part described above, whose circuitry can still be rewired after it ships — but used here not to accelerate arithmetic, rather as glue logic: absorbing the irregular connection, control and coordination work between the various chips on a board. Consider the custom adapters used in a renovation, joining two lengths of pipe whose specifications do not match.

8.5 Analog / Mixed-Signal

First, digital versus analog. Most of the chips discussed here process digital information — only ones and zeros. But the real world — temperature, sound, voltage, current — is made of continuously varying analog signals. Analog chips are the translators between those two worlds, turning real-world signals into digital form for the main processor, or the reverse. “Mixed-signal” means both digital and analog blocks on one die. This is an enormous and enormously fragmented field, running to hundreds of product categories (amplifiers, converters, interfaces and so on).

8.6 Microcontrollers (MCU)

MCU stands for microcontroller unit. Think of it as a small, cheap miniature computer with brain and limbs integrated into one part. Its compute is weak, but it is more than enough to manage one specific small job: controlling a fan speed, reading a button, running a sensor. They are everywhere — in appliances, cars, and equipment of every description. Inside an AI hall, they quietly run a great many auxiliary functions.

8.7 Digital Signal Processors (DSP)

DSP stands for digital signal processor: a chip built specifically to excel at signal-processing mathematics — stripping noise out of a muddy audio recording, decoding a radio signal. It is exceptionally efficient at the repeated multiply-accumulate operations those tasks require. Consider it a mental-arithmetic prodigy specialised in one type of repeated calculation: it does nothing else, but it does that at extraordinary speed.

8.8 Power Management ICs (PMIC)

PMIC stands for power management integrated circuit. A main processor needs several different supply voltages at once (one rail at 1V, another at 3.3V), each of which must be stable and efficient. The PMIC is the estate manager for power, splitting the incoming supply into each required rail and modulating continuously to save energy. Put simply: it is the main distribution room in a building, dividing incoming mains into the different specifications each floor and each room requires.

8.9 VRMs, Power Stages & Point-of-Load (multiphase and local power delivery)

An AI processor can draw over a thousand amps in an instant — a genuinely alarming magnitude — and ordinary power delivery simply cannot hold it. A VRM (voltage regulator module) is the power stage pressed right up against the main die, driving the voltage precisely to the level the chip demands while sustaining that enormous current. “Multiphase” means many parallel supply paths sharing the load. PoL stands for point-of-load: supplying power right where it is consumed, as close to the chip as possible, because transmitting current any distance incurs loss.

8.10 Discrete Silicon Power MOSFETs

A MOSFET is a power switching device — in plain terms, an electronic switch that can turn a large current on and off extremely fast. Power systems convert and regulate voltage stage by stage precisely by switching these devices hundreds of thousands of times per second. “Silicon” denotes the traditional material (more advanced materials come next). Consider a valve on a pipe that opens and closes at extraordinary speed, controlling flow — here, current — precisely through that rapid switching.

8.11 Wide-Bandgap Power Devices — GaN / SiC

A more advanced class of material than the silicon MOSFET above. GaN is gallium nitride and SiC is silicon carbide, known collectively as wide-bandgap materials. Switches built from them are faster, more efficient and able to withstand higher voltages than conventional silicon. AI halls are migrating to high-voltage DC distribution, which requires exactly this kind of high-efficiency device — making this a track that AI demand has activated. Put simply: silicon is the ordinary valve, wide-bandgap the premium valve that passes more flow in the same envelope with less effort.

8.12 Clocks & Timing

Hundreds or thousands of chips inside one server have to work in concert, and that requires a single metronome keeping everyone on the same beat — the clock chip. It generates the stable frequency reference that synchronises every part. Consider the conductor keeping time for an orchestra: without one, several hundred instruments (chips) descend into chaos. The faster the AI network runs, the tighter the precision that beat demands.

8.13 Sensors & Interface Logic

A data hall has to know, continuously, how hot each location is, how much current is flowing, and whether anything is out of bounds. That sensing is done by sensor chips, which convert physical temperature, current and the rest into digital signals to report upstream. Interface logic is the relay silicon that lets different chips speak to each other and keeps signals accurate in transit. Together they are the hall's nerve endings.

8.14 Baseboard Management Controllers (BMC / IPMI controllers)

BMC stands for baseboard management controller: an independent housekeeping chip on the server motherboard that stays alive even when the server has crashed — or has never been powered on — allowing an administrator to power-cycle remotely, read temperatures and diagnose faults. Put simply: it is the separately powered security desk in a building. When the main block loses power, the desk's lights stay on and someone is still watching the monitors. This is the linchpin of remote data-centre operations.

8.15 Vertical Power Delivery Silicon (VPD, feeding power from beneath the die)

As noted, an AI processor can draw a thousand amps or more — in some cases tens of thousands. At those currents, feeding power in from beside the chip is no longer sufficient; the copper simply cannot carry it. The new approach places the power silicon directly beneath the main die and feeds vertically upward, giving the shortest path and the lowest loss. Consider: rather than running a thick supply pipe from the next room, sink a well directly under the floor and draw straight up. This is a new link that emerged specifically to serve extreme-current GPUs.

8.16 HVDC Conversion Silicon (SST, 1200V wide-bandgap — rack-level high-voltage conversion)

HVDC is high-voltage direct current. AI rack power has climbed so high that hall distribution is migrating from low voltage to ±400V and even 800V DC (the same copper cross-section then carries far more power). That in turn requires conversion silicon able to operate at those voltages, typically built on the wide-bandgap (WBG) materials described above and rated to 1200V. SST stands for solid-state transformer — delivering the function of a conventional transformer in silicon.

8.17 eFuses, Hot-Swap & Load Switches (make-break protection silicon)

Servers have to support hot-swap — replacing a component without powering the machine down — or the hall cannot be maintained at all. But the instant of a live insertion produces an enormous inrush current that can destroy hardware. This class of chip is the smart fuse that controls that inrush gently on insertion and cuts off instantly on fault to protect what is downstream. An eFuse is an electronic fuse: faster to react than a conventional fuse, and resettable.

8.18 Gate Drivers & Isolation (digital isolators)

The power switches above — MOSFETs, GaN, SiC — have to be turned on and off with precision, and it is the gate driver that issues those commands. Between the high-voltage and low-voltage domains, a further electrical firewall is mandatory to stop high voltage crossing into the control circuitry and destroying both people and silicon. That is the job of the digital isolator: the signal passes, the dangerous voltage does not. Consider the armoured glass at a bank counter — speech goes through, danger does not.

8B.1 Capacitors

A capacitor is a miniature battery and voltage stabiliser in one. It can store a small charge instantaneously and release it just as fast, which lets it hold voltage steady and filter out noise when a chip's current draw swings sharply. An AI processor's draw swings violently, so a dense field of capacitors is laid out immediately around each die. There are many families: MLCC (multilayer ceramic capacitors, the highest volume by far), tantalum, aluminium electrolytic, and film. Consider a capacitor as a header tank: when the tap (the chip's draw) opens and closes abruptly, the tank keeps the pressure steady.

8B.2 Resistors & Shunts (current-sense resistors)

The resistor is the most elementary component of all, impeding current and dividing voltage. A shunt is a specialised precision resistor placed in series in the circuit: measuring the tiny voltage across it lets you infer how much current is flowing. That is how a data hall knows precisely how much power each chip is drawing at any moment. Consider a shunt as a water meter, computing flow from the pressure drop across a short section of pipe.

8B.3 Inductors & Chokes

An inductor is a wound coil, and its role complements the capacitor's: capacitors steady voltage, inductors steady current and smooth out its spikes. In the multiphase VRM power delivery described earlier, every phase needs its own inductor, so an AI motherboard carries a great many of them. A choke is a form of inductor dedicated to choking off unwanted high-frequency noise. AI's high currents have given rise to demanding new categories such as TLVR (a coupled-inductor topology).

8B.4 Magnetics & Transformers

A transformer uses the magnetic field between two coils to convert one voltage into another, stepping up or down and also isolating two circuits from each other (the armoured glass described in the isolation section is sometimes built this way). What is covered here is the small board- and module-level magnetics and transformers, plus their core raw material: ferrite cores (a magnetically permeable ceramic that forms the skeleton of the coil). AI's high-voltage DC architecture — the solid-state transformer, SST — is generating new demand for magnetics.

8B.5 Ferrites & EMI Filters

Any working circuit radiates electromagnetic noise (EMI, electromagnetic interference), which both disturbs other equipment and is subject to regulation. A ferrite bead is a small magnetic bead slipped over a signal line to absorb high-frequency noise; an EMI filter is a more complete suppression module. Consider them the silencer fitted to the circuit, damping the noise down.

8B.6 Board-Level Circuit Protection

Circuits fear two things: too much current (overcurrent) and a sudden voltage spike (a surge, from a lightning strike or static discharge). This family of components blocks both. Fuses melt open on overcurrent; TVS and ESD diodes absorb transient overvoltage and shield static-sensitive silicon; MOVs and GDTs handle heavier surges; and PTC/NTC devices limit the inrush at power-up. They are the lightning rod and airbag of the circuit board.

8B.7 Frequency Control Passives (crystals and resonators)

The clock chips described earlier cannot produce a precise beat without a reference tuning fork: the quartz crystal. Quartz vibrates at an exceptionally stable frequency when energised, supplying the whole system with its reference. Higher-end variants include TCXOs and OCXOs (temperature-compensated and oven-controlled crystals, far more accurate) and the emerging MEMS resonator (substituting a micro-mechanical structure for quartz). Consider the crystal as the orchestra's tuning fork: every instrument takes its pitch from it.

8B.8 Relays & Electromechanical Components

A relay is a mechanical switch that uses a small current to make or break a large one — there really is a piece of metal inside physically pulling in and releasing. A contactor is the higher-power version, used to break and make main circuits carrying hundreds of amps (both hall power distribution and backup-supply transfer rely on them). Unlike a purely electronic switch, it acts through genuine metal contacts, which is why it wins on large-current capability and complete isolation. AI halls' move to high-voltage DC has created a distinct HVDC contactor category.

8B.9 Board Connectors, Sockets & Cable

A connector is the plug-and-receptacle that mates two boards, or a cable to a board. A socket refers specifically to the seat that holds a CPU or memory module, letting the part be inserted and replaced rather than soldered permanently. Signal speeds inside an AI server are extreme and pin densities extraordinarily high, pushing connector precision requirements to the limit — which has, paradoxically, promoted the connector from an unremarkable small part into a link with a real technical barrier and an AI premium attached. Consider: a connector is a power strip, but what an AI hall needs is the top-specification strip that carries ultra-high-speed signals without a single error.

8B.10 Busbars & Board-Level Power Distribution Hardware

When current grows beyond what ordinary wire can carry, conduction shifts to thick slabs of metal — the busbar. What is covered here is the laminated busbar and copper conductor at board and module level (the rack- and hall-level busway belongs to the power layer). The thousand-amp currents an AI processor draws have to be delivered across the board by exactly this kind of copper. Consider a busbar as replacing many thin pipes with one large main, sized for high flow.

9.1 High Bandwidth Memory (HBM)

HBM stacks multiple memory dies vertically, like storeys of a building, then connects them into a single block through extremely fine vias (TSVs — through-silicon vias, the industry's term, functionally the lift shafts between floors). It sits immediately alongside the AI chip, delivering extraordinarily fast reads and writes and an extremely wide transfer bandwidth. Consider: ordinary memory is a single-lane road, HBM a motorway several dozen lanes wide — the only way a data torrent avoids gridlock. It is the most critical, and the most expensive, component attached to an AI chip today.

9.2 Server Main Memory / DDR5

DRAM (dynamic random-access memory) is the chip inside what people call a memory module — the primary workbench on which data is laid out while the processor computes. DDR5 is the current generation of the standard (fifth-generation double data rate), faster and more power-efficient than its predecessor. Beyond the HBM bonded to the accelerator, an AI server also carries a large complement of this conventional main memory, providing the machine's large working surface.

9.3 CXL Memory Expanders

CXL (Compute Express Link, a new channel standard for high-speed processor-to-memory links that also allows multiple processors to share one copy of the data) permits memory to be moved a little further from the chip and pooled for several processors to draw on. A memory expander is the interface card for that remote memory: when the slots beside the processor run out, it extends capacity further. Consider: instead of each desk carrying its own workbench, the office moves to a shared storage area anyone can draw from.

9.4 NAND Flash

NAND flash is the medium that holds data in the warehouse, retaining it without power. It is the core material in solid-state drives and USB drives. It is graded by storage density — the denser, the cheaper and the slower (TLC, QLC and so on in industry shorthand). An AI factory has to stockpile vast quantities of training data, and demand for this high-capacity medium rises accordingly.

9.5 Enterprise SSDs

An SSD (solid-state drive) assembles a set of NAND flash dies plus a controller into a drive that is purely electronic, with no mechanical platter. The enterprise version is the one built for data centres: more durable, more reliable and higher capacity than consumer parts. It reads and writes far faster than a mechanical drive, making it the main warehouse for hot data — the data pulled frequently — inside an AI factory.

9.6 SSD / Memory Controllers

Beyond the flash that holds the data, a solid-state drive contains a brain chip that directs which flash cell each piece of data goes to, corrects errors as cells fail, and manages endurance (that logic is known in the industry as the FTL, or flash translation layer). That brain is the controller. Without it, a pile of flash is just loose material and will not assemble into a usable drive.

9.7 Memory Modules — RDIMM / MRDIMM

A DIMM is the memory module that plugs into the server board. An RDIMM (registered DIMM) adds a buffer chip so a server can carry far more modules reliably and without error; the MRDIMM (multiplexed DIMM) is the newer generation, fitting greater capacity into the same slot and running faster. AI servers demand a great deal of memory, and volumes of these high-specification modules rise with it.

9.8 Module Companion Silicon

Beyond the memory dies, a module carries several inconspicuous but indispensable small chips: one buffers commands and marshals the signals (the RCD, or registering clock driver), another relays the data (the data buffer), and another handles temperature sensing and identity records (the SPD hub). Together they let a fully populated module run fast and stably.

9.9 Near-Memory / Processing-in-Memory (PIM)

Conventionally, data sits in memory and the processor fetches it to compute — and that round trip costs both power and time. Processing-in-memory (PIM) proposes doing the simpler computations inside the memory itself, saving the trip. Consider: rather than hauling all the goods back to head office to be counted, count them in the warehouse. This is an early-stage architecture aimed at routing around the data-movement bottleneck.

9.10 Emerging Non-Volatile Memory (NVM)

Non-volatile memory (NVM) means storage that retains data without power. This line item is the collective term for a group of technologies still between laboratory and low-volume production: MRAM (magnetic memory, recording ones and zeros in magnetic orientation), ReRAM (resistive memory, recording data as high or low resistance), PCM (phase-change memory) and FeRAM (ferroelectric memory). Each is trying to be both fast and non-volatile at once; all are currently confined to niche applications, and none is close to displacing mainstream memory at scale.

9.11 Nearline Hard Disk Drives (HDD)

An HDD is the traditional drive with spinning platters and read/write heads inside — slower than an SSD but far cheaper per gigabyte. “Nearline” denotes its use for warm and cold data: infrequently accessed, but occasionally needed. Most of the vast data AI training generates is not in daily use, and stockpiling it on cheap high-capacity drives is by far the most economical option. The latest generation uses laser-assisted writing (HAMR, in industry shorthand) to push capacity further.

9.12 Storage Systems & Arrays

Pack hundreds or thousands of drives into racks, add management software, and the result is a complete appliance providing storage as a service — the storage array. An all-flash array is the high-speed version built entirely from SSDs; parallel file storage is the specialised system that lets thousands of AI chips read the same body of data at high speed simultaneously. That is a hard requirement of AI training: hungry chips need to be fed, and if storage is slow, even the most expensive silicon sits idle.

9.13 HBM Base / Logic Die

Underneath the stack of memory dies in an HBM part sits a special foundation die, responsible for directing how the whole stack converses with the AI chip outside it. The newest generation (HBM4) makes that foundation progressively more complex — complex enough that it is now built on leading-edge logic processes. Which means even the foundation of a memory part now has to be fabricated by a top-tier foundry such as TSMC, binding memory and advanced logic manufacturing together.

9.14 DIMM Buffer Silicon — RCD / DB / MUX

This line item breaks out the most important of the companion chips in 9.8: the RCD (registering clock driver, marshalling and amplifying the command signals), the DB (data buffer, relaying and accelerating the data signals), and the MUX (multiplexer, pushing more data down a single line). The faster a module runs and the more densely it is populated, the more indispensable these signal-steadying chips become.

9.15 SOCAMM / LPDDR Modules for AI Servers

LPDDR was originally the low-power memory built for phones — energy-efficient and low-heat. To save power and space, AI servers have begun bringing that low-power memory in as purpose-built pluggable modules (SOCAMM and other new form factors). Consider: what used to be the large, power-hungry desktop-style module becomes a compact, efficient laptop-style one — and both power draw and density matter more to an AI rack with every generation.

9.16 Memory-Side Retimers

Once memory is moved away and pooled for sharing (see 9.3, CXL), signals travelling that distance attenuate and distort. A retimer is the relay station along the way: it re-reads the degraded electrical signal cleanly and retransmits it at full strength, so data can reach more distant memory without error. Consider passing a message down a long line — it is easy to mishear, so you station someone in the middle to repeat it clearly before it travels on.

10.1 Ethernet Switch Systems

A switch is a device dedicated to connecting and forwarding: data from a great many servers converges on it, and it decides where each packet goes next. Ethernet is the world's universal networking standard. An AI factory holds rows of switches weaving every chip and server into one high-speed fabric. The system-level business is assembling switch silicon, chassis and software into a working box sold to the customer.

10.2 Merchant Switch Silicon

At the core of that exchange sits the chip that determines how much data can be forwarded per second, and how — the switch ASIC. “Merchant” means it is sold as a standard part to equipment makers generally, rather than reserved for one company's own use. A leading-edge part forwards over 51.2 terabits per second (51.2T in industry shorthand), and it is the master switch on how fast the whole fabric runs.

10.3 Network Interface Cards (NIC)

The NIC is the part fitted to each server that lets it speak on the network; without one, a server is an island with no telephone. Chips in an AI cluster exchange data furiously, so the cards used are the high-speed versions. Among them, cards supporting RDMA (remote direct memory access, which lets one machine bypass the usual software path and read another machine's memory directly) cut transfer latency substantially, and are standard equipment for AI training.

10.4 SmartNICs / DPUs

An ordinary NIC only sends and receives. A SmartNIC, or DPU (data processing unit), adds a small processor onto the card that takes over networking, security and storage chores from the main processor, so the expensive AI silicon can concentrate on mathematics rather than housekeeping. Consider: give the senior executive a capable assistant to absorb the administrative work, leaving them free to do only the highest-value thing.

10.5 Retimers / Redrivers / Gearboxes

A high-speed electrical signal travelling any distance across a board attenuates and distorts. A retimer reads the signal out completely, regenerates a clean copy and retransmits it (a full refurbishment); a redriver simply amplifies what is there (a transfusion); and a gearbox converts between different data rates. All three are relay stations on high-speed links, and the higher the rate and the longer the distance, the more indispensable they become.

10.6 Active Electrical Cables (AEC)

An ordinary copper cable's signal collapses beyond a certain distance. An active electrical cable (active meaning it carries powered silicon of its own) embeds retimer chips at each end of the copper, cleaning the signal as it travels, so the cable reaches further while remaining cheaper and more power-efficient than optics. Consider adding a booster pump partway along a run of pipe so the water reaches further. It is the value choice for medium-distance connections inside the rack.

10.7 Passive Copper / DAC

A DAC (direct attach copper cable) is a pure-copper short-reach link with no silicon in it at all — “passive” meaning nothing inside requires power. It is the cheapest, most power-efficient and most reliable option, with the drawback that it only reaches a very short distance (a metre or two). Where chips sit close together inside a rack, copper is used wherever copper will do, because optics are both expensive and power-hungry.

10.8 High-Speed Connectors & Backplanes

A connector is the physical interface that mates two boards or two cables; a backplane is the large base board into which multiple circuit boards plug to interconnect. Data moves extraordinarily fast inside an AI rack, so precision requirements at these mating points are extreme — a small deviation and data is lost. They attract no attention, but every machine holds hundreds or thousands of them, and not one can be missing or defective.

10.9 PCIe Switch Fabric

PCIe (Peripheral Component Interconnect Express, the standard high-speed channel between a computer's CPU and its various cards) originally supported one device per link. A PCIe switch acts as a splitter, fanning one channel into many so a single CPU can host far more GPUs, drives and network cards. Consider: one socket is not enough, so you add a safe, high-speed strip that splits it into several. AI servers carry a great many devices and depend on this for expansion.

10.10 CXL Fabric Switches

As noted above, CXL is the new channel that lets processors and memory share one copy of the data. A CXL fabric switch is the exchange within that shared network, flexibly interconnecting multiple processors with a large pool of shared memory so any of them can attach when needed. It is the key link in turning memory into a public resource pool — still early, but widely regarded as the future direction of AI memory architecture.

10.11 Scale-Up Fabric — NVLink / UALink

“Scale-up” means binding dozens of GPUs within a single rack tightly enough that they behave as one giant chip, each able to reach the others' memory as if it were its own and work the same problem together. NVIDIA's NVLink is the de facto standard for this — extremely fast, and available from nobody else. UALink is the open alternative developed jointly by other vendors. Consider wiring a row of people's brains together neurally so that they think as one. This is the critical amplifier on an AI cluster's effective compute.

10.12 InfiniBand

InfiniBand is a networking technology purpose-built for high-performance computing, delivering lossless transport at very low latency: extremely fast, and essentially error-free. It is popular in AI training because when thousands of chips compute in lockstep, none can afford to fall behind because the network stalled. It comprises dedicated switches and network cards (HCAs, or host channel adapters, in industry shorthand). The technology is almost exclusively controlled by NVIDIA.

10.13 Network OS / Electrical DCI

This item covers the hardware side of two things. First, decoupling the switch's operating system from its hardware (letting the user swap network software as freely as installing a PC operating system — the open-source approach is known as SONiC). Second, the electrical transport equipment for data centre interconnect (DCI, linking two geographically distant data centres over high-speed lines). The larger AI gets, the more it needs several halls joined into one larger whole.

10.15 PCB Midplane / Backplane Assemblies

This is the high-speed base board inside the rack — the midplane or backplane — into which every compute board plugs to become fully interconnected, together with the complete any-to-any wiring it carries. Dozens of chips in an AI rack must each connect directly to every other at high speed, which places extreme demands on the board's layer count and precision. It is the physical substrate on which scale-up fabrics such as NVLink are actually realised.

10.17 Plastic Waveguide / e-Tube (emerging)

A new medium sitting between copper cable and optical fibre: a purpose-made plastic tube that guides a high-frequency signal the way a fibre guides light, aiming to combine copper's low cost with optics' reach and efficiency. It remains at an early, pre-productisation stage, pushed mainly by start-ups. Consider building the electrical signal a dedicated slide so that it travels further with less effort.

11.1 Pluggable Optical Transceivers

This is the mainstream form of that electro-optical translator: a small box that plugs into a switch port, integrating the laser, the silicon and the rest, converting electrical signal into transmitted light and received light back into electricity. “Pluggable” means it can be inserted and swapped like a USB stick. Generations are defined by rate (400G/800G/1.6T; higher is faster), and this is the highest-volume, most directly AI-levered product in the entire optical layer.

11.2 Optical Engines

An optical engine packages the optical and electrical halves of a transceiver into a more compact sub-assembly — the translator's core movement, pre-built and handed downstream for integration. It is the intermediate step from the pluggable module toward tighter integration sited closer to the chip, and it lays the groundwork for more advanced packaging such as CPO (next item).

11.3 Co-Packaged Optics (CPO)

Conventionally the optical module plugs in at the edge of the switch. CPO moves the optical engine into the same package as the switch silicon or the AI chip, sitting flush against it, so the optical path shrinks to its minimum, saving power and raising speed. Consider bringing the translator from the doorway to a seat at the negotiating table: less walking, less energy burnt. This is the next-generation direction the industry argues about most, and it could reshape the whole optics-and-switching structure — but volume production is still early.

11.4 Silicon Photonics / PIC

Silicon photonics uses mature chip-fabrication processes to build optical functions — modulating and guiding laser light — directly onto a die (the industry calls that die a PIC, or photonic integrated circuit). The advantage is that optical devices can then be made at scale and at low cost exactly as ordinary chips are. It is the underlying technology platform for CPO, high-end transceivers and other advanced optical approaches.

11.5 Lasers — EML / DFB / CW

The laser is the source of light inside a transceiver; without it there is no optical signal to send. This item covers the high-end laser classes used for long-reach, single-mode fibre: EML (electro-absorption modulated laser), DFB (distributed feedback laser) and CW (continuous wave). They are built on a specialised semiconductor material, indium phosphide (InP), the technical barrier is high, and they are among the most valuable core materials inside a transceiver.

11.6 Lasers — VCSEL

The VCSEL (vertical-cavity surface-emitting laser) is a different laser class that emits vertically from the die surface; it is cheap and easy to manufacture at volume, but suits only short-reach transmission. It is used mainly in multimode modules on intra-rack and other short links. Consider DFB/EML the finely engineered long-haul engine and the VCSEL the economy short-haul unit — each owning its own stretch of distance.

11.7 Optical DSP / PHY

An optical signal usually arrives distorted. The optical DSP (digital signal processor) is a chip dedicated to correcting and restoring it, spending compute to reconstruct the degraded waveform so that long-reach, high-rate transmission still arrives faithfully. It is the highest-value chip in a high-end transceiver after the laser itself. But the newer LPO approach (next item) aims to design it out, which introduces genuine uncertainty here.

11.8 Linear-Drive Optics (LPO)

LPO (linear pluggable optics) is a newer approach that dispenses with the power-hungry optical DSP, using simpler linear amplification to drive the optical devices directly — lower power, lower cost, lower latency, at the price of far more demanding system design. It is the direction AI clusters are embracing in the pursuit of power savings, and it is encroaching on the traditional optical DSP's territory.

11.9 External Laser Source (ELS)

In advanced approaches such as CPO and silicon photonics, a laser packaged alongside the chip radiates heat into it and is hard to repair when it fails. The external laser source moves the laser outside, standing alone, and pipes its light in over fibre — separating the thermal load and allowing the laser to be replaced on its own. Consider mounting the engine outside the car: easier to cool and easier to service.

11.10 Optical Connectors / Fibre / MPO

This item is the fibre itself plus the connectors that join it (the multi-fibre interface that terminates many strands at once is called MPO/MT). An AI hall holds thousands of fibre strands, and these connectors are what dock them tidily and reliably. Tolerances are tight: a slightly contaminated or misaligned interface and the light simply does not get through.

11.11 Passive Optical Components

“Passive” means these small devices draw no power and amplify nothing; they act on the light passively. They include isolators (letting light travel one way only, suppressing reflection), filters (selecting a specific wavelength), multiplexers and de-multiplexers (combining many optical channels onto one fibre or splitting them apart, known in the trade as WDM), and splitters. They are the inconspicuous but indispensable fittings of the optical path, combing, routing and cleaning up the light.

11.12 Optical Circuit Switching / Coherent DCI / Photonic Packaging

This item covers three high-end activities: optical circuit switching (OCS, switching the light directly without converting to electrical first — Google already runs it at scale in its own AI network); coherent DCI (using advanced modulation to push very-long-reach datacentre-to-datacentre optical transport to extremely high capacity); and dedicated foundry packaging for photonic devices. All three push optics toward greater scale and greater distance.

11.13 Electro-Optic Modulators — MZM / EAM / TFLN

The modulator is the device that inscribes the electrical signal onto the light. The laser only emits a steady beam; the modulator switches and varies that beam rapidly in step with the data, which is how information is loaded into the light. Several technology routes compete: silicon-based MZM, indium-phosphide EAM, and the higher-performing thin-film lithium niobate (TFLN). The higher the rate, the more punishing the demands on the modulator.

11.14 Receiver Photonics — PIN / Ge / APD / TIA

When the light reaches the far end, something has to turn it back into an electrical signal. Photodetectors (PIN, germanium detectors, APD avalanche photodiodes) convert faint light into a faint current; the TIA (transimpedance amplifier) sits immediately behind, amplifying that faint current into a usable signal. Together they form the receive side of the transceiver, bookending the laser on the transmit side.

11.15 In-Package Micro-Optics — FAU / Isolators

Approaches such as CPO and silicon photonics that move optics inside the chip package require a set of very small optical parts to dock fibre to die precisely: the FAU (fibre array unit, holding many fibres in fixed alignment to the chip), edge couplers, micro-lens arrays, in-package isolators and the like. What they solve is the exquisitely fine alignment problem of getting light accurately into and out of the photonic die.

11.16 Semiconductor Optical Amplifier (SOA)

Light weakens as it travels. The SOA (semiconductor optical amplifier) is a chip that amplifies light in place, topping it up in shared-laser and long-reach settings so it can travel further or be divided among more channels. It is the gain device underpinning external laser sources and long-reach architectures.

11.17 Hollow-Core Fibre (HCF)

Ordinary fibre is solid glass, and light travelling through glass is marginally slower and lossier. Hollow-core fibre confines the light mostly to a hollow air core, giving lower loss and higher speed (light moves faster in air than in glass). It is compelling in AI settings chasing the last increment of latency, but it remains a frontier speciality fibre made in small volumes.

12.1 AI Server ODM (Original Design Manufacturer)

The ODM (original design manufacturer) is the principal contract manufacturer on this chain: the customer — a cloud provider, say — states the requirement, and the ODM designs it, builds prototypes, then manufactures complete AI servers at scale, finally badging them for the customer or delivering them directly. The distinction from a branded vendor is that the ODM does not normally sell to the end user under its own name. Consider the contract factories of the apparel trade: the label supplies design and orders, the factory makes the garment. Sub-items here include AI server ODM work, hyperscaler reference-platform manufacturing (building to a standard design supplied by the hyperscaler), board-level integration services, and EMS contract manufacturing (electronics manufacturing services, pure build-to-print assembly).

12.2 Branded AI Servers (AI Server Branded OEM)

OEM (original equipment manufacturer) here means the server vendors that carry their own brand and sell complete solutions and services. Relative to an ODM they add brand, sales channel, enterprise-grade technical support and after-sales service; the customer buys not merely a machine but a warranty of continuity around it. Sub-items include branded AI server platforms, HPC-dedicated servers (high-performance computing) and full solutions with enterprise support. Consider buying the same server two ways: going to the OEM is like buying a branded computer with nationwide warranty cover, where somebody answers when it breaks. That is why they earn an additional layer of service and brand margin above pure contract manufacturing.

12.3 GPU Baseboard & Compute Tray

The baseboard is a large circuit board carrying several GPUs soldered or socketed together at once (NVIDIA's HGX eight-GPU baseboard, for instance), so that those GPUs are interconnected at high speed on the board itself. The compute tray then packs the baseboard, CPUs, memory, power delivery and cold plates into a drawer-style module that slides into the rack. Consider the baseboard a motherboard with eight brains plugged into it, and the tray the drawer holding that motherboard and sliding in and out as a unit. Sub-items include multi-GPU baseboards, accelerator compute trays, GPU carrier boards, accelerator switch trays, tray power distribution boards and tray management controller boards. This is the highest value-density assembly step in the entire machine, and the one closest to the silicon.

12.4 Rack-scale Integration

This is the genuinely new — and most critical — segment thrown up by the AI era. Servers used to be sold one box at a time and racked up on site; now, with something like NVIDIA's GB200 NVL72, 72 GPUs, switches, power delivery, liquid cooling and cabling are pre-integrated in the factory into a complete rack-level SKU (a product number ordered and delivered as a single unit), shipped to site and put to work as soon as power and water are connected. Consider the difference between buying a pile of flat-pack parts and assembling them at home, and buying a fully built, fully tested show flat and moving it wholesale. Sub-items include rack-level reference design integration, integrated compute rack SKUs and rack-level validation services. This segment consolidates scattered contract work into a single high-complexity, high-barrier integration business.

12.5 Thermal Fans & Heatsinks (in-system air cooling)

Even once liquid cooling is adopted, a great many components in the rack still rely on air, and this segment is the air-cooling hardware inside the machine. The heatsink is the block of metal seated on the chip, covered in fins to enlarge the surface area, from which a fan carries the heat away. Within it, the vapour chamber (an evacuated flat cavity that spreads heat rapidly by vaporising and condensing a liquid) and the heat pipe (a slender tube working on the same principle) are the efficient devices that move heat from the chip and spread it across the fins. Sub-items include server fan modules, heatsink fin stacks, vapour chambers, heat-pipe modules, air-baffle kits and fan speed control boards. Consider the vapour chamber a sheet of conductive iron that instantly smooths a single hot point across a whole surface.

12.6 Chassis, Enclosure & Mechanical

This is the skeleton and shell of the machine: the chassis, the metal box holding the motherboard and components; the rack frame, the large steel cabinet in which servers are stacked; plus sled rails (the runners that let a server slide out like a drawer), tool-less service mechanisms (removable by hand, without a screwdriver), EMI shielding (containing electrical noise that would disturb neighbouring equipment) and mounting frames for liquid-cooling manifolds. Consider these the shelving and drawer runners of the rack: unglamorous, but AI racks are heavy and densely packed, and poorly executed mechanical work leaves the whole rack unstable and awkward to service. Liquid-cooled AI racks demand far more of structural strength and load rating than traditional racks did.

12.7 Motherboard & PCB Integration

This segment is the precision assembly that solders chips and components onto the board. The PCB (printed circuit board) is the foundation slab carrying all the silicon. What makes an AI server hard is that signals run extremely fast, the board carries a great many layers (high-layer-count HDI, high density interconnect), and it requires backdrilling (removing the redundant portion of a plated through-hole so it does not distort the high-speed signal). Consider an ordinary motherboard a two-storey house and an AI motherboard a thirty-storey tower where the wiring on each floor must not interfere with any other; the yield and process barriers to building such a board are steep. Sub-items include server motherboard assembly, accelerator baseboard PCB assembly, high-layer-count HDI server PCBs, backdrilled high-speed PCBs, board-level BGA rework and functional board test.

12.8 Edge & On-prem AI Appliances

Not all AI runs in a large datacentre. Edge refers to small deployments close to the user, in the field; on-prem (on-premises) refers to equipment an enterprise deploys in its own machine room. This segment turns AI compute into self-contained, ready-to-run appliances with hardware and software included. Sub-items include edge AI inference appliances, ruggedised AI boxes (tolerant of heat, vibration and dust, for factory and outdoor use), enterprise on-prem AI appliances, micro-datacentre appliances and telecom edge AI servers. Consider compressing a small datacentre into a single rack or even a single box, carried to the factory floor, the retail store or the base station and put straight to work — the data never travels back to the cloud, so response is faster and privacy better preserved.

12.9 System Test & Burn-in

An assembled machine cannot simply be shipped; it must first be tortured to make sure it does not fail on arrival. Burn-in runs the machine at high load and high temperature for a long stretch, forcing the units prone to early-life failure to reveal themselves and be culled. Consider the extreme road test a new car undergoes before it leaves the plant: far better it break in the factory than strand the customer on the motorway. Sub-items include rack-level functional test, GPU cluster burn-in, thermal soak system validation (verifying cooling and stability under sustained heat), interconnect validation and acceptance-test documentation services. An AI rack is worth several million dollars, and a failure on the customer's floor means enormous downtime cost — which is why this gate matters more and more.

12.10 Internal Cable Harness & Wire Management

An AI rack contains hundreds or thousands of individual runs: power, high-speed signal, fibre, sensor wiring. This segment builds those runs into orderly harnesses (a pre-bundled, pre-terminated loom) and manages how they are routed. Sub-items include internal power harnesses, high-speed signal harnesses, liquid-cooling sensor harnesses, cable management arm assemblies (the articulated arm that lets cabling extend with a server pulled out of the rack without tearing), cable labelling and traceability kits (every run tagged for maintenance) and in-rack fibre harnesses. Consider several hundred cables left in a tangle: the cooling airflow is blocked and no technician can tell one run from another. How well the cabling is dressed bears directly on both thermal performance and serviceability.

12.11 Copper Cable Cartridge & Backplane Spine

In a rack such as NVIDIA's NVL72, all 72 GPUs must be fully interconnected (each able to reach every other directly), and that is done with a dense set of copper cables. This copper assembly is built into a cartridge or spine — a copper backplane running up the back of the rack like a vertebral column — and plugged in at the rear. Consider it the rack's central nerve trunk, joining every GPU behind the machine over the shortest, fastest copper path into a single super-unit. Sub-items include passive copper expansion cartridges (no active silicon, pure copper conductors), rack spine copper cable assemblies and blind-mate copper cartridges. This is the physical realisation of in-rack high-speed interconnect such as NVLink, and the signature new component of the AI rack.

12.12 Fan Wall & Sidecar Mechanical

High-power racks frequently add a sidecar — an auxiliary cabinet hung beside or behind the main rack — specifically to house fan walls, power or cooling modules, because the main rack is already full. The fan wall is an entire wall of fans forcing or drawing air through the rack as a unit. Sub-items include rack fan-wall sidecars, power sidecar mechanical frames, cooling sidecar sheet metal, serviceable filter-fan modules and sidecar airflow plenums. Consider adding an outbuilding beside a tower block to house the large fans and power gear the main structure cannot accommodate. AI rack power is high enough that the sidecar has become the standard way of absorbing the extra cooling and power.

12.13 Rack-scale Final Assembly, Burn-in & Service

This is the last mile of the entire chain: assembling every part and subsystem into a complete rack, running full-rack test and burn-in, and shipping it to the customer's site. For liquid-cooled racks there is one further critical operation — the leak test, pressurising the whole rack's cooling loop and confirming that not a drop escapes before it can ship. Sub-items include full-rack assembly services, liquid-cooled rack leak testing, rack-level burn-in services and rack packing and logistics. Consider the final inspection and hand-over of a fully fitted apartment; with a liquid-cooled rack in particular, a failed leak test means several million dollars of silicon could be destroyed the moment water is introduced. This segment is what turns a rack from a heap of parts into a finished product ready to be energised.

13.1 Pure-play Foundry

A pure-play foundry builds chips for others and sells no chips under its own brand. Consider a world-class printing house: you (NVIDIA) bring the artwork, it prints the physical object using the most exacting process available, puts your imprint on it, and never competes with you in selling the product. A wafer is a round disc of silicon (commonly 30 cm across) on which hundreds or thousands of chips are built at once. Pure-play foundry is the highest-barrier form of this business, because only one or two companies worldwide can reliably run the most advanced nodes (3 nm, 2 nm) at volume.

13.2 IDM Foundry

An IDM (integrated device manufacturer) is the all-in-one company that designs its own chips, owns the fabs that build them, and sells them itself. Consider that a pure-play foundry runs only the print works, whereas the IDM writes the book, prints it and sells it. IDM foundry refers to such a company renting out the fab capacity it has left over once its own needs are met.

13.3 Specialty / Mature Foundry

Not every chip needs the leading edge. Many chips — power management, analog, display drivers — are cheaper and more reliable when built on a slightly older, thoroughly proven node, and that is mature-node foundry. Specialty process means a process tailored to a particular class of chip (a line built specifically for analog mixed-signal work, or for MEMS microelectromechanical sensors). Consider the niche printer specialising in one kind of paper and one kind of binding: it does not chase the newest equipment, but within its own corner nobody does the job better.

13.4 Memory Fab

These are the fabs dedicated to memory, covering the two main classes: DRAM (dynamic random access memory, the computer's working desktop, wiped when power is cut) and NAND (flash, the warehouse behind a solid-state drive, which retains data without power). AI training is voracious in memory, above all the HBM high-bandwidth memory discussed later, whose substrate comes out of a DRAM fab. Consider that if the foundry builds the brain, the memory fab builds the memory.

13.5 Compound / Photonic Specialty Process

Everything above concerns chips built in silicon, but silicon is not good enough everywhere. High-voltage, high-power conversion, for instance, calls for wide-bandgap materials such as silicon carbide (SiC) and gallium nitride (GaN), which withstand high voltage at high efficiency — precisely what the rack-level high-voltage DC discussed in the electrical layer requires. The lasers in an optical module that turn electrical signal into light, similarly, must be built in compound semiconductors such as indium phosphide (InP) and gallium arsenide (GaAs). And then there is silicon photonics, which etches the optical path directly into the silicon. This item is the specialised fabs that build these non-silicon and photonic devices.

13.6 Photomask Manufacturing

Chips are built by lithography, printing the circuit pattern onto the wafer with light (covered in detail in the next layer). The photomask is the master plate carrying that circuit pattern: light passes through it onto the wafer and the pattern is reproduced. Consider the engraved face of a rubber stamp — the lithography tool is the stamping machine, the mask is the engraved face itself, and without that plate even the most expensive scanner prints nothing at all. The most advanced EUV (extreme ultraviolet) masks are exceptionally difficult to make, and the number of players worldwide can be counted on one hand.

13.7 Conventional Assembly & Test (OSAT — wirebond / flip-chip)

A finished die on the wafer is still a bare, fragile sliver with no way to be wired up; it has to be given a coat and pins before it can be used, and that step is packaging. Testing it afterwards makes the pair assembly and test. The outsourced factories that take on this work are called OSATs (outsourced semiconductor assembly and test). Wirebonding connects the die's pads to the package pins with extremely fine gold or copper wire — the most traditional method; flip-chip turns the die over and attaches it to the substrate through a grid of small solder bumps, giving denser and faster connections. Consider packaging to be dressing the chip in a coat with a great many pins, and the OSAT to be the tailor and the inspection house combined.

13.8 Wafer-Level / Fan-Out Packaging

Conventional packaging cuts the dies out first and packages them one at a time; wafer-level packaging inverts that, packaging every chip at once while they are all still attached to the wafer — more efficient and thinner. Fan-out is one technique within it: when the die itself is too small to accommodate all the pins, an extra ring of area is extended around the die for re-routing and additional connections, the way a canopy is stretched out beyond the ribs of an umbrella. Consider conventional packaging as sewing one coat at a time by hand, and fan-out as cutting and sewing in batch from a full bolt of cloth — with the option of broadening the shoulders on the smaller frames.

13.9 Advanced Packaging — 2.5D/3D

This is the single most critical and most constrained link of the AI era, and it deserves a full explanation. Transistors are approaching the limit of how much smaller they can be made, so the industry has changed tack: rather than making one chip bigger, it packs several dies (a compute die plus a ring of HBM memory) tightly together to work as one. How is that done?

2.5D: place a slab of silicon called an interposer beneath the several dies, pre-etched with extremely dense wiring, so that dies set on top can communicate at high speed. Consider seating several specialists at one table already laced with cabling, so that no conversation has to take the long way round. TSMC's signature process CoWoS (chip-on-wafer-on-substrate) belongs to this class, and it is currently one of the largest capacity bottlenecks for AI silicon — though TSMC's expansion does not map one-for-one onto NVIDIA's shipments, since HBM, substrates, yield, test, networking and power delivery are all constraining at the same time.

3D: stack the dies vertically outright. TSMC's SoIC (system on integrated chips) and Intel's Foveros both belong here.

Hybrid bonding: weld the copper pads of two dies face to face directly, with no adhesive at all — extremely high density, and the leading-edge technique for 3D stacking.

Consider that if conventional packaging dresses each chip in its own coat, advanced packaging zips a whole team into a single set of overalls so that they work shoulder to shoulder with no lag between them.

13.10 System-in-Package (SiP, Heterogeneous Integration)

SiP (system-in-package) puts an entire set of differently specialised chips (processor, memory, power, RF and so on) into one housing, so that the small box is itself a complete subsystem. Heterogeneous integration means mixing chips of different processes and different kinds into that one assembly. Consider that advanced packaging joins several compute specialists together, whereas SiP sweeps the whole project team — compute, memory, power, signalling — into a single box.

13.11 Panel-Level Packaging / CoPoS

The packaging described so far is mostly performed on a round wafer, and the round edge wastes a good deal of area. Panel-level packaging switches to a large rectangular panel — like replacing a round pizza with a rectangular tray — fitting more units per batch, wasting less material and lowering cost, which suits large and expensive AI packages particularly well. CoPoS (chip-on-panel-on-substrate) is the effort to carry the CoWoS approach across onto the rectangular panel. Consider swapping a round baking tin for a rectangular one: more loaves per bake, nothing lost at the corners.

13.12 Known-Good-Die / HBM Stack Test (KGD)

Advanced packaging is expensive, and once several dies are joined together a single bad one writes off the entire costly package. Every die must therefore be confirmed good before assembly, and a die verified in advance in this way is a KGD (known-good die). HBM memory is several layers of DRAM stacked up, and each added layer — plus the finished stack — must be tested, which is HBM stack test. Consider authenticating every piece individually before assembling a limited-edition Lego set, rather than discovering one defective brick at the end and writing off the whole box.

14.1 Lithography (EUV/DUV)

Lithography is the most central and most expensive step in building a chip: light is passed through the photomask described above to print the circuit pattern onto a wafer coated in photosensitive material. The lithography scanner is the machine that does this, and it is the most expensive and most critical tool in the entire plant. Consider an exquisitely precise stamping machine: the mask is the engraved face, the wafer the paper, and it must print billions of features thousands of times finer than a human hair within an area the size of a fingernail. The shorter the wavelength, the finer the pattern it can print: DUV (deep ultraviolet) is the older but still dominant tier; EUV (extreme ultraviolet, shorter still) is the only route to the most advanced chips below 5 nm, and exactly one company in the world can build an EUV scanner.

14.2 Etch

Lithography only prints the pattern; etch is what actually removes the unwanted material along that pattern, carving the real trenches and structures. It divides into dry etch (bombarding the material away with a plasma, by far the most common) and wet etch (dissolving it chemically). Consider lithography as drawing the lines on a board and etch as cutting away along them, turning a flat plate into a three-dimensional structure. AI chips carry very many layers and very deep features, and etch precision determines the outcome directly.

14.3 Deposition (CVD/PVD/ALD/Epitaxy)

A chip is built up layer upon layer, and each structural layer begins with laying an extremely thin film of material onto the wafer — the deposition step. Several methods exist: CVD (chemical vapour deposition, reacting a gas at the wafer surface to grow a film); PVD (physical vapour deposition, vaporising a metal and condensing it onto the wafer, commonly for metal layers); ALD (atomic layer deposition, laying down a single atom's thickness at a time for the finest layers); and epitaxy (growing a high-quality crystalline film whose lattice matches the material beneath). Consider deposition as painting the wafer: CVD grows the film through a chemical reaction, PVD sprays it on, ALD applies it atom by atom with the finest possible brush — and etch then carves away the paint that is not wanted, the cycle repeating hundreds of times to build a chip.

14.4 Chemical Mechanical Planarisation (CMP)

Every few layers the surface of the chip becomes uneven, and it must be flattened before building can continue, or the subsequent lithography will not hold focus. CMP (chemical mechanical planarisation) softens the surface with a chemical slurry while physically polishing it with a pad, leaving the wafer as flat as a mirror. Consider levelling each floor of a building before laying the next course of bricks — CMP is the machine that pours the softening agent and grinds at the same time.

14.5 Ion Implantation

Pure silicon conducts poorly; turning it into a useful transistor requires doping — introducing small quantities of other elements to alter its conductivity. Ion implantation accelerates those dopant atoms into high-speed ions and fires them precisely into specific regions of the wafer, as at a target. Consider injecting different fillings at defined points in a piece of dough so that different parts take on different flavours — different conductivity — which is exactly how the switching behaviour of a transistor is tuned.

14.6 Thermal Processing (RTP / Diffusion Anneal)

After implant or deposition the wafer usually has to be heated to a high temperature and annealed, settling the newly implanted atoms into their correct positions, repairing the damage the bombardment caused, or growing the material into the desired structure. RTP (rapid thermal processing) ramps up and down within seconds; a diffusion furnace instead treats the wafer slowly over a long period at high temperature. Consider firing pottery in a kiln, where heat and time decide the hardness and the quality of the finished piece.

14.7 Cleaning / Surface Prep

Chips are built at nanometre scale, where a speck of dust smaller than a virus landing on the wafer can destroy the die outright, so cleaning is repeated before and after every process step. Cleaning tools flush particles and contaminants off the wafer surface with ultrapure water and chemicals. Consider that every stage of surgery is preceded by thorough sterilisation; every stage of chip manufacturing likewise washes the wafer spotless, coming and going.

14.8 Metrology / Inspection

Chips are built at an invisible nanometre scale, so dimensions must be measured and defects found continuously during manufacture, or an entire lot can be scrapped with no idea where it went wrong. Metrology measures dimensional data — line width, film thickness, whether the overlay is accurately aligned. Inspection finds particles, defects and flaws on the surface. Consider the quality inspector with a microscope at the end of the line, measuring whether each part is within tolerance while picking out the defective ones under magnification. This is the eye of yield.

14.9 Automated Test Equipment (ATE)

Once a chip is built and packaged, every unit must be powered up and exercised to check that it functions correctly and meets its performance specification. The machine that gives the chip this electrical examination is ATE (automated test equipment). It applies a range of electrical signals, reads the results back, and judges whether the part is good and how fast it can run. Consider the full medical examination every chip receives before it leaves the factory; AI chips are large and complex, so testing takes longer and demands more of the tester.

14.10 Burn-in

Some chips are good when they leave the line but fail soon after use, and these infant-mortality units must be caught before shipment. Burn-in subjects the chip to prolonged punishment at elevated temperature and voltage, forcing those destined to fail early to fail now so they can be screened out, leaving only the genuinely durable parts. Consider running a new car several thousand kilometres in extreme conditions before it leaves the plant, surfacing latent faults there rather than in the customer's hands.

14.11 Handlers / Probers

Testing requires a machine to deliver each chip precisely to the test position and then sort the tested parts into bins by result; that machine is the handler. Where the dies must be tested while still attached to the wafer, before dicing, a prober is needed to land hundreds or thousands of very fine needles simultaneously and accurately on the pads of each die to power it up and test it. Consider the handler the conveying and sorting arm on the test line, and the prober the precision hand able to hold a thousand pins down at once.

14.12 Assembly / Packaging Equipment

This is the equipment used by the packaging houses of Layer 13, including the dicing saw that cuts a wafer into individual dies, the die attach tool that bonds the die to the substrate, the wire bonder that connects the pins with fine wire, and the moulding machine that encapsulates the chip in protective compound. Consider that if the packaging house is the tailor dressing the chip in its coat, these are the shears, the sewing machine and the press.

14.13 Sub-fab Abatement / Gas Delivery

Chipmaking consumes large volumes of toxic and flammable speciality gases and chemicals, and generates hazardous exhaust, so the level beneath the cleanroom (the sub-fab) houses a complete support system: vacuum pumps (evacuating the process chambers), abatement (scrubbing hazardous exhaust to specification before release) and gas and chemical delivery cabinets (routing speciality gases safely and precisely to every tool). Consider the cleanroom the operating theatre and the sub-fab the unglamorous supply, extraction and waste system one floor below, which must never stop.

14.14 Test Sockets / Contactors / Probe Cards

Test equipment is expensive and general-purpose, but every chip has its pads in a different place, so an interface part is needed to connect the chip accurately to the tester: the probe card (the board covered in needles that lands on dies on the wafer) and the test socket (the carrier into which a packaged chip is inserted for test) are these contactors. They wear out and must be replaced on a schedule — they are consumables. Consider the tester as the diagnostic machine and the probe cards and sockets as the disposable needles and interfaces: the machine is used over and over, the contacts must be changed once they have made enough contact.

14.15 Hybrid Bonders for Advanced Packaging

The hybrid bonding described under advanced packaging — welding the copper pads of two dies face to face with no adhesive — depends on this exceptionally precise bonder, which must align two dies to within a fraction of a hair's breadth before pressing them together. It comes in wafer-to-wafer (W2W) and die-to-wafer (D2W) forms. Consider aligning two films printed with circuitry to nanometre accuracy and bonding them seamlessly: a hair out of true and the whole chip is scrap. This is the most advanced tool in advanced packaging.

14.16 Temporary Bond/Debond & Wafer Thinning

Advanced packaging frequently requires the wafer to be ground extremely thin (so that many layers can be stacked), but a wafer that thin shatters at a touch, so before processing it is temporarily bonded to a sturdy carrier for protection and released afterwards — temporary bond and debond. Grinding the wafer down is called backgrinding. Consider mounting a very thin sheet of paper on card before cutting it, then peeling it away afterwards, so that the paper does not tear the moment it moves.

14.17 Advanced Package 3D Metrology / Inspection

Advanced packaging stacks dies and buries them inside the package, where an ordinary microscope cannot see, so dedicated techniques are needed to check for internal defects: X-ray to see through to the internal solder joints, ultrasonic inspection to listen for internal voids, and metrology of the bonded interface for warpage and gaps. Consider giving the stacked chip sandwich a CT scan and an ultrasound: every layer can be examined without cutting it open.

14.18 Actinic EUV Mask / Blank Inspection

EUV, as noted, is the crux of the leading-edge node — and a single minuscule defect on the EUV photomask is copied onto every wafer, generating scrap in bulk. The difficulty is that inspecting an EUV mask is best done by illuminating it at the same wavelength EUV itself uses (actinic inspection), and essentially one company in the world can build such a tool. Consider the dedicated medical examination given to the most expensive master plate of all — where the examination machine is itself a global monopoly.

14.19 Field Service / Spares / Refurbishment

Machines worth tens of millions of dollars are not a one-off transaction: they run for years, require scheduled maintenance and replacement spares, need emergency repair on site when they fail, and can be refurbished and resold when old. Equipment vendors therefore carry a large, stable after-sales revenue stream. Consider high-end medical equipment, where the enduring business is not the sale of the CT scanner but the years of maintenance contracts, parts and upgrades that follow. This business is increasingly paired with AI defect-analysis software, which makes it more valuable still.

15.1 Silicon Wafers (Wafer Substrates)

The silicon wafer is the foundation on which a chip is built. High-purity silicon is melted, drawn into a long round single-crystal ingot, then sliced like a sausage into discs and polished to a mirror flatness. Every chip circuit is ultimately drawn onto that wafer, layer by layer. Consider the wafer the levelled plot of land a building sits on: if the ground is uneven or dirty, whatever is built on it comes out crooked. Several types are distinguished within this group: prime polished wafers (the basic mirror-finished wafer), epitaxial wafers (a further, more perfect layer of thin silicon crystal grown on the wafer, giving better electrical performance), silicon-on-insulator (SOI, with an insulating layer sandwiched inside the wafer for lower power and better noise immunity, used in high-end chips), float-zone wafers (extremely high-purity silicon for power devices) and reclaimed monitor wafers (the pacing wafers used to check tool condition, which can be reclaimed and reused).

15.2 Photoresist (including EUV Resist)

Photoresist is a specialised coating whose properties change on exposure to light. In manufacture, a layer of resist is spread evenly over the wafer, light is shone through the mask (the plate bearing the circuit pattern), and the exposed and unexposed regions end up chemically different; a wash then leaves the circuit pattern on the wafer. Consider photographic film: expose it and the image appears. The shorter the wavelength, the finer the line that can be drawn and the more advanced the chip, so resists are generationally defined by the light they serve: i-line / KrF (krypton fluoride, the older generation, drawing coarse lines), ArF dry / ArF immersion (argon fluoride, immersion adding a layer of water between lens and wafer to draw finer lines, and the mainstream for advanced nodes) and electron-beam resist (written with an electron beam rather than light, used for masks and ultra-fine patterning). The most advanced EUV resist (extreme ultraviolet, of very short wavelength, indispensable for the most advanced AI chips) is broken out separately as item 19.

15.3 Resist Ancillaries (Companion Chemicals)

Resist is not simply applied and exposed; a full set of companion chemistries is required around it. This group is those companions: developer (the liquid that washes the pattern out after exposure), anti-reflective coatings (ARC / BARC, bottom anti-reflective coating, laid beneath the resist to stop light reflecting back and blurring the pattern), topcoat (laid over the resist as protection, keeping water out during immersion lithography), adhesion promoter (so the resist grips the wafer and does not lift), edge-bead remover (washing away surplus resist at the wafer edge) and resist strip (the chemistry that cleans residual resist off once the pattern has served its purpose). Consider the resist the main dish and this group the oil, salt and seasoning: without any one of them the dish cannot be made.

15.4 Photomask Blanks

Lithography, as described, uses a plate bearing the circuit pattern; that plate is the photomask (or reticle). The mask blank is that plate before any pattern has been written: an exceptionally flat, pure quartz glass substrate with an opaque metal film already deposited, awaiting the circuit pattern from the downstream step. Consider the blank a sheet of premium photographic paper before printing: a fractional shortfall in quality and the entire batch of chips printed from it is scrap. The group divides into DUV mask blanks (for deep ultraviolet, the mainstream), EUV multilayer mask blanks (for extreme ultraviolet, requiring dozens of nanometre-scale films to be deposited, which very few in the world can do — a genuine chokepoint), phase-shift blanks (using optical interference to render a crisper pattern) and pellicles (an ultra-thin transparent membrane covering the mask to keep dust off the plate).

15.5 Wet Chemicals

Through the course of manufacture the wafer is repeatedly cleaned, etched and rinsed, and the liquid chemistries used are collectively the wet chemicals. How pure must they be? Impurities are controlled to parts per trillion, because a trace of metal contamination can destroy an entire wafer. Consider preparing an infant's formula in which one unwanted molecule in the water is one too many. The group divides by use: electronic-grade hydrofluoric acid (the strong acid that etches silicon dioxide, with the most exacting purity requirement of all), sulphuric acid / hydrogen peroxide clean (SPM, removing organic contamination), hydrochloric, nitric-phosphoric and ammonia-based cleans and etches (each removing a different contaminant or etching a different material), semiconductor solvents (for dissolution and cleaning) and metal etchants (removing surplus metal layers precisely).

15.6 Specialty Electronic Gases

Many steps in chip manufacture take place inside a sealed vacuum chamber, accomplished by flowing in particular gases — to grow a film on the wafer, say, or to etch material away. These ultra-high-purity gases are the speciality electronic gases. Consider the gas supply in a kitchen: invisible, indispensable, and without it nothing on the stove can be lit at all. The group divides into deposition gases (growing films), etch gases (etching precisely), chamber cleaning gases (clearing residue from the tool chamber), dopant gases (introducing trace impurities into silicon to alter conductivity), lithography laser gases (feeding the scanner's laser) and bulk ultrahigh-purity gases (nitrogen, hydrogen, argon and helium, consumed in very large volumes).

15.7 CMP Slurries & Pads

A chip is built up layer by layer, and each layer must be polished absolutely flat before the next can go on; that flattening step is CMP (chemical-mechanical planarisation). It etches chemically while abrading mechanically, polishing the surface to atomic-scale flatness by both means at once. The consumables here are: slurry (a suspension of extremely fine abrasive particles, in silica-based, ceria-based, metal and silicon-carbide variants for different materials), the polishing pad (the pad that carries the slurry and rotates against the wafer) and the conditioning disk (the tool that dresses the pad itself to maintain its roughness). Consider the slurry the fine grit used to mill soya, the pad the millstone, and the conditioner the tool that re-cuts the grooves in that stone.

15.8 Sputtering Targets (PVD Metal Targets)

A chip carries many layers of metal interconnect (copper, tungsten and the like) — so how is the metal deposited onto the wafer? One method is sputtering: a disc of high-purity metal serves as the target, ions bombard it, and the metal atoms knocked loose travel across and deposit on the wafer layer by layer. That bombarded disc is the sputtering target. Consider a bar of high-purity metal soap, shaved away a little at a time and spread onto the chip. Targets are classified by metal: copper and aluminium targets (main interconnect), titanium and tantalum barrier targets (laid between copper interconnect and silicon to stop copper diffusing and contaminating it), tungsten and molybdenum targets, cobalt and ruthenium advanced-interconnect targets (the new metals replacing copper at the leading edge), magnetic alloy targets (for memory) and transparent conductive oxide targets (for displays and speciality devices).

15.9 ABF Package Substrates (FCBGA Substrates)

A finished chip is a bare sliver of silicon that cannot be soldered to a circuit board directly; it has to be mounted on a base first, and that base is the package substrate. It works like a precisely engineered interchange, fanning the thousands of minute contacts on the die out layer by layer into pins that can be soldered to the motherboard. The larger and more complex the AI chip, the more demanding the substrate — which makes this a genuine bottleneck in AI packaging. At the core of the group is ABF buildup film: high-end substrates are built by stacking insulating films, and the critical one is ABF, the overwhelming majority of which is supplied worldwide by a single Japanese company (which reports its own share at above ninety per cent and is the de facto industry standard). Consider that ABF film is a proprietary sauce supplied almost exclusively by Ajinomoto, the monosodium glutamate maker, from whom essentially every high-end substrate manufacturer in the world must buy — the improbable story of a seasoning company monopolising a chip material. The group also includes organic FCBGA substrates (the mainstream high-end substrate), coreless substrates (thinner), BT resin substrates and embedded bridge substrates (such as Intel's EMIB, burying a small silicon bridge inside the substrate to link two dies).

15.10 Advanced High-Layer PCB

The PCB (printed circuit board) is what is colloquially called the motherboard or circuit board — the green board onto which chips and components are soldered and joined by copper traces. The signals in an AI server are numerous and very fast, calling not for an ordinary PCB but for an advanced board with a great many layers (dozens of copper layers stacked) and very low signal loss. The group includes high-layer-count laminate, low-loss laminate (letting high-frequency signals travel far without attenuating), HDI (high-density interconnect, with extremely fine and dense traces) and copper-clad laminate (CCL, the base raw material of a PCB — an insulating board faced with copper foil). Consider CCL the cloth from which a PCB is made and the advanced PCB the tailored suit cut from it; AI is making the tailoring specification steadily more exacting.

15.11 Lead Frames

The lead frame is a more traditional packaging skeleton: a thin metal frame, stamped or etched, holding the die at its centre with metal leads extending around it; the die is connected to those leads by fine wire, and the leads emerge outside the package to be soldered to the circuit board. Consider it a metal picture frame for the chip, with legs. The group divides into stamped lead frames (punched at high speed from a die, cheap and high-volume), etched lead frames (chemically etched to a finer geometry, for high-end use) and power package lead frames (thicker frames with better heat dissipation for power devices). It is used mainly on relatively mature chips with modest pin counts — rarely on AI main processors, but extensively in power and discrete devices.

15.12 Bonding Wire

The fine metal wire mentioned above, connecting the die to the leads, is the bonding wire. It is very much finer than a human hair (micron-scale), and a dedicated machine welds it on one strand at a time to join the pads on the die to the pins on the package. Consider it the finest wiring in the world of chips; a single package may contain hundreds or thousands of strands. By material: gold wire (the most traditional and most reliable, but expensive), copper wire (cheap, highly conductive, now the mainstream), palladium-coated copper wire (copper plated with palladium to resist oxidation, more stable) and aluminium wire (common in power devices).

15.13 Die Attach Materials

Fixing the bare die onto the package base requires a chip-specific adhesive or solder — the die attach material. It must not merely hold the die firmly but usually also conduct heat away from it and carry current through it. Consider the specialised thermally conductive adhesive used to bond a hot ceramic tile to its base. The group divides into die-attach paste (in paste form), die-attach film (a thin film, used for stacked die), solder die attach (fixing with a solder alloy, with good thermal conduction), sintered metal die attach (fixing with sintered silver or copper particles, with excellent thermal conduction, for power and high-end chips) and temporary die attach (holding the die during processing and peeling away afterwards).

15.14 Mould Compounds & Encapsulants

Once the die is attached and wired, it needs a hard shell to protect it from dust, moisture and knocks, and the raw material of that shell is the mould compound (principally epoxy resin). It softens on heating, flows into the mould around the die, and cures into a hard casing. Consider a black epoxy hard-candy shell poured over the chip and set — the black chip package we all recognise is essentially this. The group divides into epoxy mould compound (EMC, the mainstream black casing), liquid encapsulant (poured in to fill gaps), mould underfill (moulding and filling the gap beneath the die in one operation), glob-top (a bead of resin dropped over the die) and wafer-level mould compound (for moulding an entire wafer at once).

15.15 Thermal Interface Materials (TIM)

Heat generated by the die has to reach the heatsink or lid to be shed, but two solid surfaces pressed together always leave microscopic gaps, and the air in those gaps insulates and blocks the heat. Thermal interface material fills that gap, displaces the air and lets heat pass through cleanly. Consider the layer of thermal paste between chip and heatsink: without it, the chip runs a fever under a blanket. AI silicon generates startling amounts of heat, and the quality of the TIM determines directly whether the chip overheats and throttles, which makes it a critical link in AI thermal management. The group divides into thermal grease (in paste form), phase-change materials (softening under heat to conform better), gap pads (soft pads filling larger gaps), thermally conductive adhesives (conducting heat while bonding) and underfill (filling the gap between die and substrate and improving reliability).

15.16 Ceramic & Glass Package Substrates

Besides the organic (plastic-family) substrates described above, package substrates are also made from ceramic or glass, which withstand higher temperatures and higher frequencies and are more stable, for power, RF and high-reliability applications. The group divides into alumina ceramic packages (the familiar white ceramic base), aluminium nitride ceramic packages (AlN, with notably good thermal conduction, for power devices), LTCC / HTCC substrates (low- and high-temperature co-fired ceramic, firing multiple ceramic layers and their circuitry into one high-density body), glass-core package substrates (using glass as the core for flatness and signal quality, regarded as the next-generation direction for large AI chip substrates) and through-glass-via interposers (drilling glass for vertical interconnect). Consider the organic substrate a plastic chopping board and the ceramic or glass substrate a fine ceramic or toughened-glass one: harder and more durable, but dearer.

15.17 Silicon Interposer & RDL Buildup Dielectric

The most advanced AI packages — a GPU sitting alongside HBM memory, for instance — place several dies side by side on a silicon intermediary board and connect them through the extremely fine wiring on that board. This is the silicon interposer, and it is the core of advanced packaging such as TSMC's CoWoS. The group includes silicon interposer wafers (the large silicon disc that serves as the intermediary), silicon bridge dies (a small silicon bridge spanning only the seam between two dies, cheaper than a full interposer), RDL dielectric (redistribution layer dielectric, the insulating film used to draw an extra layer of wiring on the die surface, in photosensitive and non-photosensitive forms), package buildup dielectric (Ajinomoto's ABF and its like again) and temporary carrier materials (supporting ultra-thin structures during processing and released afterwards). Consider the silicon interposer the foundation of an exquisitely precise miniature interchange built for several chips at once.

15.18 Photosensitive RDL Dielectric (PSPI, PBO)

Among the redistribution-layer dielectrics of the previous group, one class is distinctive: it is itself photosensitive and can be developed like a resist, the circuit pattern exposed into it directly with light, dispensing with a separate lithography step. These are the photosensitive RDL dielectrics. Most are high-performance polymers (heat-resistant plastics) acting as insulator and imaging medium at once. The group divides into photosensitive polyimide (PSPI, the most widely used), photosensitive PBO (polybenzoxazole, a higher-performing variant), photosensitive BCB dielectric, low-temperature-cure RDL dielectric (curing at a temperature low enough not to damage adjacent materials) and wafer passivation polymers (the final protective insulating layer on the wafer surface). Consider that an ordinary dielectric must be laid down and then patterned separately, whereas the photosensitive kind is laid down and patterned by light on its own, saving a process step. It is used extensively in fan-out packaging and HBM memory stacks.

15.19 EUV Resist Ecosystem

EUV (extreme ultraviolet lithography) is the technology the most advanced AI chips must use, drawing the finest lines with very short wavelength light. But EUV resist and its companion materials are far harder to make than ordinary resist, and form a small ecosystem of their own. The group divides into chemically-amplified EUV resist (the traditional route), metal-oxide EUV resist (the newer route, with higher resolution), dry EUV resist (an entirely new deposition-based process developed jointly by ASML and Lam), EUV underlayer (laid beneath the resist to improve imaging) and EUV developer and rinse chemistry. Consider ordinary resist as conventional photographic paper and this set as the complete package of specialised chemistry and consumables formulated for the most exacting ultra-high-definition development — the barrier so high that only a handful of companies worldwide can participate.

15.20 Advanced Package Thermal Submaterials

Large AI dies run at very high power and generate fierce heat, more than ordinary TIM can handle, so advanced packaging brings in a set of specialised high-end thermal submaterials. The group divides into sintered silver (silver particles sintered into an exceptionally conductive joining layer, for top-tier heat removal), liquid-metal TIM (using a liquid metal alloy, far more conductive than grease but carrying corrosion risk), integrated heat spreader materials (IHS, the metal lid capping the die, spreading and carrying heat away), lid attach materials (the thermally conductive adhesive bonding the lid to the package), high-conductivity package underfill and vapour-chamber wick materials (the porous structure inside a vapour chamber that circulates the working fluid by capillary action). Consider this group the full set of advanced fever-reducing equipment issued to a chip running very hot indeed.

15.21 High-Purity Quartz, Quartzware & Ceramic Consumables

When a wafer is processed at high temperature inside a tool chamber, the vessels and parts that hold, surround and carry it must be made of very high-purity quartz (high-purity fused silica) or speciality ceramic, or they will out-gas impurities at temperature and contaminate the wafer. This group is those consumable chamber wares: quartz crucibles (the large bowl holding molten silicon while a single-crystal ingot is pulled, destroyed after a single use — a pure consumable), quartz process tubes and boats (the tube and rack that load and carry wafers in a furnace), quartz showerheads and viewports, high-purity process ceramic parts, and ceramic chamber rings and liners (consumables protecting the inner wall of an etch chamber, replaced on a schedule). Consider these the heat-resistant crockery and furnace lining consumed repeatedly in a fab's high-temperature ovens, replaced once worn.

15.22 Specialty Rare Gases & Critical Minerals

This group is the raw material of the raw materials: speciality gases and metal ores produced in small quantity yet sitting across critical nodes. Neon (the gas feedstock for the scanner's laser, which spiked when Ukrainian supply was cut), helium (for process and cooling, in tight global supply), krypton and xenon (rare gases used in etch and lithography), gallium and germanium feedstock (the key metals of compound semiconductors, mainly produced in China, which has used them as an export-control lever), tantalum, tungsten and molybdenum feedstock (the refractory metals behind targets and devices) and rare-earth polishing feedstock (the ceria in the CMP slurries above comes from rare earths). Consider this group the ores and gases at the very bottom of the chain: unnoticed in ordinary times, but let one source be cut off or restricted and every link above it shudders.

15.23 Wide-Bandgap SiC/GaN Substrate & Epi

Ordinary chips are built in silicon, but in power applications demanding high voltage, high current and high efficiency, silicon is not enough and two wide-bandgap semiconductors take over: silicon carbide (SiC) and gallium nitride (GaN). (Wide bandgap means the material withstands higher voltage and temperature at lower loss.) In the AI datacentre they are being drawn into the rack by the spread of high-voltage DC power delivery, where they make more efficient power devices. This group is their substrate and epitaxial feedstock: SiC substrates (silicon carbide wafers, exceptionally hard to grow and exceptionally expensive), SiC epitaxial wafers, bulk GaN substrates, GaN-on-Si epitaxial wafers (growing gallium nitride on cheap silicon to save cost), GaN-on-SiC epitaxial wafers (better performance, used in RF) and emerging gallium-oxide substrates (a newer, experimental material). Consider silicon ordinary steel and SiC/GaN the speciality alloys that tolerate higher temperature and voltage, reserved for the heavy-current work.

15.24 ALD/CVD Precursors (GAA, High-k Precursors)

As described above, films are grown on the wafer by flowing in gases, and among those gases is a distinctive, very high-end class of chemicals called precursors: compounds containing a specific metal or element which, delivered into the tool, decompose and plant that element onto the wafer precisely, layer by layer, only a few atoms thick. Consider the precursor a speciality ink cartridge for 3D printing, loaded with the atomic feedstock to be printed. The most advanced AI chips use GAA (gate-all-around, the next-generation transistor structure) and high-k metal gate (using a specialised high-dielectric material for the gate insulator), and both make severe demands of the precursor. The group divides into silicon-oxide and silicon-nitride precursors, high-k hafnium and zirconium precursors (for the high-k insulating layer), metal-gate work-function precursors, tungsten, molybdenum, cobalt and ruthenium precursors (for growing metal interconnect), selective-deposition inhibitor precursors (so the film grows only where it should) and III-V MOCVD precursors (for compound semiconductors).

16.1 Front-end Synthesis Tools (EDA synthesis / front end)

A chip designer first writes out the chip's logical function in a hardware description language (much like writing code); this is the RTL (register-transfer level, best understood as the chip's functional source code). Synthesis is the step that automatically translates that source into a real circuit diagram built out of individual logic gates (AND, OR and other elementary circuit units). Write the instruction "add these two numbers" and the synthesis tool turns it into a specific mass of circuitry and wiring. This slot also covers the checking tools that confirm the source was written correctly in the first place.

16.2 Physical Design Tools (EDA physical design)

The previous step turned function into a circuit diagram, but that diagram is still only logical connectivity: nothing has a physical position. Physical design decides where on the die each circuit element sits, how the wires run, and how to guarantee that signals arrive on time — what the industry calls place & route and timing closure (making sure a signal reaches its destination within its allotted window). It is the equivalent of planning a city: siting every building, routing every road, and still guaranteeing the delivery vans arrive punctually. This is the largest and most valuable single block in all of EDA.

16.3 Verification & Simulation Tools (EDA verification / simulation)

You cannot learn by trial and error once a chip is built — a single tape-out runs to tens of millions of dollars — so the chip has to be run over and over in software first to see whether it misbehaves. That is verification and simulation. Simulation means having software pretend to be the working chip, feeding it every kind of input and checking the outputs are right. A stricter method, formal verification, uses mathematics to prove that certain functions cannot fail. This slot also sells verification IP: ready-made verification blocks that save you writing the tests yourself.

16.4 Emulation & Prototyping (EDA emulation / prototyping)

Pure software simulation is too slow — one second of chip behaviour can take hours to compute — so the industry built dedicated hardware machines to accelerate it. These are emulators: multi-million-dollar boxes. There are also prototypes built from FPGAs (chips whose function can be rewritten repeatedly) that let software teams debug early on something close to real hardware. An emulator is, in effect, a full-scale mock-up you can test-drive before the chip itself exists.

16.5 Analogue Design & Signoff Tools (EDA analog signoff)

Most of what has been described so far is digital circuitry — nothing but ones and zeros. But a chip also carries analogue circuitry, handling continuous voltage and current signals such as power delivery and clocking, and designing it is an entirely separate discipline. The tools in this slot handle analogue schematic capture, compute the circuit's true electrical behaviour using SPICE (the standard for circuit simulation), and perform signoff — the final sign-off confirming that power and timing are within specification and the chip can be sent to manufacturing. Signoff is the building inspection before handover: nothing ships until it passes.

16.6 PCB & Package Design Tools (EDA PCB / package design)

Once the chip itself is designed it still has to be soldered onto a circuit board (the PCB, or printed circuit board — the green board inside every electronic product) and sealed into a package (the protective shell around the die that brings its pins out to the world). The tools in this slot draw the routing for boards and packages, and compute signal integrity and power integrity — making sure high-speed signals do not distort in transit and that the power supply holds steady.

16.7 Chiplet & 3D-IC Design Enablement (chiplet / 3DIC design enablement)

The most advanced AI silicon is no longer a single monolithic die but several smaller dice — each one a chiplet — assembled and stacked like Lego. This is the world of chiplets and 3D-IC (three-dimensional integrated circuits). Assembling and stacking them requires dedicated design tools to plan the job: which die goes where, how the dice talk to each other at high speed, how the heat escapes, and whether the stack will warp under its own height. It is the difference between putting up a single detached building and master-planning an interconnected complex — the complexity rises sharply.

16.8 Processor IP (processor IP)

This is the largest IP business in the layer. Processor IP means licensing out the design of a CPU, GPU or AI accelerator core: the customer pays for the blueprint and builds it into their own chip rather than designing a processor from nothing. The famous name is Arm of the United Kingdom — almost every mobile chip in the world, and a growing share of server silicon, carries a CPU core licensed from Arm. RISC-V is an open, royalty-free instruction set (the command language a chip understands) that has been challenging Arm in recent years. A processor IP company, in short, sells the blueprint for the chip's brain and collects a royalty on every unit shipped.

16.9 Interface IP (interface IP)

For a chip to talk to the outside world — to memory, to the network, to other chips — it needs interface circuitry: PCIe (the high-speed lane inside a computer that connects graphics cards and drives), Ethernet, DDR and HBM for memory, UCIe (the emerging standard for connecting chiplets), and SerDes (circuitry that converts parallel data into a high-speed serial stream and back). These interfaces are extremely hard to design and highly standardised, so designers simply buy ready-made interface IP and drop it in. In AI silicon it is worth more than anywhere else, because an AI chip's whole job is to move data at furious rates.

16.10 Memory & Foundation IP (memory / foundation IP)

Every chip needs a set of elementary building blocks inside it: on-die memory (SRAM, the fastest small-capacity memory on a chip), the standard cell library (prefabricated elementary logic gates, the standard bricks of the building), I/O libraries and so on. Every chip uses them, and each must be tuned to a specific fab's process, so they too are sold as IP. This is the builders' merchant of chip design.

16.11 Analogue, Security & AI IP (analog / security / AI IP)

This slot holds specialist function blocks of every kind: PLLs (phase-locked loops, the circuitry that generates a stable clock beat for the chip), data converters (turning analogue signals into digital and back), RoT (Root of Trust, an absolutely trusted vault circuit on the die responsible for boot verification and key handling), NPUs (neural processing units, IP cores built specifically for AI arithmetic), and the network-on-chip (NoC, the internal motorway along which the chip's blocks talk to one another).

16.12 Design Services & ASIC Houses (design services / ASIC houses)

Some companies — a cloud provider, say — want their own custom AI silicon but have no full chip design team, so they outsource the entire chip to a specialist design services firm that carries it from design all the way to hand-off for manufacture at TSMC. This is ASIC design service (an ASIC being a chip built for one specific purpose). It is what you do when you want an unusual building put up and know nothing about engineering: you appoint a main contractor and hand them the lot. Through the AI boom, the large customers designing their own silicon — Google, Amazon, Meta — all have a firm of this kind standing behind them.

16.13 Cloud EDA (cloud EDA)

Taking the EDA tools above and running them on demand in the public cloud rather than installing them on local machines, so that a design house need not own an expensive server farm and can instead rent compute when it has a large verification run to get through. This is a change in delivery model rather than a market in its own right, and it has no independent pure plays.

16.15 Multiphysics Simulation (multiphysics / thermal / SI / PI / warpage simulation)

As chips stack higher and draw more power, looking at the electrical behaviour alone is no longer enough: heat, electromagnetics, mechanical stress and warpage all have to be computed together. That is multiphysics simulation. An AI chip at full load runs blisteringly hot, and thermal expansion will lift and warp the package, so all of it has to be worked out in software in advance. The premier asset here is Ansys, now owned by Synopsys.

16.16 Electronic-Photonic Co-design (electronic-photonic co-design / CPO EDA)

AI datacentres increasingly move data using light rather than copper — light is faster and loses less — which has produced the technique of packaging optical components directly alongside the die (CPO, co-packaged optics). But electronics and photonics obey two entirely different sets of physics, so designing them together requires dedicated co-simulation tools and a photonic process design kit (PDK, effectively the design manual for the optical components).

16.20 Design-for-Test & Silicon Lifecycle IP (DFT / silicon lifecycle management IP)

Every chip has to be tested for defects once built, but you cannot put a probe on all that internal circuitry, so test circuitry must be buried in the design from the outset. This is DFT (Design for Test), within which ATPG (automatic test pattern generation) writes the examination paper the chip then has to sit. A related class of IP keeps monitoring the chip's health — temperature, ageing, performance degradation — after it has left the factory and is running inside a server; this is silicon lifecycle management (SLM). DFT is the pre-delivery inspection; SLM is the fitness tracker worn thereafter.

16.21 AI-assisted Design (AI for EDA / generative design)

Using AI to design the chip itself: machine-learning algorithms try tens of millions of place-and-route configurations automatically and find designs better than a human engineer would reach. This is a capability upgrade happening inside the EDA tools — Synopsys's DSO.ai, Cadence's Cerebrus — rather than a standalone product line.

16.22 Chiplet Marketplaces & Assembly Kits (chiplet marketplaces / ADK)

Once chips are assembled out of small dice, the industry needs a marketplace in which ready-made chiplets can be bought and sold, and a common assembly design kit (ADK) that tells you how dice from different vendors fit together and how to verify the result. It amounts to building a standard-parts store and an assembly-manual library for the chiplet era. It is still very early, and the standards are being driven largely by the chip majors and the foundries.

17.1 Container Orchestration (container orchestration / Kubernetes)

A container packages a program together with everything it needs to run into one standard box that will run on any machine. When you have thousands of such boxes to manage, you need a dispatcher that decides automatically which machine each box lands on and restarts it if it fails. That is Kubernetes (K8s in the trade, today's mainstream container orchestration system). Think of K8s as a giant logistics control centre, automatically assigning tens of thousands of parcels to the right shelf. Kubernetes itself is open source and free; listed companies make their money from enterprise editions and services built around it.

17.2 GPU Gang Scheduling (GPU gang scheduling)

Training a large model means several thousand GPUs computing in lockstep — one short and the job does not run, much as a choir has to come in together. Gang scheduling ensures that the whole batch is either allocated and starts together, or waits together; it never allows half of them to work while the other half sits idle. This is a scheduling problem peculiar to AI training.

17.3 GPU Virtualisation & Partitioning (virtualization / GPU partitioning)

An expensive GPU running a small job is badly underused. GPU partitioning divides one physical GPU in software into several slices handed to different users simultaneously without interference (MIG and vGPU in the technical vocabulary). It is the equivalent of subdividing one large office into separate lettable desks. The confidential-computing variants go further and guarantee that no tenant can see another tenant's data.

17.4 MLOps Platforms (MLOps platforms)

MLOps (machine learning operations) is the end-to-end platform for running the business of training AI models: data preparation, training, evaluation, deployment, monitoring and retraining, assembled into a production line. If writing code has its development process, training AI models needs its own factory-floor management system, and that is MLOps.

17.5 Pipeline & Workflow Orchestration (pipeline / workflow orchestration)

An AI or data job usually breaks into many steps run in sequence or in parallel — fetch the data, clean it, train, deploy. Workflow orchestration uses software to draw those steps as a dependency graph (a DAG, or directed acyclic graph — a flow chart that moves forward step by step and never loops back) and triggers each step automatically in order.

17.6 Experiment Tracking & Model Registry (experiment tracking / model registry)

Training a model means running hundreds or thousands of experiments with different parameters, which requires tooling to record the configuration and the result of each one so they can be compared — experiment tracking. The models that come out then have to be registered, versioned and attributed so they can be managed and reproduced — the model registry. Between them they are an AI team's laboratory notebook and finished-goods ledger.

17.7 Model Serving & Inference Orchestration (model serving / inference orchestration)

Once a model is trained it has to go into service so that users can call it to answer questions — that is inference. But user demand rises and falls, so software has to route each request to a suitable GPU, scale capacity up automatically at peak, and batch multiple requests together for efficiency. The whole apparatus is inference orchestration: a restaurant opening extra service windows at the lunch rush and combining like orders into a single pass through the kitchen.

17.8 Observability & Monitoring (observability / monitoring)

Once a system is running you need to know at any moment whether it is healthy, where it is slow and what is about to break. That is observability, and it rests on collecting three things: metrics (the numbers), logs (the record of events) and traces (the path a single request took through the system). It is the datacentre's full-body scanner and wall of screens. This is one of the few slots in the layer with a substantial population of listed pure plays.

17.9 AIOps Analytics (AIOps / analytics)

AIOps means using AI to run operations: algorithms that find anomalies in vast monitoring datasets, identify the root cause of a fault, predict which piece of equipment is about to fail, and in some cases remediate automatically, sparing an engineer from watching a screen. Conventional monitoring turns a light red; AIOps tells you which cable is about to part and reconnects it for you.

17.10 Cost, Capacity & FinOps Optimisation (cost / capacity / FinOps)

AI compute is brutally expensive, and monthly cloud bills run into the millions of dollars. FinOps (cloud financial operations) is the software that keeps watch on where the money goes, what is being wasted and what could be cheaper — spotting GPUs bought but never fully loaded, or a committed contract that would cost less. It is the datacentre's intelligent bookkeeping and cost-saving adviser.

17.11 DCIM (Data Center Infrastructure Management software)

DCIM is the software that manages the physical estate of a datacentre: which racks still have space, how much power and cooling headroom remains, where each piece of equipment sits, the asset register, and energy consumption. It is the building management suite of the datahall, letting an operator see the entire physical state of the site on one screen. Note that DCIM is software; the power and cooling hardware it manages belongs to Layers 5 and 6.

17.12 BMS/EPMS/SCADA Software (building, power and industrial control software)

This slot covers the control software for the building and the electricity beneath it. BMS (building management system) runs air conditioning, lighting, access control and the rest of the building plant; EPMS (electrical power monitoring system) watches the flow of power through the whole site in real time; SCADA (supervisory control and data acquisition, the general-purpose industrial platform for equipment monitoring) gathers sensor data and issues control commands. This is industrial control software that factories and office towers have used for decades, redeployed to run datahalls (the hardware being monitored sits in Layers 5 and 6).

17.13 Energy, Thermal & CFD Modelling Software (energy / thermal / CFD modeling)

Before a datahall is built, the way air will move, heat will disperse and power will flow has to be simulated on a computer. CFD (computational fluid dynamics) is the software for computing airflow and liquid flow. You blow the air through the virtual building first and see where the heat pools, rather than discovering after commissioning that one row of racks runs too hot.

17.14 Compilers, Runtimes & Kernel Libraries (compilers / runtimes / kernel libraries — the CUDA layer)

This is one of the true sources of NVIDIA's moat and is worth stating precisely. The code an AI programmer writes cannot be fed straight to a GPU; a translation layer and toolbox sits in between. The compiler turns the code into instructions the GPU can execute; the runtime schedules the GPU while the program runs; and the kernel libraries are pre-written, exhaustively optimised implementations of the common mathematical operations, matrix multiplication above all. NVIDIA bundles the lot as CUDA — best understood as the complete programming platform and toolset for NVIDIA GPUs. Every AI programmer in the world writes against CUDA, and switching to another vendor's silicon means relearning the craft. That is NVIDIA's deepest software moat. AMD's counterpart is ROCm; Intel's is oneAPI.

17.15 Data Platforms, Lakehouses & Vector Databases (data platform / lakehouse / vector DB)

AI consumes enormous quantities of data, which has to be stored and queried somewhere. The lakehouse is the modern data platform that fuses the data lake (tip everything in first) with the data warehouse (organised for querying). The vector database is a creature of the AI era: what it stores is not conventional tables but the strings of numbers — vectors — that text and images are converted into, allowing retrieval by semantic similarity. It is the key to giving a large model external memory and retrieval, known in the trade as RAG (retrieval-augmented generation). In effect, the vector database lets the AI look up the relevant material before it answers.

17.16 Networking Software, SDN & AI Fabric Management (networking software / SDN / fabric management)

Several thousand GPUs in an AI cluster are welded into one machine by an ultra-high-speed network, and that fabric has to be configured, scheduled and kept clear of congestion by software. SDN (software-defined networking) lifts the network's brain out of the hardware and puts it under centralised software control, so traffic can be re-shaped flexibly. AI fabric management then has to solve the particular problem of keeping thousands of GPUs from gridlocking the network when they all transmit at once.

17.17 Digital Twins & Live Ops Simulation (digital twin / live ops simulation)

A digital twin is an exact virtual copy of a real datahall, fed with live data from the physical site, in which an operator can simulate what happens if a given machine fails, or how much additional load would push the hall into thermal trouble, before acting. It is the flight simulator in which a pilot rehearses every emergency before taking a real aircraft up. NVIDIA's Omniverse is the flagship platform here.

17.18 GPUaaS Control Plane & Compute Brokering (GPUaaS control plane / brokering)

The neoclouds package GPU capacity and rent it out, and behind that sits a control plane managing who has rented what, how it is provisioned, how it is billed and how capacity is allocated across multiple clouds. Brokering is the business of finding a customer the cheapest GPUs with the most availability across several providers. This is the front desk and dispatch system of the GPU rental trade.

17.19 Power, Thermal & Energy Optimisation Software (power / thermal / energy optimization SW)

Using software to be intelligent about electricity and water: scheduling jobs around power prices and carbon intensity, adjusting the cooling setpoint dynamically (how cold the plant actually needs to run), and capping rack power to prevent a trip. Google famously applied AI to its cooling and cut the energy it consumed substantially. This slot is the software optimisation; the hardware being optimised sits in Layers 5 and 6.

18.1 Network & Infrastructure Security (network / infra cybersecurity)

This is cybersecurity in its most classical form: the firewall (the gatekeeper that blocks bad traffic at the network's entrance), DDoS protection (against a distributed denial-of-service attack, in which an attacker floods your site with junk requests until it collapses), endpoint protection (keeping each individual server and machine from being compromised), and SASE (networking and security packaged as a cloud service so users connect securely from anywhere). Together they are the city wall, the gatekeeper and the patrol of the datacentre network.

18.2 Identity & Access Management (identity / access management, IAM)

IAM governs who gets in and what they are allowed to touch: the credentials and multi-factor authentication employees log in with, special protection for the highest-privilege accounts such as administrators (PAM, privileged access management), and the governance that keeps identities compliant. Zero trust is the organising idea of this slot: trust nobody by default, re-verify identity on every access, and stop waving people through merely because they are inside the corporate network. Zero trust means badging in at every door in the building rather than roaming freely once past the lobby.

18.3 Cloud & Application Security (cloud / application security)

With the business moved onto the cloud, the cloud environment needs protecting in its own right. CSPM (cloud security posture management) checks cloud configuration automatically for holes — a storage bucket accidentally left open to the whole internet, say. CNAPP is the full cloud-native application protection platform. Alongside them sit tools built specifically to protect containers, Kubernetes and data held in the cloud. Call it the complete burglar alarm and safety inspection for a house you rent in the cloud.

18.4 Security Operations Centre (SOC / SIEM / XDR)

The SOC (security operations centre) is the enterprise's security control room, a team plus a system watching around the clock for attacks. SIEM (security information and event management) is its core software: it aggregates security logs from everywhere in the company and surfaces the suspicious patterns automatically. XDR (extended detection and response) joins up threat signals from endpoints, the network and the cloud so that a detection in one place triggers a coordinated response everywhere. SOAR (security orchestration, automation and response) then runs the remediation playbook automatically once a threat is found. The SOC is the security command centre, SIEM its wall of monitors, and XDR the system that puts every camera on one network.

18.5 Managed Security Services (managed security services, MSSP)

Many companies cannot build a security team of their own, so they outsource monitoring, threat response and compliance to a specialist firm running the job 24/7. That firm is an MSSP (managed security services provider) — the equivalent of a residential block contracting a professional security company rather than hiring its own guards. This slot contains security software companies alongside large IT services firms and telecom operators.

18.6 Physical Security & Access Control (physical security / access control)

Everything above defends against the hacker online; this slot defends against the intruder on site: surveillance cameras, access control (badge or fingerprint to enter the hall), biometrics for identity verification, perimeter intrusion detection that raises the alarm if the fence is climbed, visitor management and the security control room. With that much expensive silicon inside, physical protection matters just as much.

18.7 Managed Datacentre Services & Remote Hands (managed DC services / remote hands)

A customer's equipment sits in somebody else's facility (colocation) but the customer is not on site, so the facility's staff have to act on their behalf: reboot a machine, plug or unplug a cable, install a drive. That is remote hands. The more skilled version of the same service is smart hands. The slot also covers the NOC (network operations centre, the shift room watching the network around the clock), racking and de-racking equipment, and cabling. It is, in short, the facility doing the physical work on your behalf.

18.8 Datacentre O&M and Field Operations (DC O&M / field operations)

Behind a facility that runs every day of the year stands a professional operations team performing preventive maintenance (servicing on a schedule, before anything fails) and predictive maintenance (using sensors to catch equipment on its way to failure), emergency repairs on critical plant, spares management, shift cover and energy operations. This is the AI factory's building management, electricians and shift supervisors.

18.9 Disaster Recovery & Business Continuity (disaster recovery / business continuity)

If the facility catches fire, is attacked or goes down, the data must not be lost and the business must not stop. This slot covers regular data backup, second-scale switchover to standby systems when something breaks (failover), and the isolated vault that defends against ransomware by keeping backups somewhere the malware cannot reach. The AI era adds one more requirement: training a large model runs for weeks, and an interruption partway through cannot mean starting over, so the run has to be saved and resumed (checkpoint-restart). Think of it as the datacentre's full insurance policy plus a save function.

18.10 Compliance & Risk Management (compliance / risk management, GRC)

Large institutions have to satisfy a mass of regulation — data protection, industry codes — and be ready for audit at any time. GRC (governance, risk and compliance) is the software that holds those rules, risks and audit evidence together, checking automatically whether anything has crossed a line and collecting the evidence as it goes. It is the corporate compliance officer plus an audit file that is always ready.

18.11 Confidential Computing & GPU Trusted Execution (confidential computing / GPU TEE)

Data can be encrypted at rest and in transit, but at the moment it is actually being computed it is usually in the clear — and that is a hole. Confidential computing keeps data encrypted in hardware even while it is being processed, so that not even the cloud provider can see it. The hardware that delivers this is the TEE (trusted execution environment, a compartment on the die that outsiders cannot see into under any circumstances). It matters particularly in AI, because nobody is comfortable putting an expensive model and sensitive data onto somebody else's GPU. NVIDIA's latest GPU generations have the capability built in.

18.12 Remote Supply Chain & Runtime Attestation (remote supply chain attestation)

A server leaves the factory, travels to the facility, is installed and starts running — but has anyone tampered with it along the way, or planted a back door? Attestation is the use of cryptography at each step to verify that the device and its firmware really are genuine and unaltered. It amounts to issuing every machine a tamper-evident certificate of provenance covering the whole journey.

18.13 Firmware & BMC Security Resiliency (firmware / BMC security resiliency)

Firmware is the low-level code written directly into the hardware and the first thing to run at power-on. Compromise it and antivirus software cannot help you, because firmware sits beneath the operating system. The BMC (baseboard management controller) is a separate small chip inside a server that allows it to be managed remotely even while the host is powered down — which makes it a prize target in its own right. This slot exists to make sure boot firmware is trustworthy (secure boot), that tampering is automatically recovered from, and that vulnerabilities are patched promptly.

18.14 Encryption, Key Management & HSM (encryption / key management / HSM / KMS)

The security of encrypted data rests entirely on the key not being stolen. KMS (key management service) generates, stores and rotates those keys securely. The HSM (hardware security module) is a tamper-resistant, tamper-evident safe in hardware: the keys live inside it, nobody can extract them, and prising it open destroys the contents. If encryption is the lock on a safe, the HSM is the key room — unpickable, and buried in a bunker.

18.15 Post-Quantum Cryptography Migration (post-quantum crypto migration, PQC)

Once quantum computers mature they may be able to break almost all of today's encryption. Post-quantum cryptography (PQC) is the new generation of algorithms designed to resist that. Migrating to it is a substantial undertaking: first you have to find every piece of legacy cryptography in use across the organisation (the CBOM, or cryptographic bill of materials — essentially an inventory of every lock in the building), then replace it algorithm by algorithm. It is fitting every door in advance with a lock that cannot be picked in the future. The field is still early, but the large institutions have already begun to position for it.

18.16 Zero-Trust Microsegmentation (zero trust microsegmentation for clusters)

On a traditional network, an attacker who gets inside can move sideways at will. Microsegmentation divides the internal network into small compartments, requires authentication for traffic between them, and so confines an attacker who breaches one compartment to that compartment. It matters especially for AI clusters, where the east-west traffic between thousands of GPUs (server-to-server communication) also has to be controlled. It is the difference between a warehouse subdivided into countless locked rooms and one open floor you can walk straight across.

18.17 AI Security Posture Management (AI security posture management, AI-SPM / AI-BOM)

AI has itself become something that is attacked and must be governed. AI-SPM inventories which models and datasets a company is actually using and where the risk exposure lies, and keeps checking. AI-BOM (AI bill of materials) lists the models, datasets and components that go into a given AI system, so that problems can be traced to their source. Between them they are a register and a health record for every AI application in the organisation. This is a new and fast-growing niche.

18.18 AI Runtime Guardrails & Agentic Security (AI runtime guardrails / agentic security)

Once a large model is in service, people will try to talk it into disclosing things or doing harm. Guardrails put a filter on the model's inputs and outputs, blocking malicious prompts and stopping content that should not be produced. Prompt injection defence deals specifically with carefully crafted language used to trick a model into exceeding its authority. And once AI becomes an agent — able to call tools and carry out multi-step tasks on its own initiative — its identity and permissions have to be constrained and everything it does has to be audited. This is the hard hat, staff badge and body camera for an AI that acts.

18.19 OT & Cyber-Physical Facility Security (OT / CPS facility security)

The industrial control equipment that runs power and cooling inside the facility — BMS, PDUs, UPS, the cooling plant — is also networked, and can also be attacked. A successful attack could simply switch off the power or the cooling for an entire site, with catastrophic results. OT security (operational technology being the systems that control physical equipment, as distinct from the IT that runs the office computers) exists to protect that equipment: inventorying what is connected, monitoring the industrial network and managing its vulnerabilities. It is the last line of defence against an attacker who is not interested in stealing data and simply wants to pull the switch.