How to Prevent and Remove Fouling in Data-Center Liquid-Cooling Loops
Fouling is the only liquid-cooling failure mode that never announces itself: no drip, no alert, just creepingly worse temperatures. The four deposit families, the closed-loop hardness budget, what a deposit layer costs in cold-plate temperature and pressure drop, the four lines of defense, and how fouled loops are actually cleaned - with 14 cited references.
A loop is commissioned, filled with clean water, and everything is green. Eight months later a GPU rack starts throttling on warm afternoons. No leak, no pump alarm, no failed component — just a slow, invisible layer building on the inside of the pipes and the cold plates. Fouling is the only liquid-cooling failure mode that never announces itself: no drip, no alert, just creepingly worse temperatures.
This article covers the whole arc of the problem — what fouling actually is in a data-center coolant loop, where the material comes from (the arithmetic is surprising), what a deposit layer costs you in cold-plate temperature and pressure drop, and then the four lines of defense: the water-quality envelope, hydraulics that keep deposits suspended, clean commissioning, and chemistry run on data. It closes with what to do when prevention has already failed — how fouled loops are actually cleaned, and how to decide between cleaning and replacement.
Four families of fouling — plus one factory defect
"Scale" is the word most people use, but mineral scale is only one of four deposit families that grow in coolant loops. They differ in chemistry, in how fast they form, and in what removes them — which is why the first diagnostic question is always what is the deposit made of, not how do we clean faster.
| Family | What it is | Typical driver | Remedy class |
|---|---|---|---|
| Particulate | Silt, brazing debris, solder balls, weld spatter, fibers, seal rubber crumbs | Poor commissioning; component wear; open ports | Filtration & flushing (mechanical) |
| Mineral scale (precipitation) | Calcium carbonate, calcium phosphate, calcium sulfate crystallizing on hot surfaces | Hard water ingress; evaporative concentration; wrong fill water | Water chemistry; acid cleaning |
| Corrosion product | Iron oxide (magnetite), copper oxide, zinc compounds from the loop's own metals | Wrong chemistry, galvanic couples, depleted inhibitors, glycol acids | Inhibitor control; derouging; passivation |
| Biological (biofilm) | Slime layers of bacteria and extracellular polymer on wetted surfaces | Stagnant zones, contaminated fill, no biocide strategy, warm loops | Biocide program; alkaline/surfactant cleaning |
The fifth contributor is not a deposit family but a factory defect in the fluid itself: glycol degrades. Ethylene and propylene glycol oxidize over time — accelerated by heat, oxygen and metal catalysis — into organic acids that drag pH down, attack the inhibitor package, and aggressively corrode aluminum. The protective oxide film on passivated aluminum is stable only between roughly pH 4.0 and 8.5; below that the passivation dissolves and bare metal faces the coolant [1]. An unbuffered glycol loop can poison itself over a couple of years without a single external contaminant entering it.
copper 399 | brass 119 | carbon steel ~59 — the wall it coats
CaCO₃ scale 2.26–2.93 | CaSO₄ 2.31 | magnetite 2.88 | analcite 1.27
biofilm 0.63 — four times more insulating than mineral scale at equal thickness
// a deposit layer is a ceramic or gel blanket on a metal wall.
// the thinner the channel it coats, the more it costs — see below.
Where the material comes from: the closed-loop budget
Here is the arithmetic most people get wrong. Cooling-tower fouling intuition says: hard water + evaporation = scale. A technology cooling loop is closed — nothing evaporates from it, so the hardness in the fill water is all the hardness there will ever be, unless more water enters.
Run the budget. OCP's water-quality guideline caps total hardness at 30 ppm as CaCO₃ in treated fill water [3]. A 400-litre rack loop filled at that ceiling carries at most 12 grams of precipitable calcium carbonate — in total, forever. Spread across roughly 8 m² of wetted cold-plate surface in the rack, that is a film of about 0.0006 mm: unmeasurable. Bulk hardness in a genuinely closed loop cannot foul it.
So if scale shows up in a closed loop, the question is never "is our water hard" — it is "what is continuously feeding the system?" The real suspects:
- Makeup water topping — every litre added to replace a leak or a repair drain brings fresh hardness, chloride and oxygen. A loop that "slowly loses pressure" and gets topped off monthly is an open system wearing a closed system's name.
- Cross-loop leakage — a plate heat exchanger with an internal weep lets facility water (hard, biological, chlorinated) season the TCS continuously.
- In-situ generation — corrosion product is fouling manufactured from the loop's own walls. Under-deinhibited water, galvanic couples (the aluminum cold plate downstream of a brass fitting), or glycol-derived acids all convert pipe metal into deposit. Magnetite and copper oxide then ride the flow until they lodge in the finest channel available.
- Biological growth — biofilm is generated on-site from trace nutrients and needs no external feed at all, only stagnation and warmth.
- Maintenance events — every opened quick disconnect, every component swap, every top-up from an unfiltered jug is a particulate injection.
What a deposit layer actually costs
Two mechanisms, and the second one is dominant at data-center channel sizes.
Thermal: the blanket in series with your cold plate. A deposit adds its own resistance in series with the wetted surface. Take a 1,000 W accelerator plate with roughly 64 cm² of wetted footprint — a heat flux near 156 kW/m² — and a clean convective film resistance of about 6.7 × 10⁻⁵ m²·K/W (h ≈ 15 kW/m²·K, typical for a microchannel plate). One quarter of a millimetre of calcium carbonate — k ≈ 2.6 W/m·K — adds 9.6 × 10⁻⁵ and costs about +15 °C on the silicon; the wetted-surface resistance multiplies by 2.4. A 0.1 mm biofilm — k ≈ 0.63 — is worse: about +25 °C, and a 3.4× multiplier. Field figures agree in direction: published case tables put ~0.25 mm of mineral scale at roughly a 10% heat-transfer loss for ordinary exchangers [4], and biofilm on copper at 30% loss per single micrometre of growth [5]. On a 1 kW device those numbers are the difference between spec and throttling.
Hydraulic: the allowance that closes a microchannel. Industrial fouling allowances assume wide tubes. Do the same geometry against cold-plate channels and the conclusion inverts. Assume a mineral layer of 0.68 mm — an allowance industrial water systems would consider ordinary for a shell-and-tube exchanger:
25 mm main → D_eff 23.65 mm → pressure drop roughly +12% — noticeable, tolerable
3 mm channel → D_eff 1.65 mm → pressure drop an order of magnitude worse
1 mm channel → fully closed
// even a 0.1 mm wall layer in a 1 mm laminar channel multiplies dP by ~2.4
// (cold-plate microchannels: a fraction of a millimetre across [6])
This is the whole reason fouling discipline in a liquid-cooling room is stricter than in any building system: the tubes got small. The 50 µm ceiling on particulate size in OCP's early rack guidance was written against exactly this geometry [7] — and a 50 µm particle is enormous next to a channel only a fraction of a millimetre across, which is why serious loops filter finer than the rack spec, not just to it.
Line of defense 1 — hold the water-quality envelope
The water quality table is not paperwork; it is the fouling prevention system. Two sources define the envelope for water-based TCS loops — OCP's cold-plate fluid guideline (values for treated, inhibited fill) and the ASHRAE-derived limits in early rack specifications:
| Parameter | OCP (treated fill/operating) | ASHRAE-derived rack limits | Fouling link |
|---|---|---|---|
| pH | 8.0–10.5 | 7–10.5 | Low pH corrodes; high pH precipitates scale |
| Total hardness (as CaCO₃) | < 30 ppm | < 20 ppm | Mineral scale |
| Chloride | < 50 ppm (304 SS) / < 500 ppm (316 SS) | < 5 ppm | Pitting corrosion → oxide fouling |
| Sulfate / sulfides | — | < 10 / < 1 ppm | Deposits, MIC |
| Turbidity | < 5 NTU | < 20 NTU | Particulate loading |
| Total suspended solids | < 5 ppm | < 3 ppm | Channel blockage |
| Conductivity (25 °C) | < 1,500 µS/cm (after treatment) | 0.2–20 µS/cm (DI loops) | Ionic contamination indicator |
| Bacteria | < 1 CFU/mL at fill; < 100 CFU/mL operating | < 100 CFU/mL | Biofilm → under-deposit corrosion |
| Cu / Fe corrosion products | < 0.2 / < 0.1 ppm | — | Corrosion rate indicator |
| Corrosion inhibitor / biocide | required (azole package for copper) | required | Chemistry floor |
| Max particulate size | — | 50 µm | Filter specification |
Two reading notes. First, the OCP numbers describe treated fluid — after inhibitor and buffer packages are dosed, conductivity and pH legitimately sit higher than a DI loop's; what matters is a documented baseline and drift from it, not a universal number [8]. Second, glycol loops are judged against the glycol guideline's own chemistry windows, and degraded glycol can violate them from the inside — test the fluid, not just the water.
Line of defense 2 — keep it moving (hydraulics as fouling control)
Deposits need residence time. Hydraulics deny it to them.
- Stay inside the velocity band. In mains, the working band runs from the erosion cap (2.1 m/s in copper-bearing circuits above 3 inch, 1.5 m/s in flexible hose) down to a floor where particles stop being carried and start settling. Sizing a loop for quiet operation at 0.6 m/s builds a settling tank. The full diameter-and-velocity arithmetic is in our pipe-sizing article.
- Eliminate dead legs and low points. Under-floor stubs, gauge lines and spare tees are where solids settle, biofilms seed and chemistry goes local. Every dead leg is a future deposit source with no flow across it to dilute the verdict.
- Filter in two places, not one. The CDU's main-loop filter (typically ≤ 50 µm) protects cold plates from circulating debris; a side-stream (bypass) filter polishing to single-digit microns slowly cleans the bulk volume without loading the main loop's pressure budget [7]. Watch filter differential pressure as a trend — a dP that climbs over weeks is the cheapest fouling instrument you own.
- Match cleanliness to channel size. Component-level particle-count targets (ISO 4406-style cleanliness codes) belong in the purchase specification for racks and replacement cold plates — a "clean" loop full of 30 µm solder balls is dirty by definition.
Line of defense 3 — commission clean
Most particulate fouling in new rooms is installed during construction and never removed. Mill scale, brazing flux, solder balls, thread tape and dust wait quietly in the headers until the first month of operation carries them into the finest passages on the site. Ecolab's commissioning guidance for direct-to-chip loops makes the point bluntly: materials harmless in conventional piping obstruct narrow passages in liquid-cooled systems, and the structured start-up sequence — hydrotest, chemical cleaning, staged flushing, final preparation — is the control [9].
The sequence that works:
2. Chemical cleaning — alkaline/surfactant for oils and organics; inhibited acid only where scale/oxide is confirmed; copper systems need inhibitors in the wash [9].
3. Staged flushing at 3–5 ft/s (1–1.5 m/s) — coarse-to-fine filtration, ending single-digit micron; verify with particle counts and clarity, not by clock time [9][10].
4. Passivation — citric (or nitric) treatment per ASTM A967 removes free iron and re-builds the stainless oxide layer [10].
5. Fill with treated fluid through fine filtration; capture the baseline sample set the same week [11].
Steps 3–5 have their own equipment discipline — OCP's mobile fluid-cart guidance exists because fill and flush quality depends on filtered, instrumented transfer rather than jugs and hoses [12]. Specialty flushing contractors run exactly this playbook with skid-mounted pumps, staged cartridge filtration and documented cleanliness verification [13].
Line of defense 4 — run the chemistry on data
A closed loop drifts slowly, which is a gift: sampled on a schedule and trended, it announces fouling conditions months before a cold plate feels them. The operating cadence that catches the drift:
| Signal | Cadence | Drift pattern → diagnosis |
|---|---|---|
| pH, conductivity (inline) | continuous | Falling pH → glycol degradation or nitrifying contamination; climbing conductivity → ionic ingress, inhibitor overfeed, resin exhaustion [8] |
| Turbidity / TSS | continuous or weekly | Rising → corrosion product generation or biofilm sloughing upstream |
| Filter differential pressure | continuous trend | Steady climb → particulate source active in loop |
| Cu / Fe ions | quarterly lab panel | Rising Cu → copper corrosion (check azole reserve); rising Fe → steel circuit corrosion or rouge [11] |
| Azole inhibitor reserve | quarterly | < ~100 ppm → copper protection lapsing; dose, don't guess [14] |
| Bacteria (CFU/mL) | quarterly (monthly if warm loop) | Climbing CFU → biocide strategy failing; expect biofilm in dead legs first |
| Glycol condition | semi-annual | Falling pH with rising acidity → glycol oxidation; plan inhibitor top-up or fluid replacement |
| Full baseline comparison | 3 / 6 / 12 months | Compare against commissioning baseline panel, not against memory [11] |
When it is already too late: remediation
If a loop shows fouling symptoms — rising cold-plate ΔT at constant flow, filter dP climbing, throttling racks — the sequence is diagnose, fix the feed, then clean. Cleaning before diagnosing re-fouls the loop within months.
- Identify the deposit. Lab analysis of a filter catch or a sacrificial fitting: mineral (acid-soluble), iron oxide (rouge, acid + reducing agent), biofilm (alkaline/surfactant, oxidizing biocide), particulate (mechanical). Acid on a biofilm strips the mineral skeleton and leaves the polymer; alkaline on scale does almost nothing. Method follows material [9].
- Fix the feed first. Pressure-test the exchanger boundary, stop the topping-off, restore the inhibitor and biocide program. Otherwise step 3 is a subscription, not a service.
- Clean at cold-plate-safe conditions. Microchannel cleaning runs controlled low-pressure circulation at ~1–1.5 m/s with staged filtration down to ~5 µm and particle-count verification — debris driven into microchannels cannot be flushed back out [6]. Isolate racks or pods; clean one hydraulic group at a time.
- Re-passivate after aggressive cleaning. Acid cleaning strips the protective oxide along with the deposit. Citric (or nitric) passivation per ASTM A967 rebuilds the stainless surface; copper circuits get their azole film restored [9][10]. Skipping this step turns a cleaning into a corrosion initiation event.
- Plan the waste. Flush water carries the chemistry you just used — containment, neutralization and disposal must be arranged before the first drain, not after [9].
- Know when to replace instead. A cold plate with hardened scale baked into its microchannels at high heat flux, or one that has already run months with a partially blocked channel array, often recovers less flow and temperature margin than a replacement costs — especially when a spare exists. Cleaning restores chemistry; it does not un-bake ceramics.
The one-page checklist
[ ] Fill water meets OCP envelope; hardness ≤ 30 ppm as CaCO₃; treated-fluid baseline archived [3]
[ ] Inhibited fluid only — azole for copper circuits, buffer for glycol; biocide strategy documented
[ ] No unfiltered top-ups, ever; make-up events logged as chemistry events
[ ] Velocity band respected in every main; zero dead legs in as-builts
[ ] CDU main filter ≤ 50 µm + side-stream polish; dP trended from day one
COMMISSION
[ ] Hydrotest water not left stagnant; chemical clean → staged flush 3–5 ft/s → passivation [9][10]
[ ] Particle-count verification before fill; baseline chemistry panel in week one [11]
OPERATE
[ ] Quarterly lab panel vs baseline; inline pH/conductivity trended with alarm limits [11]
[ ] Cu/Fe ions + azole reserve checked; glycol condition semi-annual
REMEDIATE
[ ] Deposit identified before any chemistry; feed stopped before any cleaning
[ ] Cold-plate-safe circulation, staged 5 µm filtration, re-passivation, waste plan [6][10]
Fouling is not a fluid problem or a filter problem; it is a system boundary problem — whatever crosses the loop's boundary, chemically or hydraulically, is what you will later scrape out of it. Hold the envelope, keep it moving, commission clean, and trend everything. If you are specifying a loop and want the fouling, velocity and diameter arithmetic checked against your actual rack configuration, send us the layout and we will quote the system that fits it. Related reading: sizing coolant pipe diameter from CDU operating conditions, rack-mount vs cabinet CDU architectures, and how to calculate dew point to prevent condensation.
References
- Renkeer, “AI Is Reshaping Liquid Cooling and Water Quality Monitoring” — glycol degradation to organic acids lowering pH, the aluminum passivation window of pH 4.0–8.5, conductivity and turbidity monitoring practice, and the ~10% heat-transfer loss from a 0.25 mm calcium carbonate layer. renkeer.com
- Maritech, “Calcium Carbonate Balance in Water” — thermal conductivities of deposits and alloys (biofilm ≈0.6 W/m·K vs calcium carbonate ≈2.3–2.9 W/m·K vs copper ≈386–400 W/m·K), biofilm heat-transfer loss tables by thickness, and scale–biofilm co-growth behavior. maritech.org
- Open Compute Project, Guidelines for Using Water-Based Transfer Fluids in Single-Phase Cold Plate-Based Liquid-Cooled Racks (2022) — treated-fluid envelope: TSS <5 ppm, TDS <1,000 ppm, conductivity <1,500 µS/cm at 25 °C, pH 8.0–10.5, hardness <30 ppm as CaCO₃, turbidity <5 NTU, bacteria <1 CFU/mL fill / <100 CFU/mL operating, chloride <50 ppm (304 SS) / <500 ppm (316 SS), Cu <0.2 ppm, Fe <0.1 ppm, azole copper-inhibitor guidance. opencompute.org
- Renkeer water-quality analysis, op. cit. — 0.25 mm (0.01 inch) calcium carbonate scale reducing heat-transfer efficiency by approximately 10% (consistent with the scale-loss ranges in [2] and [5]). renkeer.com
- Maritech, op. cit. — biofilm on copper: ~30% heat-transfer reduction at 1 µm, ~67% at 5 µm, ~81% at 10 µm. maritech.org
- Anvil Field, “Liquid Cooling Loop Commissioning Field Guide” — cold-plate micro-channels a fraction of a millimetre wide, staged coarse-to-fine flushing at 3–5 ft/s, microchannel cold plates demanding filtration to 50 µm or finer with many direct-to-chip projects finishing in the low single-digit microns, OCP's ~5 µm practical target, and cleanliness verified by sampling against acceptance criteria rather than by eye. anvilfield.com
- Lawrence Berkeley National Laboratory, “The Facility Perspective on Liquid Cooling” (SC18, LBNL/EEHPCWG/OCP rack working group) — ASHRAE-derived fluid-quality limits (chloride <5 ppm, sulfate <10 ppm, sulfides <1 ppm, hardness <20 ppm, turbidity <20 NTU, TSS <3 ppm, bacteria <100 CFU/mL, pH 7–10.5), 50 µm maximum particulate size with bypass/side-stream filtration for finer capture, and quarterly/monthly water-quality verification. datacenters.lbl.gov (PDF)
- Sensorex, “Optimize Data Center Liquid Cooling with Perfect Water Quality” — OCP treated-fluid values in context, ASHRAE guidance on charting TCS conductivity over time and investigating root causes of drift, baseline-vs-limit monitoring philosophy, and low-conductivity measurement practice. sensorex.com
- Data Centre Review, “How to build reliable liquid cooling systems” (Bob Walicki, Ecolab) — pre-commissioning cleaning for DTC loops, hydrotest-to-recirculation without stagnation, staged flushing at 3–5 ft/s, chemistry selection against metallurgy and elastomers, copper-inhibited cleaning, and wastewater planning. datacentrereview.com
- Accurate Metal Finishing, “Passivation of Stainless Steel” — free-iron removal and chromium-oxide film reformation by citric or nitric treatment, the five nitric and five citric methods of ASTM A967, alkaline-acid-alkaline practice, and the pickling-vs-passivation distinction (pickling removes scale and oxide films; passivation restores the protective film on an already-clean surface). accuratemetalfl.com
- Dober, “Data Center Leak Detection and Glycol Monitoring” — baseline chemistry panel at commissioning, follow-up panels at three and six months then annually with quarterly pH spot checks, ASHRAE TC 9.9 filtration and fluid-monitoring requirements, and the CDU as containment boundary. dober.com
- Open Compute Project, Developing a Mobile Cart for Datacenter Fluid Servicing, Rev. 1.0 — instrumented, filtered fluid transfer for fill/flush/top-up operations as the alternative to unfiltered field practice. opencompute.org
- B&W Energy Services, “High-Velocity Water Flushing for Data Centers” — contractor pre-commissioning practice for mission-critical cooling: engineered flush plans built from P&IDs, modular skid-mounted high-flow pumps and filtration, turbulent-flow debris mobilization into filter pots, and cleanliness verification by filtration inspection and membrane testing rather than water samples alone. bwenergyservices.com
- Resistance Zero, “CDU Installation, Inspection & Maintenance Checklist” — corrosion-inhibitor reserve >100 ppm azole with no nitrites, quarterly coolant sampling, filter dP trending, ISO 4406-style cleanliness targets, and loop velocity windows. resistancezero.com