Fouling vs Turbidity
Loading…
The problem
Fouling and turbidity both lift the ToF amplitude floor, and the shipped detector
runs on amplitude alone — so it cannot tell them apart. It leans on a streamflow decorrelation
guard (FLOW_CORR 0.4) that only works where a gauge exists. At Paris,
Cleveland, BGS/Thames and Bow River there is no usable gauge, so the detector calls fouling
with nothing to check it against.
This matters beyond a label. Where the offset absorbs storm turbidity, the corrected ToF understates real turbidity during storms — which at CSO-driven sites is exactly when E. coli spikes.
distance_mm started working on v0.1.20
(2026-08-21). Until then it emitted a constant sentinel and carried no information at all, so
every previous attempt to use ranging was measuring a dead feed.
The hypothesis, and the bench evidence for it
The two channels answer different questions, which is what makes a discriminator possible. Measured on the 7-unit bench cohort at /tof-ranging:
| channel | cap on → off (in water) | air → water (cap on) |
|---|
Fouling is a film on the window — a surface change. Turbidity is particles in the path — a medium change. That is the mapping this investigation exists to test in the field.
These are bench numbers on a cap, not field numbers on a biofilm. A cap is a hard surface at a fixed offset; fouling is soft, gradual and irregular. The bench result says the channels are separable in principle. It does not say the effect size or the direction will carry over, and that is precisely what the field data has to settle.
Field readiness
An algorithm cannot be built on units that are not returning valid ranging, so the first question is how much of the deployed fleet actually carries the signal.
By customer
| customer | deployed | alive | on the fix | usable | streamflow gauge? |
|---|
The cross-check
Amplitude alone cannot separate the two conditions, because both move it. The distance is what breaks the tie — and only conditional on an amplitude change:
| condition | what physically changes | amplitude | distance |
|---|---|---|---|
| turbidity | particles in the path | moves | unchanged — the surface it ranges against is the same |
| fouling | sediment on the cap | moves | shortens — the return surface has moved closer |
So the signature to look for is amplitude rising while distance falls. Each unit below is one point: horizontal is its amplitude trend, vertical its distance trend. The lower-right quadrant is the fouling signature.
Is a fouling trend even resolvable?
A slope means nothing without its own uncertainty. lag-1 noise — successive differences over √2 — cancels any smooth trend, so what survives is reading-to-reading scatter: the floor a fouling signal has to clear.
| unit | customer | dist mm/day | kcps %/day | corr | n | window |
|---|
Ground truth — the cleaning event registry
A cleaning is the one moment we know fouling was present and then removed, so the registry
(D1 sensor_maintenance, served at /api/maintenance/all) is what any
fouling algorithm has to be trained and judged against.
auto-detect rows are the
current detector's own output — the thing this work exists to improve — so
training on them would be circular and would simply relearn the amplitude rule with extra steps.
Nothing reports a servicing automatically. A cleaning exists in the record only if a person logged it, which is why human rows are so heavily outnumbered. That is the binding constraint on this work, not the sensor.
Registry contents
| event_time | barcode | event_type | source | deployed? | after the fix? |
|---|
The confirmed cleanings
Everything above is a hypothesis until a cleaning is observed through a working distance channel. Six such episodes now exist, all confirmed by the people who did it rather than inferred from a step in the data: Little Calumet on 2026-08-28, and five Boulder Creek units cleaned in sequence on the afternoon of 2026-08-31.
Both channels are judged against their own lag-1 noise — successive differences over √2, which cancels any smooth trend and leaves reading-to-reading scatter. A null result only means something stated against the smallest step the window could have resolved, so that bound is published beside it.
The distance channel is no longer silent, but it is not yet a lever. The earlier single-episode reading — that the ranging did not move at all at Little Calumet — does not survive a longer window: with 350 post readings instead of 21 it shows a resolvable +2 mm. BC-13, the most heavily fouled unit, moved −8 mm at 11.6× its own noise. But the four remaining episodes all sit at ±2 mm, and the sign is not consistent: Little Calumet moved away from the sensor, BC-13 towards it.
distance_mm is
quantised at 1 mm, so 2 mm is two steps, not a rounding artifact — but with
8–9 post readings on four of the five, a two-step median shift is weak evidence.
Nothing here supports the clean bench story yet, in either direction. The bench said cap removal moves the range ~27.5 mm; the largest field step is −8 mm and the rest are 2 mm. What can be said on this evidence is narrow: a fouling film thick enough to quadruple the amplitude floor does displace the ranged surface, by single-digit millimetres, in a direction that is not yet predictable.
The Paris question has been answered, and the answer removes it from this analysis rather than adding to it. Three City of Paris probes (500147, 500190, 500167) are now confirmed cleanings and are in the registry: Marion Delarbre's team opened each probe and wiped the interior on 2026-08-18, staggered across that morning. But 2026-08-18 is before the 2026-08-21 ranging fix, so their distance data comes from the broken feed and cannot be pooled with what came after. They are real ground truth for the amplitude channel and say nothing about the distance lever. The episode filter enforces this automatically: it takes human-logged cleanings on deployed units at or after the fix, which is why three confirmed cleanings are deliberately absent from the cards above.
Can we actually tell them apart?
The cleanings above are one row of a 2×2: what the channels do when fouling is removed. On their own they cannot answer the question, because they say nothing about what happens when turbidity rises and fouling does not change. Without that second row you can measure a discriminant’s sensitivity but not its specificity — and firing on both conditions is precisely the failure of the amplitude-only detector this work exists to replace.
River discharge supplies the missing row, on the same assumption the shipped flow-decorrelation guard already makes: when the river rises, the water carries more particles.
| unit | site | hours | r(flow, amplitude) | r(flow, distance) |
|---|
So the blocker is not specificity, it is sensitivity. Distance does not confuse turbidity for fouling; it simply does not reliably notice fouling either — it resolved a step on two of the five cleanings, and the sign was inconsistent between them. The honest position is therefore narrower and more useful than “it does not work”: if the distance channel can be made to respond to fouling dependably, it is already clean of the interference that defeats the amplitude. That is a sensitivity problem — films, thicknesses, geometry — not a selectivity one, and it points the next round of work at the bench rather than at more field labels.