What the drift experiments have established

The short version

The largest error in the record is light released inside the sensor when it is handled: an additive step on the zero that holds for one to two hours and relaxes over about ten. Temperature is correctable on each unit's own coefficient, with a small unit-specific residual. Once a unit is settled and left alone its zero moves under 1 %/day. The gain moved about 15 % over seven weeks without handling, and the FDOM puck does not track it. A procedure that never touches the unit between its zero and its reading cuts the error at 50 ppb from 10.9 to 4.2 ppb.

1Handling releases light, and that is the largest error

Every time a unit is disturbed its S-TLF steps up, on every unit, by an amount that does not scale with the reading. Each cap opening adds a median 1.07 S-TLF on 8 of 8 units, flat across a 15-fold spread in pedestal, so the step is additive and it is compared across events in S-TLF rather than in percent. Ranked on that scale, median over the eight units, 2 to 8 min after against the 6 min before (daily-calibration test, run 7):

What orders the four is how completely the water at the window is exchanged. The out-and-back step matches the burn-in figure for handling a submerged sensor, about +14 %, while the same handling done entirely in air moved the burn-in fleet a median −0.06 % (n = 22).

How it behaves

• It saturates. The same submerged cap removal repeated an hour later moved the fleet −0.7 % with 2 of 8 units up. The blank-to-test transfer that stepped +1.50 S-TLF (+8.1 %) in runs 1 to 5 and +2.29 (+11.8 %) in run 6 came in at +0.03 (+0.4 %) in run 7, which followed a cap opening. On 1 October a shake and then a twist under water released +1.85, +1.81 and +1.62 between them on the three low-background units, the same total as the out-and-back step, whatever share the shake took. The simplest reading is a reservoir of roughly fixed size per unit.
• The raised level is flat for the first one to two hours, then decays over about ten to fourteen hours at constant temperature to a baseline that repeats within 2 to 10 % morning to morning, on both overnights the units spent in DI.
• It is optical. Nine Chl-a units with the same board, SiPM and bias sweep sat in the same bath on 1 October and were shaken and twisted at the same instants. Their signal chain registers no light from the water, and they showed no step. The added signal on TLF units scales 1 : 4 : 16 with LED drive and leaves the no-gain pedestal unchanged, so it is LED light returned from in front of the window.
• Temperature does not produce it. The 25 September cap removal steps +8.4 % with the quench correction off and +7.4 % with it doubled; faking it would take 5.1 °C of cooling against 0.30 °C measured.

Ruled out: SiPM saturation and slow recovery (the step holds flat for 40 min where recovery would decay), temperature, photobleaching of the cavity water (a removal in 50 ppb gave no larger step), anything carried in the cavity water such as particulates, a gas film or a thermal gradient (a shake exchanges that water and gives +0.3 to +0.5 against +1.2 to +1.9 for opening the joint), air carried in on reinstall (8 of 8 units, every time, at the same size, with de-bubbling every time), and cap orientation (caps keyed since run 5, steps unchanged).

Still standing: stress relaxation in the compressed elastomer of the cap joint, the window or filter shifting on its mount, and a pinned capillary film at the cap-to-bezel contact line. Pressing the cap under water without turning it separates the first two from the third; the same disturbance in a 10 °C and a 25 °C bath separates elastomer creep from a film; a dial indicator on the cap and window over 10 h says which surface moves.

2After per-sensor correction, does the barrel baseline go flat?

The barrel is the one record where all four batches sit in the same water at the same time, and its 20 to 21 August heater-then-chiller ramp separates temperature from elapsed time. The question is whether, after correcting each sensor for quench, thermal lag and drift, what remains is flat.

Leaving out Chl-a, raw scatter of — falls to — once temperature is removed and to — once drift is removed as well.

The same model run over the burn-in campaign (Stage 10) passes 24/24 units in the cold hold and 23/24 in air, windows spanning 0.3 and 3.4 °C, but only 7/25, 5/25 and 12/24 on the wide-swing outdoor windows a deployed sensor lives in.

3Temperature is correctable on each unit's own coefficient

On the burn-in fleet each sensor's coefficient is the same in water and in air and across two weeks, when it is fitted on 5 h windows. Fitted jointly with a linear time slope over 72 to 77 h windows the coefficients look unstable (between-window correlation +0.014), because drift is not linear and a long window folds its curvature into the coefficient. Short windows remove that.

Quench is fitted per sensor, on that unit's own sipm_temperature, and no fleet or batch constant is used anywhere. Where a window spans too little temperature to identify a coefficient, it also carries too little quench to need one: under 0.5 °C is worth less than 1 % at a typical −1.7 %/°C.

It does measurable work in the calibration. On the eight drift-cal units the coefficients run from −0.89 to −2.61 %/°C, and window temperatures across the seven runs span 15.9 to 28.3 °C. Switching the correction off moves the 50 ppb reading from 98 to 92 % and the median error from 4.63 to 7.56 ppb.

4Drift is mostly settling after a disturbance

Most of what this program has called drift is a transient following something that was done to the unit, and each kind of disturbance has its own timescale:

The burn-in record, read in that light

The burn-in drift windows are bounded by events and opened 2 h after each one, so every one of them starts inside the relaxation that follows a handling event. Cut that way, — of — windows are separable (time and temperature not collinear) and drift is estimable per sensor; the two that are not are the post-UV window and the barrel tank phase. Earlier windows that straddled several interventions measured the steps: “Water, days 2 to 5” crossed the UV dose, the blackout caps, the basin carry and an in-place shake, and read −13.56 %/day against a half-window disagreement of 20.74.

Temperature is fitted jointly per unit, so quench cannot leak into the time slope. w is how many separable windows a unit contributed and rel how many pass the split-half check. Immersion is 16:45Z, where ToF steps 140 → 46 kcps in all three batches at once; local time is UTC−6.

Air against water, and the batch spread

The burn-in fleet is the only batch measured in both media, so its two rows are the same sensors under the two conditions. Air is the only medium in which all four batches have data.

Which windows carry a number

A joint fit means something only where time and temperature are separable inside the window. |corr| above 0.9 means the two regressors are collinear, so the window is reported as not separable.

What it looks like

The same 24 burn-in units in both panels, 20 h apart, each temperature-corrected and normalized to its own first settled hour. The line is the fleet median, the band the interquartile range.

In water the fleet falls about 1.3 % over 44 h with a narrow band, so the units move together. In air the median barely moves and the band straddles zero.

An uncontrolled window

The same units in water on a 15 °C diurnal swing. Per-unit rates spread from −15.7 to −1.1 %/day across the interquartile range, so no single drift number describes the fleet.

Bars are the drift each window reports; the marker is how far the two halves of the same window disagree. Where the marker exceeds the bar, the number measures nothing.

How long a baseline holds on these windows

Error against a 6 h baseline as a function of the baseline's age, on the burn-in windows.

On these windows a baseline is within about 2 % at 6 h and 15 % off by 12 h. That is the span of the post-handling relaxation, and each window opened 2 h after an intervention. On an undisturbed, settled unit the zero holds far longer (above); what limits a stored zero is the next disturbance.

5Gain changes slowly, and offset and gain are both required

Four corrections sit between a raw sweep and a number, and each does a different job.

The amplitude is a ratio to each unit's own reference sweep, so it normalizes a unit against itself and leaves every unit near 1 at its reference state regardless of how many counts per ppb it delivers. Only a per-sensor gain makes two sensors comparable. On the 16 July dilution ladder, 13 sensors, the gain spans 10.6× and the blank 2.2×; across the wider burn-in fleet the gain spans 27×. One shared offset and gain in place of per-sensor values takes R² from 0.815 to 0.120 and RMSE from 7.14 to 15.58 ppb (scripts/dilution_feature_compare.mjs). The blanks fall into two groups, about 0.52 to 0.64 and 0.96 to 1.12, matching a two-tier gain structure (about 93 to 118 against 404 to 983). One unit, 50091, has a negative fitted gain (−0.00054 at R² 0.30). The amplitude estimator ties absolute mon2 on well-behaved units (pooled R² 0.9087 against 0.9097 on the 12 both fit) and rescues the hard ones: it fits 13 of 14 where mon2 fits 12, with 50091 going from 0.4835 to 0.8131.

How the gain moved

Every cap opening the eight drift-cal units have had is logged, so 6 August to 22 September is 47 days with no handling. A gain is the difference of two levels taken with no cap opening between them, so an additive step cancels out of it; an offset is a level and carries the step.

• The five units that ran continuously rose +4 to +42 %, a fleet-wide shift of about +15 %. 500128, powered down for four of those weeks, is the only one whose gain fell, by 25 %.
• That the gain tracks sample cycles is not established. The correlation with cycles is 0.84 taking run 1 as the current gain and 0.20 to 0.94 taking any other run; among the five continuous units it runs −0.62 to +0.55. Cycles span only 1.11-fold among those five, so 500128 is the single low-cycle point. A different tryptophan stock, dosing in place in August against bath transfers in September, and temperature were all uncontrolled between the two dates.
• Handling moves the offset and leaves the gain alone: within the September week the gain shows no systematic trend (+1 % per run, mixed signs, r² near zero).

An aging term on the stored gain works, and is not yet a physical rate. Aging each unit's stored gain on its own sample-cycle count, refitted leave-one-unit-out and at every reference run, beats the alternatives on 7 of 8 choices: 6.02 ppb median error against 7.67 with one fleet scalar and 9.67 with no correction. The fitted rate swings from −0.08 to +6.65 % per 10,000 cycles with the reference run and changes sign at run 5, so what it reliably does is remove a fleet-wide shift and treat 500128 differently. A third commissioning ladder per unit, spaced in cycles, would test the shape.

6The procedure these findings lead to, and what it reaches

Commissioning, once per sensor. The unit goes into a vessel of DI and does not come out, and the cap does not come off. Read DI, spike the same vessel to 5 ppb and read, spike to 50 ppb and read. Gain is (S-TLF at 50 ppb − S-TLF at DI) / 50 on the unit's own quench coefficient; repeat the ladder two or three times, store the median, and store the unit's cycle count with it.

Every run. Read DI, then the sample, with nothing touched between them: no cap opened and the unit not lifted. ppb = (S-TLF_sample − S-TLF_DI) / gain, with the stored gain aged to the current cycle count and every term temperature-corrected on the unit's own coefficient. A new or re-immersed low-background unit needs about a week in water before its zero is stable enough to store.

Scored on the identical 47 unit-runs every arm can reach, run 1 excluded (no arm is scored against its own gain) and 500227's run 4 ladder excluded (that unit never saw the 50 ppb solution: its reading is its 5 ppb level plus one handling step).

Reproducibility is set by the reading. Every procedure row lands at 17 to 18 % day to day whatever the gain source, because dividing by a constant cannot change a coefficient of variation; the daily calibration sits at 23 % because its divisor is re-measured every run. Nothing clears the 10 % bar the protocol set. The gain's age shows up in accuracy instead: 4.2 ppb on a gain from the same week, 6.5 on one seven weeks old, 5.6 once that old gain is aged on cycles.

Per unit, seven of eight read 91 to 111 % at 50 ppb with 2.4 to 5.6 ppb median error. 500128 reads 125 % with 13.4 ppb (section 8).

7Bubbles need supersaturated water

Pulling a vacuum on a sealed container of DI grows something on the optical window, and releasing the pressure removes it. 500226 reached 13.4× its baseline under vacuum and returned to 0.93× when the container was opened.

Temperature is excluded: the SiPM moved at most 0.17 °C across the vacuum, worth at most 0.4 %, against +1,238 %.

7bThe degassing rig

Five units, 500220, 500225, 500226, 500108, 500146, in a sealed pressure container of DI, in at 09:29 local on 20 August with the vacuum applied at 10:50. The 81 min between them gives each unit an undisturbed water baseline, so a move at the vacuum line is attributable to the gas.

The vacuum steps

The degassing record

The record begins at immersion (09:29), so there is no dry-air record and no air-to-water step here. Each unit is normalized to its own median over the 30 min before the vacuum, so 1.0 is the settled in-water baseline. Rows the out-of-water detector flags are kept: at immersion that flag is the transition.

What each intervention showed

The tank blocks the cellular uplink while sealed, so readings buffer on the device and back-fill when it is opened; the export is keyed on database row id so late rows with old timestamps are collected in full.

8The four batches, and units that look like hardware

Three optics builds from the August 2026 variant run (FDOM, Chl-a, TLF) and the older burn-in fleet.

Temperature response differs between builds

Drift in air, by batch

Same window for all four (08-19 to 08-20, in air, temperature-corrected). The reliability column is the split-half check.

The medians differ, Chl-a steepest at −2.38 %/day and the burn-in fleet nearly flat at −0.28, but no batch passes the split-half check, and batch is confounded with age (the variant builds were 1 to 2 days old, the burn-in fleet 13 to 15) and with time since handling, so a batch difference in drift is not established.

Response to the barrel shake, by batch

All units shaken at one instant in shared water, temperature-corrected; 41 have enough readings either side to score.

The FDOM, new TLF and burn-in medians sit within 1 % of zero, and no unit moved more than 5 % on both channels. Twelve units moved more than 5 % on S-TLF with ToF unchanged; seven of them rose, the direction of the handling step (section 1).

Units that look like hardware

Every unit in the barrel is fitted and reported, and each is scored against its own batch, never the fleet. A unit is listed when it exceeds 4 MAD above its batch on an axis or trips an absolute fault rule; three or more independent axes is rework, two investigate, one watch. The barrel ran a long temperature ramp rather than a hold, so corrected residual is used only as a tie-breaker.

What is still open

• What releases the light on handling, and how long the reservoir takes to rebuild. The three tests in section 1, and one submerged cap removal repeated at 30 min, 1, 2, 4 and 8 h with nothing else touched.
• The temperature residual. Split the fleet, four units blank-capped and four with pucks in one bath, ramp and soak across the widest range the bath reaches, swap caps and repeat, and fit ln(S-TLF) on sipm_temperature per unit per state. That measures the pedestal coefficient over a deliberate range and the span's own temperature response for the first time; it does not give tryptophan's coefficient.
• How fast the gain moves, and against what. A ladder repeated at 2,000, 5,000, 10,000 and 20,000 cycles, dosed in place with no handling, would set the recalibration interval and test whether cycles are the variable.
• Whether a submerged puck fit holds water against the window. In run 7 the span was 33 % smaller; subtracting each unit's own step reproduces most of it (median factor 1.15, range 0.91 to 1.81), and water held between puck and window would do the same. One submerged run cannot separate them.
• Why the Chl-a batch returns no light, in DI and in rhodamine.
• Whether the four remaining new-TLF units are sound. They sit above every other batch on corrected residual; a steady hold rather than a 27 °C ramp separates build from window.
• The gap between bench and field. 4 to 43 CFU/100 mL on the bench against 100 in the field.