Honest, Leakage-Free Operational Earthquake Forecasting: A Multi-Region CSEP Testbed with Pre-Registered Negatives and the Horizon-Dependent Value of Geodetic Context
Version 2.1 (revised). Operational earthquake forecasting (OEF) issues calibrated conditional probabilities of future seismicity; the Epidemic-Type Aftershock Sequence (ETAS) model is its de-facto benchmark, and under fair, prospective, CSEP-style testing no machine-learning temporal point process has been shown to beat a well-fit ETAS. This preprint describes a complete, honesty-first OEF system and, on it, a leakage-free multi-region CSEP testbed built to answer, rather than assert, four questions. The system fits a regime-tiled space-time ETAS with a full hygiene pipeline (rolling magnitude of completeness, moment-magnitude homogenization, dual-catalog declustering, propagated uncertainty), against a mandatory adaptive smoothed-seismicity Poisson null and a transparent Reasenberg-Jones fallback, calibrated by isotonic regression with a genuine epistemic-plus-aleatory uncertainty triad, and it emits both gridded and catalog-based forecast representations so over-dispersion is scored honestly. Skill is established only by winning CSEP comparison tests (information gain per earthquake, IGPE, in nats) against both baselines, under a strict forecast clock that engineers against five leakage modes, with pre-registered ship rules and shuffled-label negative controls. Four artifact-backed findings result, each with its honest negative kept on the record. (i) ETAS adds skill over the stationary null only where there is triggering to exploit (Japan +0.072 nats at one day; low-seismicity interiors exactly 0.0), quantifying the high-versus-low-seismicity bias. (ii) A convex log-score-optimal stack of ETAS-family variants earns a real, significant global gain (+0.011 over 751 events) that does not generalize to the canonical active margins; under the pre-registered rule it is a no-ship, and a temporally adaptive variant is refuted as a multiple-comparison artifact. (iii) The binding one-to-seven-day consistency failure is count over-dispersion, not spatial shape: a catalog-based number test passes a window the Poisson number test rejects, and the frozen-intensity forecast systematically under-counts by about 28 percent because it omits within-window secondary triggering. (iv) The value of a GNSS-strain geodetic context covariate, added to a Hawkes-structured neural point process, is horizon-dependent: it does not beat ETAS at the one-to-seven-day operational horizon (mean IGPE -0.053 over eight weekly windows) but beats it robustly at thirty days (global +0.115 over 2166 events, positive in 9/10 windows and in every high-seismicity region), because the calibrated model deploys a time-flat geodetic background rather than triggering. We conclude that base tiled ETAS is at or near the practical ceiling for mean-rate one-to-seven-day IGPE over global M>=5, and that the honest place for a geodetic covariate is a longer, background-dominated outlook. The manuscript includes the full methodology and equations, the leakage-free evaluation protocol, complete per-region and per-experiment results, five purpose-driven figures generated deterministically from the committed artifacts, and appendices. This is an independent research and education tool; it is not an operational alarm system and must not be used for life-safety decisions. Code, configurations, provenance manifests, and all committed artifacts (MIT): https://github.com/fsantibanezleal/CAOS_SEISMIC . Static forecast viewer: https://seismic.fasl-work.com .