Khalid Zafer Ahmed Alkalali. Real-time atmospheric refraction coefficient determination using ERA5 reanalysis data for line-of-sight visibility and geodetic applicationsJ. Geodesy and Geodynamics. DOI: 10.1016/j.geog.2026.08.009
Citation: Khalid Zafer Ahmed Alkalali. Real-time atmospheric refraction coefficient determination using ERA5 reanalysis data for line-of-sight visibility and geodetic applicationsJ. Geodesy and Geodynamics. DOI: 10.1016/j.geog.2026.08.009

Real-time atmospheric refraction coefficient determination using ERA5 reanalysis data for line-of-sight visibility and geodetic applications

  • Atmospheric refraction is a fundamental source of systematic error in geodetic surveying, EDM, and line-of-sight visibility analysis. Practitioners assume κ = 0.13, a mid-twentieth-century average inappropriate for most conditions. This system computes κ directly from ERA5 reanalysis data (137 model levels) via the Copernicus Climate Data Store API, applying the Smith-Weintraub approximation to the ITU-R P.453-14 refractivity formula, including the water-vapour wet term, regressing the retrieved profile to derive dN/dh, and computing κ = -RE×dN/dh×10-6. Spherical-Earth geometry determines ray clearance and visibility, with the minimum required κ found via binary search. Additional modules compute EDM corrections, leveling curvature-and-refraction corrections, and refraction-independent height differences from reciprocal zenith angles, across four operational modes. Validation on the Sağrak-Shkhara path (493.26 km) yields κ = 0.1215 (ICAO dry baseline), 0.1300 (37 pressure levels, Δκ = +0.0085), and 0.1467 (L137, Δκ = +0.0252), verified against four WMO radiosonde soundings (mean κ = 0.1369, within 6.7% of ERA5). All configurations yield κ below the required visibility threshold, yet the path holds the Guinness World Record for the longest photographed line of sight. This reflects ERA5's 0.25° resolution and data quality, not a computational error. Computing refractivity from reanalysis data is already established in the GNSS-troposphere and radio-occultation literature; the contribution here is integrating the P.453 wet-term computation, spherical ray geometry with a required-κ search, and multiple geodetic correction modules into one operational pipeline. Four further paths test the method: Pic de Finestrelles-Pic Gaspard (443 km, κ = 0.1458 vs. required 0.1373) and a nocturnal Świętokrzyskie-Warsaw inversion path (145 km, κ = 0.2235 vs. required 0.1670) are correctly predicted near threshold; Karagöl-Elbrus (474 km) and Cerro Champaquí-Aconcagua (483.5 km) are predicted as wide-margin cases, the latter distinguishing geometry-limited from transparency-limited visibility. These five paths span four verification methods, seasons, hemispheres, and climatic regions.
  • loading

Catalog

    Turn off MathJax
    Article Contents

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return