Introduction

As light travels through the Earth’s atmosphere, because the atmospheric vertical density decreases continuously from the bottom up, it is refracted, meaning that the trajectory of light is not a straight line, but a curve that bends toward the center of the Earth1,2. Using the atmospheric refraction properties of light, the following previous studies have been carried out. Based on the refraction angle and atmospheric refraction model with high accuracy, the apparent height with higher accuracy can be calculated, so that the accuracy and reliability of autonomous satellite navigation can be improved3. In addition, the high precision calculation for the atmospheric refraction can improve the accuracy of locating and measuring air targets by radar4 and the positioning accuracy of ground objects by the satellite imaging5. Lunar eclipses, stellar occultation observations, and exoplanet transits are also affected by atmospheric refraction to varying degrees, such as distinguishing between cloudy, clear, and hazy skies for the exoplanet atmospheres using refraction6,7. It is found that the high precision calculation of atmospheric refraction is widely used in the field of atmospheric detection, engineering technology and astronomy. Therefore, in order to obtain a more accurate starlight refraction Angle in the range of 0–90\(^{\circ }\) from apparent zenith, it is necessary to develop a new mathematical model, and adopt a more advanced semi-empirical atmospheric parameter model to provide corresponding atmospheric temperature and pressure data

In current studies to calculate atmospheric refraction, the atmosphere is often treated in two layers, one is the troposphere, which assumes that the temperature decreases at a constant value with increasing altitude, and the other layer is the stratosphere from the tropopause to a certain altitude, where the temperature is constant1,5,8,9. The simplified method of dividing the atmosphere into two layers, which is commonly used, often introduces large errors in calculating refraction angles and makes it difficult to calculate atmospheric refraction at observed altitudes above the stratosphere. By comparing the calculated values of refraction angles under various spherical atmospheric models, it is found that the difference between refraction angles becomes larger with the increase of apparent zenith distance10. Under the condition of large apparent zenith distance (75–\(90^{\circ }\)), the calculation result of atmospheric refraction angles is more uncertain8,11. In addition, in the current research, semi-empirical atmospheric parameter models such as the United States Standard Atmosphere (USSA) and COSPAR International Reference Atmosphere (CIRA) are often used to obtain atmospheric parameter information, and then obtain the calculation results of the starlight refraction Angle. However, the above semi-empirical atmospheric models have a long history and low accuracy, resulting in a large error in the calculation results12,13. At the same time, the National Centers for Environmental Prediction (NCEP) of the United States, the European Centre for Medium Range Weather Forecasts (ECMWF) and other institutions also provide the measured grid data of the Earth’s atmospheric parameters on different time scales, based on the above data, the mathematical model of starlight atmospheric refraction is also constructed, and the change law of starlight refraction Angle under different spatial and temporal coordinates under different environmental parameters is given14,15. However, this data is the average value in the grid area, and the calculation result also has great uncertainty. Therefore, in order to obtain a more accurate starlight refraction Angle in the range of 0–\(90^{\circ }\) from apparent zenith, it is necessary to develop a new mathematical model, and adopt a more advanced semi-empirical atmospheric parameter model to provide corresponding atmospheric temperature and pressure data.

In this paper, we present a method for calculating the atmospheric refraction angle for all apparent zenith distance (0–\(90^{\circ }\)) and any observed altitude. Assuming that the Earth’s atmosphere conforms to the spherically symmetric structure (Local spherical symmetry assumption, only applied to the spherical symmetry structure for the light transmission path), the atmosphere is stratified according to the constant height \(\Delta h\), and the stratified height of each atmosphere layer is obtained. Based on the NRLMSIS 2.0 model16,17 and the ideal gas law, the temperature and pressure values at the height of each layer are given, so as to obtain the atmospheric refraction index of each layer. The e-index model is used to fit the atmospheric refraction index profile, and then the refraction index gradient of each layer is obtained by the best fitting e-index model. Then, Snell’s law is used to calculate the apparent zenith distance at each layer height along the path of light. Finally, the atmospheric refraction angle is obtained by the integral of atmospheric refraction.

Calculation and modeling of ray bending angle

Fig. 1
figure 1

Observational geometry of atmospheric refraction of light. The apparent and real positions of the observed objects are marked in the figure. Bending error \(\alpha\), apparent zenith distance \(Z_0\) at the observation position, and zenith distance at the ith atmospheric layer are also shown in the figure.

Full size image

The observed geometry of light atmospheric refraction is shown in Fig. 1. The apparent and real positions of the observed objects are marked in the figure. Point O is the center of the Earth. The red dot represents the observation position, the distance from the surface is \({h_0}\), and \({h_\textrm{T}}\) represents the height range where atmospheric refraction occurs, the value is set as 200 km in this paper, that is, the refractive index is always 1 above 200 km. \(Z_0\) is the apparent zenith distance at the observation position, and \(Z_i\) is the zenith distance at the path of light propagation at the ith atmospheric layer. \(\alpha\) is the bending error of the light, which is the refraction angle we want to calculate. It is the angle between the tangent line at the observed position and the tangent line at the target position as the light travels through the atmosphere, as shown in Fig. 1. The bending error of light passing through the atmosphere is generally calculated by the refraction integral, as shown in the following Eq. (1),

$$\begin{aligned} \alpha = \int _1^{n_0}\frac{\textrm{tan} Z }{n}dn \end{aligned}$$
(1)

where the lower limit of integration 1 represents the refractive index outside the atmosphere, the upper limit of integration \({n_0}\) represents the refractive index at the observation position, and Z is the zenith distance. Equation (1) has a deficiency, that is, when Z approaches \(90^{\circ }\), the integrand is infinite, which makes the result of integration difficult to calculate. We refer to the operation in reference8 to get the deformed form of Eq. (1), which avoids the problem of \(\textrm{tan} Z\) approaching infinity by eliminating Z in the integrand,

$$\begin{aligned} \alpha = -\int _0^{Z_0}\frac{r \frac{dn}{dr}}{n+r \frac{dn}{dr}}dZ \end{aligned}$$
(2)

where \(Z_0\) is the apparent zenith distance at the observation position, r is the distance from the point on the path of light to the center of the Earth, n is the refractive index of the atmosphere, and \(\frac{dn}{dr}\) is the refractive index gradient. Equation (2) is numerically calculated by the trapezoidal rule, as follows,

$$\begin{aligned} \alpha = \frac{1}{2}\sum _{i=1}^{Num-1}(Z_{i+1}-Z_i)(f_i + f_{i+1}) \end{aligned}$$
(3)

where \(f_i = \frac{r_i \frac{dn_i}{dr_i}}{n_i+r_i \frac{dn_i}{dr_i}}\), i represents the ith atmospheric layer, and its values are 1, 2, 3... Num-1, where \(Num = \frac{h_{\textrm{T}} - h_{0}}{\Delta h}\). In this paper, \(\Delta h\) as the constant height of two adjacent atmospheric layers is taken as 0.1 km. The distance \(r_i\) from the ith layer to the center of the earth is \(R + h_0+ i\cdot {\Delta h}\), where R is the radius of the Earth. \(n_i\) is the atmospheric refractive index at the ith atmospheric layer, which is calculated by the following equation18 in this paper,

$$\begin{aligned} n=1+\frac{(n_g - 1)P}{101325(1+m t)} - \frac{5.5\cdot 10^{-8}\cdot P_{H_{2}O}}{133.3224(1+m t)} \end{aligned}$$
(4)

where \(m=\frac{1}{273.16}\), t, P, \(P_{H_{2}O}\) represents atmospheric temperature, pressure and vapour pressure, and the unit is \(^\circ {\textrm{C}}\), Pa, Pa respectively, \(n_g = 1+(2876.04 + \frac{3\cdot 16.288}{\lambda ^2}+\frac{5\cdot 0.136}{\lambda ^4})\cdot 10^{-7}\), where \(\lambda\) is wavelength, and the unit is \(\mu\)m. It is found that the calculation of refractive index is related to atmospheric parameters such as temperature, pressure and water vapor pressure. The contribution of water vapor pressure to atmospheric refraction is too small to be considered in this paper. The NRLMSIS 2.0 model is used to calculate the temperature and total mass density at each layer height of the atmosphere, and then the atmospheric pressure at each layer height of the atmosphere is given by combining with the ideal gas law. The profile of atmospheric temperature and pressure within the altitude range of 0–200 km calculated by NRLMSIS 2.0 is shown in Fig. 2. Panel (a) and (b) show the atmospheric temperature profile and pressure profile in the altitude range of 0–200 km based on the NRLMSIS 2.0 model respectively. Thus, the atmospheric refractive index \(n_i\) of each atmospheric layer can be calculated based on the temperature and pressure profiles in Fig. 2 and equation (4).

Fig. 2
figure 2

Panel (a) shows the atmospheric temperature profile in the altitude range of 0–200 km calculated by the NRLMSIS 2.0 model. Panel (b) shows the atmospheric pressure profile in the altitude range of 0–200 km based on the NRLMSIS 2.0 model and ideal gas law.

Full size image

In addition, in order to obtain the bending error \(\alpha\), it is also necessary to solve the zenith distance and refractive index gradient at each layer of the light’s propagation path in the atmosphere. Based on Snell’s law applied to spherically symmetric atmospheres, the zenith distance of light in each atmospheric layer can be obtained as follows,

$$\begin{aligned} n_{0}r_{0} {\textrm{sin}}Z_{0} = n_{i}r_{i}{\textrm{sin}}Z_{i} ={\textrm{constant}} \end{aligned}$$
(5)

where \(n_0\) is the refractive index at the observed position, \(r_0\) is the distance from the observed position to the center of the Earth, \(Z_0\) is the apparent zenith distance from the observed position, \(n_i\) is the refractive index at the ith layer, \(r_i\) is the distance from the ith layer to the center of the Earth, \(Z_i\) is the zenith distance of the light at the ith atmospheric layer.

The e-index model is used to fit the refractive index profile based on the NRLMSIS 2.0 model, and the relation between refractive index and geocentric distance is obtained. The specific form of e-index model is as follows,

$$\begin{aligned} \left\{ \begin{array}{lr} N(r)=N_0\cdot ^{\beta (r-r_0)} & \\ n(r) = 1+ N(r)\cdot 10^{-6}& \\ \end{array} \right. \end{aligned}$$
(6)

where \(N_0\) is the refractivity at the observation position, and the relation between refractivity and refractive index is \(N_0 = (n_0 - 1)\cdot 10^{6} (\textrm{ppm})\), \(\beta\) is an unknown parameter, which is obtained by fitting. In this paper, the nonlinear least square fitting method is used to solve the unknown parameter \(\beta\), and the specific realization uses the nlinfit function of MATLAB.

Fig. 3
figure 3

Panel (a) shows the comparison between the refractivity based on NRLMSIS 2.0 and the best-fitting e-index model in the altitude range of 0–200 km. Panel (b) shows the comparison between the refractivity based on NRLMSIS 2.0 and the best-fitting e-index model in the altitude range of 30–200 km. Panel (c) shows the comparison between the refractivity based on NRLMSIS 2.0 and the best-fitting e-index model in the altitude range of 60–200 km.

Full size image

Discussion

Previous studies usually divide the Earth’s atmosphere into troposphere and stratosphere. The temperature of the troposphere decreases with height according to a certain gradient, while the temperature of the stratosphere remains constant. The pressure is written in a form related to height, so as to calculate the refractive index and give the refractive index gradient8,9,19. In this paper, the values of the refractive index gradient \(\frac{dn_i}{dr_i}\) at each atmospheric layer in Eq. (3) are given by the fitted e-index model. Panel (a), (b) and (c) in Fig. 3 show the comparison of refractivity based on NRLMSIS 2.0 and the best-fitting e-index model at the observed altitude of sea level, 30 km and 60 km from the surface respectively. The best fitting values of \(\beta\) are − 1.1078\(\times 10^{-04}\), − 1.5056\(\times 10^{-04}\), − 1.3823\(\times 10^{-04}\), respectively. NRLMSIS 2.0, launched in 2020, is an upgraded version of NRLMSISE-00, which can output the temperature, total mass density and number density profile of atmospheric components from sea level to the Earth’s thermosphere. The model can output atmospheric parameter values with high accuracy, and the atmospheric parameter values are in good agreement with the measured results of other means20,21,22. The input parameters of the NRLMSIS 2.0 model are as follows: the date and time are 2022-12-07T12:00:00, the geographical latitude and longitude of the observation target source are (\(22^{\circ }\) E, \(6^{\circ }\) N), the solar activity index F107 is 60 sfu, and the geomagnetic activity index Ap is 10 nT.

According to the input parameters of the NRLMSIS 2.0 model above, the optical bending errors when the apparent zenith distance is within the range of 10–\(90^{\circ }\) and the observed height \(h_0\) is 0, 30 km and 60 km are calculated based on Eq. (3), as shown in Table 1. It is found that the bending error of light increases with the increase of the apparent zenith distance, and the bending error increases faster when the apparent zenith distance is close to \(90^{\circ }\). At the same time, it is found that the bending error of light decreases with the increase of observation height, because the higher the observation height, the thinner the atmospheric concentration, the shorter the propagation distance of light in the atmosphere.

Table 1 The calculation results of bending error of light rays, where the wavelength of light \(\lambda\)=0.58 \(\upmu\)m (unit: arcseconds).
Full size table

Tables 2 and 3 show the comparisons between Hohenkerk’s calculation results8 and those of the proposed method. Table 2 is the comparison of the bending errors under the condition of large apparent zenith distance, and Table 3 is mainly for the condition of zenith distance less than \(70^{\circ }\). It is found that although there is a difference between the two results, the difference does not mean that the proposed method is not feasible. The maximum difference between the two results is 33.79 arcseconds when the zenith distance is \(90^{\circ }\), and the minimum difference is 0.22 arcseconds when the zenith distance is \(10^{\circ }\). The difference between the two results is caused by the systematic error of the method itself. In addition, the input parameters are not completely consistent, such as temperature and pressure. But it also shows the feasibility and reliability of the method proposed in this paper to calculate the optical bending error.

Table 2 The comparison between Hohenkerk’s calculation results and those in this paper, for the large apparent zenith distance, \(\lambda\)=0.50169 \(\upmu\)m and \(h_0\) = 0. (unit: arcseconds).
Full size table
Table 3 The comparison between Hohenkerk’s calculation results and those in this paper, for the apparent zenith distance is less than \(70^{\circ }\), \(\lambda\)=0.574 \(\mu\)m and \(h_0\) = 0. (unit: arcseconds).
Full size table
Table 4 The comparison of optical bending errors at \(h_{\textrm{T}}\)=200 km and \(h_{\textrm{T}}\)=300 km, where \(\lambda\)=0.58 \(\upmu\)m and \(h_0\) = 0. (unit: arcseconds).
Full size table

The above results are all calculated when \(h_{\textrm{T}}\), which is the height of refraction, is 200 km. In order to verify the rationality of this value, the bending error is calculated when \(h_{\textrm{T}}\)=300 km, as shown in Table 4. It is found that even at the sea level when the apparent zenith distance is \(90^{\circ }\), the difference between the calculated bending error of \(h_{\textrm{T}}\)=200 km and \(h_{\textrm{T}}\)=300 km is only 0.00072 arc seconds. In Table 4, the minor discrepancies between the results for \(h_{\textrm{T}}\) = 200 km and \(h_{\textrm{T}}\) = 300 km (e.g.a difference of 0.00072 arc seconds at \(Z_0\)= \(90^{\circ }\)) are due to numerical precision limitations in the computational process, such as the finite step size (\(\Delta h\)=0.1 km) used in the trapezoidal integration and rounding errors. These differences are negligible (on the order of \(10^{-4}\) arc seconds) and do not affect the physical validity of the assumption that \(h_T\) = 200 km is sufficient for accurate calculations. Therefore, it is reasonable to assume that there is no atmospheric refraction when the altitude is above 200 km.

Conclusions

We present a new method for calculating the bending error of light rays for all zenith distance and any observation height. Assuming that the Earth’s atmosphere conforms to the spherical symmetric structure, based on the atmospheric parameters such as temperature and pressure provided by the NRLMSIS 2.0 model, the refractive index at each layer height of the atmosphere is given. The refractive index profile based on NRLMSIS 2.0 is fitted by nonlinear least squares algorithm, and the best fitting e-index model is obtained, so that the refractive index gradient at each atmospheric layer height is obtained. According to Snell’s law applied to spherically symmetric atmosphere, the zenith distance of light in each atmosphere is obtained. Finally, the bending error of light rays passing through the Earth’s atmosphere is obtained by numerical calculation of the deformation form of atmospheric refraction integral by trapezoidal rule. The bending errors of \(h_T\)=200 km and \(h_T\)=300 km are calculated respectively. It is found that even when \(Z_0\)=\(90^{\circ }\) and \(h_0\)=0, the difference between them is approximately \(10^{-4}\) arcseconds, indicating that it is reasonable to assume \(h_T\)=200 km. This method can not only calculate the bending error of light in the range of 0–\(90^{\circ }\) from apparent zenith, but also the bending error of light for any observation height. The feasibility and reliability of the proposed method are proved by comparing with the results of previous studies.