Terrain-based adaption of propagation model loss parameters using non-linear square regression

Abstract

Reliable and real-time propagation loss modeling play a significant role in the efficient planning, development, and optimization of macrocellular communication networks in a given terrain. Thus, the need to adapt or tune an existing model to enhance its signal prediction accuracy in a specified terrain becomes imperative. In this paper, we proposed and applied a non-linear square regression method based on the Levenberg-Marquart (LM) algorithm to adapt and improve the empirical propagation loss estimation accuracy of the Egli model for two major cities in Nigeria. A comprehensive propagation loss measurement acquired over Long Term Evolution (LTE) mobile broadband networks operating at 2630 MHz for four different cities was collected using TEMS investigation tools to achieve the Egli model adaption. Results indicate that the adapted Egli model displays a high estimation accuracy over the Gauss-Newton (GN) algorithm leveraging the non-linear regression method employed to benchmark the propagation loss estimation. Using six standard statistical indicators, the adapted Egli model displayed lower estimation errors than the classical Egli model across the tested locations in the two cities investigated. Finally, the LM-adapted Egli model was compared with extensive measurements from another eNodeB in Port Harcourt different from the initial four eNodeBs investigated. The results indicate that the adapted model is suitable for deployment in related macrocellular environments.

Introduction

Radio wave propagation in typical terrestrial terrains is a stochastic phenomenon whose characteristics vary in time due to atmospheric conditions and in space due to other clutter and environmental obstructions [1]. Primarily, this is the situation at the ultra-high frequency (UHF) radio propagation domain wherein a clutter of physical buildings, trees, hills, human activities, and vehicular movements, including the ever-constant changing and uncontrollable atmospheric conditions, dominate [2, 3]. The resultant effect of this stochastic phenomenon is large-scale attenuation and reduction of the propagated signal from radio waves between the transmitting and receiving channels [4]. Such reduction in strength or power of the propagated radio signal over a communication channel is known as propagation path loss [5,6,7].

Over the past few decades, several researchers have attempted to develop different models that can reliably estimate the signal propagation path loss over two or more radio communication terminals [8, 9]. Such reliable path loss estimations play a crucial role in the proper planning, development, and management of any wireless communication systems networks [10, 11]. The problem with the existing models is that none can be generalized to all environments, urban, rural, or suburban terrain. All existing models were developed to estimate path loss at a particular operating frequency in a specified environment [2]. One of such terrain-based models is the Egli propagation model developed from terrain-based measurements [12].

In order to adaptively modify an existing propagation model parameters to fit into practical field loss data, the use of least-squares (LS) regression has generally been the thriving method reported in several works [13,14,15]. For instance, an LS regression approach is studied in [13] to fine-tune the Hata model offset parameters to estimate field-measured propagation loss in a typical urban terrain. The work in [11] presents an adaptation of propagation model parameters toward achieving efficient cellular network planning using a robust LAD algorithm. A comparative investigation of least square methods for tuning the Erceg pathloss model was presented [14]. In [15,16,17,18,19], the authors explored the LS method to study and modify the offset parameters of COST-231 [20], Erceg [21], Hata [22], and Walficsh-Bertoni models [23] to enhance their predictive capabilities on-field loss data in different radio signal propagation terrains. The results indicate a need for a more generalized propagation loss model to predict pathloss in terrain-based environments accurately.

The study [24] also adopted a recursive algorithm corresponding with LS regression to calibrate the Okumura-Hata model estimation capacity for CDMA networks. Further information concerning pathloss measurements and modeling in CDMA networks have been reported [25,26,27]. In [28], an adapted LS algorithm, which the authors termed “Minimax LS algorithm,” was employed for automatic-based tuning of the Ericsson model to fit field measurements. However, the problem with the LS regression is that it approaches its abysmal performance in modeling and handling high stochastic and non-linear propagation loss datasets [28]. The limited performances of the existing models present a considerable gap in the literature, and the need to fill this gap is not out of place.

This work aims to examine and adapt the parameters of the terrain-based Egli model for suitable signal loss estimation using the non-linear square regression method. The study presents comprehensive propagation measurements over LTE mobile broadband networks at 2630 MHz for four different cities in Port Harcourt and Uyo in Nigeria. Measured pathloss was obtained using TEMS investigation tools, and the adaption of the Egli model to the empirical propagation loss data is proposed. The resultant model would be of great relevance to wireless cellular operators or network companies for enhanced network planning and efficient management of telecommunication networks in the two cities under consideration and any other related wireless propagation environment.

The rest of this paper is organized as follows. The methodology is presented in the “Methods” section. This section comprises the field measurement environments, tools, and procedures. Additionally, the classical Egli propagation loss model, adaptation of the classical Egli propagation loss model, and statistical performance metrics are discussed in this section. The results and helpful discussions are presented in the “Results and discussions” section. The results reported include measured propagation loss estimation and error distribution statistics using the classical Egli model and adapted Egli model for the cells under consideration. The measured propagation loss estimation and error distribution statistics using the classical Egli and adapted Egli models for the cells tested are broached. The estimated coefficients and statistics for the cells being tested are highlighted briefly. Additionally, the computed first-order estimates statistics for the cells considered are outlined. The calculated first-order estimates statistics for the cells investigated are presented. The LM-adapted Egli model was validated. Finally, the “Conclusions” section gives a concise conclusion to the paper.

Methods

The block diagram shown in Fig. 1 provides the methodology upon which this work is guided. It starts by acquiring the eNodeB transmission parameters via a detailed site survey. After conducting the site survey, actual field measurements involving signal data were initiated. This was followed by the definition of the propagation loss model in focus, and the steps followed to adapt the model based on the field loss data using the non-linear square regression method.

Fig. 1

Block diagram showing the methodology of pathloss measurements and modeling

eNodeB transmission parameters

A site survey of the eNodeB transmission parameters of the commercial LTE network used in this work was conducted to obtain robust results. The parameters considered include the eNodeB total reference signal transmit power (P_tot), carrier transmission frequency (F_cf), antenna gain of the transmitter (G_t), the antenna gain of the receiver (G_r), transmit antenna height (h_B), cable loss (C_l), and feeder loss (F_l).

Field measurement environments, tools, and procedure

This section presents the measurements environment, tools and procedure, and RF network data measured.

Measurement environment

Field measurements were conducted in two close medium-sized cities, Uyo and Portharcourt, located in the south-south zone of Nigeria. Uyo is the capital city of Akwa Ibom State, situated on latitudes 04° 59′ N and longitudes 07° 53′ E. It has a population density of 1200/km² (3100/sq mi), with almost yearly warm and overcast long wet and short dry seasons. Port Harcourt is the capital city of Rivers State, located on latitude 4° 051′ N and longitude 7° 0′ E. It is the fifth largest congested city in Nigeria, with a population density of 1900/km² and a weather condition similar to Uyo. Both cities are built up with a mixture of commercial/residential building clusters and sparsely human/vehicular traffics. Also, the topography of both cities ranges from flat to horizontal plains. For more description of the cities investigated, a schematic map showing the tested eNodeBs is given in Figs. 2 and 3, respectively.

Fig. 2

A map showing eNodeBs locations in Uyo City Nigeria

Fig. 3

A map showing eNodeBs locations in Port Harcourt City Nigeria

Measurement tools and procedure

The tools explored for the data collection comprise TEMS investigation drive test software installed on Sony Ericson handsets and connected to an HP Laptop similarly to [29]. Other supporting tools include the GPS, inverter and connecting cables, and the drive test commercial vehicle. The TEMS investigation drive test tool is a real-time professional telecom service quality monitoring software. TEMS possesses the capacity to measure, record, and display different desired RF spatial and temporal datasets in log files along the drive test routes. In this investigation, the test equipment was locked to the operating frequency of 2630 MHz of the tested eNodeBs. This helps to mitigate the impact of co-channel interference from neighboring cells operating at different frequencies. Additionally, the test vehicle was driven, considering the actual road traffic conditions at a relatively medium speed of up to 30 km/h with uniformity, thereby reducing possible Doppler effects. The post-processing of measured log data files was analyzed using Map Info, MS Excel, MATLAB analytic tools.

RF network data measurements

The work concentrated on the Reference Signal Received Power (RSRP) measurements using LTE radio networks [30, 31]. Technically, the measured RSRP is a special LTE network indicator for assessing signal power and coverage level [32]. With the aid of the experimental drive test tools, the RSRP data sets in the log were collected around four randomly selected eNodeB transmission sites, with two from each city considered. The four eNodeB transmission sites belong to well-known GSM/UMTS/LTE telecom service providers operating in the two cities investigated. In terms of the propagation loss, the RSRP is defined by [33] in eq (1), and the total reference signal transmit power is defined in (2).

$$ \mathrm{RSRP}\ \left(\mathrm{dB}\mathrm{m}\right)={P}_{tot}\left(\mathrm{dB}\right)-\mathrm{Path}\ \mathrm{Loss}\ \left(\mathrm{dB}\right) $$

(1)

$$ {P}_{tot}\left(\mathrm{dB}\right)={G}_t+{G}_r-{C}_l-{F}_l-\left(10\log {N}_{rb}\right)-10\log (12) $$

(2)

where P_tot defines the eNodeB total reference signal transmit power, G_r is the receiver antenna, G_t is eNodeB antenna gain, and N_rb indicates the number of resource blocks. C_l and F_l = express the cable loss and feeder loss, respectively. Thus, in terms of propagation loss, the expression in (1) can be written as (3) and further expressed in (4). The basic parameters used for simulation are defined in Table 1.

$$ \mathrm{PathLoss}\ \left(\mathrm{dB}\right)={P}_{tot}\left(\mathrm{dB}\right)-\mathrm{RSRP}\ \left(\mathrm{dB}\mathrm{m}\right) $$

(3)

$$ \mathrm{Path}\ \mathrm{Loss}\ \left(\mathrm{dB}\right)={P}_{tot}\left(\mathrm{dB}\mathrm{m}\right)+{G}_t+{G}_r-{C}_l-{F}_l-\left(10\log {N}_{rb}\right)-10\log (12)-\mathrm{RSRP}\ \left(\mathrm{dB}\mathrm{m}\right) $$

(4)

Table 1 Propagation loss computation parameters

The classical Egli propagation loss model

Consider a propagated signal from a transmitter to the receiver. In free space, the received signal, P_r as a function of the transmission frequency f_t, transmit power P_tr, over a communication distance, d is given by [34] in eq (5):

$$ \frac{P_r}{P_{tr}}=\frac{G_{tr}{G}_r{\lambda}^2}{4{\pi}^2{d}_c^2} $$

(5)

where $\lambda =\frac{c}{f_t}$

The propagation loss in dB can be obtained from (1) as given in (6) and its simplified form in (7).

$$ Pl\ \left(\mathrm{dB}\right)=10{\log}_{10}\frac{P_r}{P_{tr}}=-10{\log}_{10}\frac{G_{tr}{G}_r{\lambda}^2}{4{\pi}^2{d}_c^2} $$

(6)

$$ Pl\ \left(\mathrm{dB}\right)=32+20{\log}_{10}\left({f}_{t(MHz)}\right)+20{\log}_{10}\left({d}_c\right)-{G}_{tr}-{G}_r $$

(7)

The expression in (5) is the popular free space path loss model [34]. In order to cater to the effect of receiver antenna height H_r and transmitter height H_tr, and typical terrain obstacles, which the authors in [34] discounted in formulating (7), Egli modified the free space model based on extensive practical signal measurement conducted at VHF/UHF transmissions in the USA, yielding eq (8):

$$ {P}_r={G}_{tr}{G}_r{\left(\frac{H_{tr}{H}_r}{d_c^2}\right)}^2{\beta}^2{P}_{tr} $$

(8)

where $\beta =\frac{40}{f_{t(MHz)}}$

Similar to eq (6), the propagation loss in dB can be obtained from (8) as given in (9):

$$ Pl\left(\mathrm{dB}\right)=76.3+20{\log}_{10}\left({f}_{t(MHz)}\right)-20{\log}_{10}\left({H}_{tr}\right)-10{\log}_{10}\left({H}_r\right)+40{\log}_{10}\left({d}_c\right) $$

(9)

where the parameters, G_tr, G_r, H_tr, and H_r have been defined in the preceding equations.

The expression in (9) is the classical Egli model, and it shows that doubling the communication distance and transmission frequency increases the propagation loss by a factor of 4 and 2, respectively. Based on (9), Egli only considered 40 MHz to 1GHz transmission frequency range and communication distances from 1 km to 50 km [12]. This limited range of operation of the model poses significant limitations that need to be resolved. In addition, the model was developed by assuming a terrain profile similar to plane earth with limited vegetation, irregular topography, and severe building obstructions. Therefore, the need for Egli’s model adaptation becomes very important.

Adaptation of the classical Egli propagation loss model

First, to modify the classical Egli model, we introduce some constant free adaption parameters into (9), resulting in eq (10):

$$ {Pl}_{Egli}\left(\mathrm{dB}\right)={A}_1+{A}_2{\log}_{10}\left({f}_{t(MHz)}\right)-{A}_3{\log}_{10}\left({H}_{tr}\right)-{A}_4{\log}_{10}\left({H}_r\right)+{A}_5{\log}_{10}\left({d}_c\right) $$

(10)

Now, let Y_mdefine the measured signal propagation loss and ${\hat{Y}}_m$ be the Egli model with A₁,A₂, A₃, A₄, andA₅ being the adaptation parameters to be obtained based on the field signal propagation measurements in a non-linear square sense. It is given by eq (11):

$$ H=\sum \limits_{m=1}^n{\left({Y}_m-{\hat{Y}}_m\right)}^2=\sum \limits_{m=1}^n{\left({Y}_m-{A}_1-{A}_2{\log}_{10}\left({f}_{t(MHz)}\right)+{A}_3{\log}_{10}\left({H}_{tr}\right)-{A}_4{\log}_{10}\left({H}_r\right)+{A}_5{\log}_{10}\left({d}_c\right)\right)}^2 $$

(11)

where n indicates measured signal propagation data number. From (11), the challenge of determining the values of A₁,A₂,A₃, A₄, and A₅ based on field measurement can be transformed into an optimisation problem as given by (12). The evolving partial derivatives are given in eq.(13) to (17). Details of the mathematical procedures are available [35,36,37]:

$$ \underset{A_1,{A}_2,..,{A}_n}{\min }H\left({A}_1,{A}_2,..,{A}_n\right)=\underset{A_1,{A}_2,..,{A}_n}{\min}\sum \limits_{m=1}^n{\left({Y}_m-{\hat{Y}}_m\right)}^2 $$

(12)

$$ \frac{\partial H}{\partial {A}_1}=2\sum \limits_{i=1}^n\;\left({Y}_m-{\hat{Y}}_m\;\right)\;\frac{\partial {\hat{Y}}_m}{\partial {A}_1}=0 $$

(13)

$$ \frac{\partial H}{\partial {A}_2}=2\sum \limits_{i=1}^n\;\left({Y}_m-{\hat{Y}}_m\;\right)\;\frac{\partial {\hat{Y}}_m}{\partial {A}_2}=0 $$

(14)

$$ \frac{\partial H}{\partial {A}_3}=2\sum \limits_{i=1}^n\;\left({Y}_m-{\hat{Y}}_m\;\right)\;\frac{\partial {\hat{Y}}_m}{\partial {A}_3}=0 $$

(15)

$$ \frac{\partial H}{\partial {A}_4}=2\sum \limits_{i=1}^n\;\left({Y}_m-{\hat{Y}}_m\;\right)\;\frac{\partial {\hat{Y}}_m}{\partial {A}_4}=0 $$

(16)

$$ \frac{\partial H}{\partial {A}_5}=2\sum \limits_{i=1}^n\;\left({Y}_m-{\hat{Y}}_m\;\right)\;\frac{\partial {\hat{Y}}_m}{\partial {A}_5}=0 $$

(17)

By setting $\frac{\partial H}{\partial {A}_1}={f}_1\left({A}_1,{A}_2,{A}_3\right)\frac{\partial H}{\partial {A}_2}={f}_2\left({A}_1,{A}_2,{A}_3\right)$, and $\frac{\partial H}{\partial {A}_3}={f}_3\left({A}_1,{A}_2,{A}_3\right)$

The expressions in (13) to (17) are converted as given in (18):

$$ \left\{\begin{array}{l}{f}_1\left({A}_1,{A}_2,{A}_3,{A}_4,{A}_5\right)=0\\ {}{f}_2\left({A}_1,{A}_2,{A}_3,{A}_4,{A}_5\right)=0\\ {}{f}_3\left({A}_1,{A}_2,{A}_3,{A}_4,{A}_5\right)=0\\ {}{f}_3\left({A}_1,{A}_2,{A}_3,{A}_4,{A}_5\right)=0\\ {}{f}_3\left({A}_1,{A}_2,{A}_3,{A}_4,{A}_5\right)=0\end{array}\right. $$

(18)

Equation (18) expresses the non-linear equation; solving for the values of A₁, A₂, A₃, A₄, and A₅ by employing analytical techniques is generally a complex task. A commonly used method of solving the above complex equation is the Gauss-Newton method [38]. Still, its application during the iterative implementation process requires a full rank matrix, thus becoming a significant limitation of the algorithm.

In this work, we engaged the Levenberg-Marquart method [39,40,41] to upturn the limitation of the Gaussian Newton algorithm in resolving the non-linear equation in (18). Thus, by employing the Levenberg-Marquart algorithm, eq (18) is transmuted as given in (19):

$$ \underset{A\in {\Re}^3}{\min }f(A)=\frac{1}{2}{\left\Vert f(A)\right\Vert}^2=\frac{1}{2}\sum \limits_{i=1}^5{f}_i^2(A),A=\left({A}_1,{A}_2,{A}_3,{A}_4,{A}_5\right) $$

(19)

The Levenberg-Marquart method can be resolved with the following eq (20)–(21):

$$ \left(J{\left({x}_q\right)}^TJ\left({x}_q\right)+{\mu}_qI.\right)\Delta {x}_q=-J{\left({x}_q\right)}^Tf\left({x}_q\right),\mu \ge 0 $$

(20)

$$ \Rightarrow \Delta {x}_q={\left(J{\left({x}_q\right)}^TJ\left({x}_q\right)+{\mu}_qI.\right)}^{-1}\left(-J{\left({x}_q\right)}^Tf\left({x}_q\right)\right)\mu \ge 0 $$

(21)

J(x_q) = f^I(A)= Jacobian Matrix

where

I ∈ ℜ^m × m and μ express the damping term and the identity matrix introduced by Leverberg-Marquart into the classical Gauss-Newton algorithm [38] to improve its performance.

Accordingly, utilizing the Levenberg-Marquart method, the parameters A = (A₁, A₂, A₃, A₄, A₅) can be obtained iteratively using eq (22):

$$ {x}_{q+1}={x}_q+{\left(J{\left({x}_q\right)}^TJ\left({x}_q\right)+{\mu}_qI.\right)}^{-1}\left(-J{\left({x}_q\right)}^Tf\left({x}_q\right)\right),\mu \ge 0 $$

(22)

Thus, for μ = 0, the expression in eq (20) becomes the classical Gauss-Newton algorithm given by eq (23):

$$ \left(J{\left({x}_q\right)}^TJ\left({x}_q\right)\right)\Delta {x}_q=-J{\left({x}_q\right)}^Tf\left({x}_q\right) $$

(23)

Similarly, eq (22) is simplified as given in (24):

$$ {x}_{q+1}={x}_q+{\left(J{\left({x}_q\right)}^TJ\left({x}_q\right)\right)}^{-1}\left(-J{\left({x}_q\right)}^Tf\left({x}_q\right)\right) $$

(24)

To implement the LM algorithm, the following steps are engaged intuitively:

LM algorithm implementation steps

• I. Initialise guess parameters,x_o for x at iteration q = 0, 1, 2, …

• II. Select the Lagrange multiplierλ_qfor each step q

• III. Calculate theΔx_qwith its expression in eq. (21)

• IV. Calculate x_q + 1 = x_q + Δx_q

• V. Evaluate Δx_q = x_q + 1 − x_qat the initial parameter,x_o

• VI. For smaller Δx_qvalues, check the rate of convergence

• VII. If convergence rate is acceptable, stop the calculation or else go a step (IV).

Performance statistical metrics

To quantitatively examine the performance of the non-linear regression approach, six standard statistical metrics are engaged. These include the root mean square error (RMSE), maximum absolute error (MAE), mean absolute percentage error (MAPE), mean absolute error (MAE), standard deviation error (STD), and mean percentage residual error (MPRE). Detailed information about these statistical metrics can be found in [35, 42,43,44,45].

Results and discussions

The results of the study and valuable discussions are presented in this section. As shown in Figs. 4, 5, 6, and 7, the top parts of the graphs reveal how the classical Egli model in eq (9) predicts the acquired propagation loss values relative to the measured distances across the study site locations. It is conspicuously clear from the graphs in Figs. 4, 5, 6, and 7 that the classical Egli model over-predicted the measured propagation loss with considerably high RMSE, MAE, MAPE, MAE, STD, and MPRE values in sites 1 to 4. Such significant error differences may be attributed to the physical terrain and topographical differences between locations where the measurement loss data is conducted and the terrain characterization where the Egli model was developed. Thus, the need to adapt (fine-tune) the Egli model to fit the measured propagation loss data is self-evident.

Fig. 4

Measured propagation loss estimation and their error distribution statistics using classical Egli model and adapted Egli model for Cell_1

Fig. 5

Measured propagation loss estimation and their error distribution statistics using classical Egli model and adapted Egli model for Cell_2

Fig. 6

Measured propagation loss estimation and their error distribution statistics using classical Egli model and adapted Egli model for Cell_3

Fig. 7

Measured propagation loss estimation and their error distribution statistics using classical Egli model and adapted Egli model for Cell_4

To adapt the Egli model to the measured propagation loss data acquired over the two study locations using the non-linear square regression method, we employ the robust Levenberg-Marquart algorithm and the commonly used Gauss-Newton algorithm. Table 2 displays the obtained adaptive coefficients of the Egli model and their descriptive statistical values after employing the non-linear square regression method for its adaptation across the propagation loss measurement locations. Furthermore, Table 2 presents the estimation errors using the Egli model before and after the adaptation to measured field propagation loss data in the study locations. Moreover, Figs. 4, 5, 6, and 7 are plotted graphs showing the estimation performance of the Egli model on the measured propagation loss before and after its adaptation in the four study locations using the Levenberg-Marquart algorithm and the commonly used Gauss-Newton algorithm.

Table 2 Estimated coefficients and statistics for Cell_1 to Cell_4

Based on the obtained adapted propagation model parameters summarized in Table 2 using the LM algorithm, the Egli model loss Pl_Egli(dB) for cell_1 to cell_4 can be written as (25)–(28):

$$ {Pl}_{Egli}(dB)=72.24+6.16{\log}_{10}\left({f}_{t(MHz)}\right)+26.20{\log}_{10}\left({H}_{tr}\right)-10.71{\log}_{10}\left({H}_r\right)+39.34{\log}_{10}\left({d}_c\right) $$

(25)

$$ {Pl}_{Egli}(dB)=73.25+9.60{\log}_{10}\left({f}_{t(MHz)}\right)-24.49{\log}_{10}\left({H}_{tr}\right)-10.53{\log}_{10}\left({H}_r\right)+30.48{\log}_{10}\left({d}_c\right) $$

(26)

$$ {Pl}_{Egli}(dB)=73.98+12.07{\log}_{10}\left({f}_{t(MHz)}\right)+23.49{\log}_{10}\left({H}_{tr}\right)-10.40{\log}_{10}\left({H}_r\right)+28.69{\log}_{10}\left({d}_c\right) $$

(27)

$$ {Pl}_{Egli}(dB)=73.35+9.93{\log}_{10}\left({f}_{t(MHz)}\right)-24.35{\log}_{10}\left({H}_{tr}\right)-10.51{\log}_{10}\left({H}_r\right)+29.39{\log}_{10}\left({d}_c\right) $$

(28)

Taking the mean of the first two eq (25)–(26) and the last two eq (27)–(28) leads to the adapted Egli models obtained for Port Harcourt City and Uyo City in (29) and (30), respectively:

$$ {Pl}_{Egli}(dB)=29.75+35.16{\log}_{10}\left({f}_{t(MHz)}\right)+{A}_3{\log}_{10}\left({H}_{tr}\right)-{A}_4{\log}_{10}\left({H}_r\right)+9.42{\log}_{10}\left({d}_c\right) $$

(29)

$$ {Pl}_{Egli}(dB)=32.16+26.90{\log}_{10}\left({f}_{t(MHz)}\right)+{A}_3{\log}_{10}\left({H}_{tr}\right)-{A}_4{\log}_{10}\left({H}_r\right)+11.54{\log}_{10}\left({d}_c\right) $$

(30)

The above expressions also reveal that the mean rate of propagated loss attenuation for Port Harcourt City and Uyo City are 3.5 and 2.9, respectively. As expected, the values also clearly reveal that the signal loss attenuation rate in Port Harcourt is about 30% higher than the one obtained for Uyo. The higher signal loss attenuation rate may be ascribed to higher cluttered buildings, other obstructions, and congested human/vehicular traffic in Port Harcourt than Uyo city.

Additionally, Figs. 8, 9, 10, and 11 provided the residual error distribution quantitatively fits of the LM and GN regression methods along with the measured propagation loss data. In terms of mean percentage residual error indicated with letter W in the graphs, the LM method attained 3.53, 3.50, 3.26, and 2.04, respectively, whereas the GN method attained more flawed values of 4.77 5.56, 11.63, and 15.09. Furthermore, plots showing normal distribution fits are shown in each figure to reveal the residual error distribution spreads using the LM and GN methods. Again, lower error spreads with LM over the GN method in each plot show a better Egli model adaptation efficiency. The enhanced estimation performance of the LM-based adapted Egli model relative to the GN measured propagation loss data can be attributed to its ability to tune multiple free parameters and achieve optimal solutions regardless of the nature of the initial guess parameters selected.

Fig. 8

Measured propagation loss estimation and their error distribution statistics using classical Egli model and adapted Egli model for Cell_1

Fig. 9

Measured propagation loss estimation and their error distribution statistics using classical Egli model and adapted Egli model for Cell_2

Fig. 10

Measured propagation loss estimation and their error distribution statistics using classical Egli model and adapted Egli model for Cell_3

Fig. 11

Measured propagation loss estimation and their error distribution statistics using classical Egli model and adapted Egli model for Cell_4

Estimation error analysis

The estimation errors are obtained using six standard statistical indicators. These comprise the root mean square error (RMSE), maximum absolute error (MAE), mean absolute percentage error (MAPE), mean absolute error (MAE), standard deviation error (STD), and mean percentage residual error (MPRE), as briefed in the “Adaptation of the classical Egli propagation loss model” section. For convenience, we also used W to refer to the mean percentage residual error (MPRE) in the results, as shown in Figs. 8, 9, 10, and 11. The computed first-order estimates statistics are presented in Tables 3, 4, 5, and 6. Specifically, the computed first-order estimates statistics for Cell_1 are presented in Table 3. The calculated first-order estimates statistics for Cell_2 are given in Table 4. The calculated first-order estimates statistics for Cell_3 are shown in Table 5, and the computed first-order estimates statistics for Cell_4 are displayed in Table 6.

Table 3 Computed first order estimates statistics for Cell_1

Table 4 Computed first order estimates statistics for Cell_2

Table 5 Computed first order estimates statistics for Cell_3

Table 6 Computed first order estimates statistics for Cell_4

The summarized fitting performance using LM and GN regression methods compared with the original Egli model on the measured loss data for each eNodeB cell are displayed in Tables 3, 4, 5, and 6. With MAE and RMSE, the LM method attained 0.18, 0.37, 2.32, 3.41 and 5.33, 2.12, 4.23, 4.56 values compared to GN-based adapted Egli model that attained 3.80, 3.32, 3.56, 5.44 and 5.12, 7.13, 8.22, 10.21, respectively. Improved fitting performances are recorded for the adapted Egli model using STD, MAPE, and MPRE statistical metrics. It is worth noting that lower computed error values with the indicators depict a better estimation accuracy.

Finally, field measurements taken from four selected eNodeB transmission sites located in Uyo and Port Harcourt in Nigeria were used in this study. The adapted Egli model was derived based on these measurements. In order to test the validity of the model, we compared the adapted Egli model with another set of measurements taken from another cell different from the initial four eNodeBs tested. The height of the new eNodeB investigated is 32 m, and other parameters of the eNodeB were observed. The results of our comparison are shown in Fig. 12. Other valuable results include A1=7.39, A2=9.18, A3=-2.60, A4=0.57, A5=41.11. MAE=3.38, MAPE=2.15, STD=2.80, RMSE=4.40, and PE=7.63.

Fig. 12

Comparison of measured pathloss from the new eNodeB and the LM-based adapted Egli model

Conclusions

A detailed investigation to improve real-time propagation path loss estimation using two closed medium-sized cities in Nigeria was presented in this paper. By employing a non-linear square regression method based on the Levenberg-Marquart algorithm, Egli’s model was adapted to get an improved fitting for propagation loss in the investigated cities. Additionally, the performances of the adapted Egli model and the classical Egli model based on the measured propagation loss data have been demonstrated using six different statistical indicators for comparative analysis. The adapted Egli model displayed lower estimation errors than the classical model across the two cities studied. The results reveal that the adjusted Egli model parameters better fit the two cities investigated and can be deployed to efficiently plan macrocellular communication networks in similar propagation environments. Finally, the work in this paper could provide valuable guidelines and references in dimensioning or optimizing cellular network deployment. Future work would focus on optimizing the parameters of the adapted Egli model for improved performance and providing correction factors to ease its applicability in other related propagation environments for emerging wireless communication systems.