A comparative study between the system reliability evaluation methods: case study of mining dump trucks

Abstract

The shovel-truck system is a widely used technique for haulage systems in surface mining operations. However, predicting the failure patterns of complex systems requires accurate failure prediction techniques. In this study, several major system reliability evaluation groups, including non-parametric, parametric, and semi-parametric methods, are investigated, and their effectiveness is compared to identify the best group for predicting the failure patterns of complex systems such as mining dump trucks, which operate in harsh environments. A historical dataset of time to failure (TTF) and maintenance data was collected. Then, the system’s reliability was evaluated using the major TTF data analysis methods. The findings demonstrated that all the major system reliability evaluation groups produced similar curves; however, the semi-parametric method outperformed the other methods. This result underscores that this system reliability evaluation group is the most effective method for complex systems. Also, it was found that the dump truck reliability dropped to 50% after 40 operation hours, demonstrating the critical importance of implementing preventive maintenance to enhance the system’s performance and ensure operation safety. In addition, this study provided an appropriate insight into the predictive methods and offered an accurate estimation of the failure pattern of complex systems, resulting in availability and productivity improvement.

Introduction

Raw material extraction is one of the fundamental links in the value chain of mineral products. This operation depends on mining equipment such as loaders, dozers, shovels, and dump trucks. Besides, these assets have become more complex and expensive so as to require more accurate maintenance to prevent operation interruption and loss of production capacity. Achieving these goals needs an efficient maintenance plan to implement inspections, preventive maintenance, and corrective maintenance. Therefore, reliability evaluation and maintenance management can remarkably affect haulage system performance and availability, leading to production capacity insurance [1].

Reliability evaluation is one of the effective metrics for developing comprehensive maintenance strategies [2]. It is employed for different applications and purposes in various engineering sectors. Figure 1 displays a network of reliability applications in previous studies.

Fig. 1

A network of reliability applications

Various researchers employed different system reliability methods for analyzing complex systems’ performance. Roy et al. [3] determined reliability and maintainability characteristics in a fleet of mining shovels. They analyzed failure and maintenance data for four shovels by dividing the shovel system into several sub-systems. Thus, the maintenance intervals were estimated for each shovel. Ghodrati and Kumar [4] employed the PHM to predict the optimal number of spare parts for the hydraulic jack in load–haul–dump (LHD) machine operations. Barabady and Kumar [5] evaluated the reliability and availability of crushing equipment using the parametric reliability method to identify the most critical components in this system. Uzgören et al. [6] assessed the reliability of two dragline excavators using the parametric reliability method and then compared the results. In addition, Barabadi et al. [7] studied mine haulage throughput capacity considering failure rate and environmental conditions. They utilized the reliability phase diagram to analyze the reliability of the haulage trucks operating in two different production lines. Morad et al. [8] utilized a parametric reliability method to estimate the reliability of mining equipment sub-systems. They divided the mining truck system into several sub-systems and then predicted the reliability of each sub-system. Pandey et al. [9] performed reliability and failure rate evaluations for critical sub-systems of three dragline excavators operating in surface mines. They intended to increase availability and decline maintenance and production costs. Angeles and Kumral [10] employed the power law process as a parametric reliability method to estimate optimal inspection and preventative maintenance scheduling in mining equipment. Allahkarami et al. [11] utilized a mixed frailty model to identify the observed and unobserved risk factors affecting the system reliability in mining systems. Moniri-Morad et al. [12] analyzed the haulage fleet production capacity by estimating the system’s reliability, availability, and maintainability (RAM). In this case, the discrete-event simulation and PHM have been combined to perform RAM analysis. Toraman Jakkula et al. [13] investigated the RAM in LHD machine operations. Toraman [14] conducted the RAM analysis to compute the performance of large-capacity trucks in mining operations. Florea et al. [15] utilized parametric models to investigate the reliability and maintainability of mining equipment with components subjected to intense wear. They determined the critical failure modes and their effects to establish a comprehensive maintenance plan for the components.

Previous studies have employed various system reliability evaluation models based on their application fields and data availability. However, a majority of these models can be classified into three major groups, including non-parametric, parametric, and semi-parametric methods [16, 17]. Hence, it is necessary to analyze each method and compare their results based on the application field. The main aims and contributions of this work can be summarized as follows:

• Predicting the failure patterns of complex systems operating in the mining industry

• Investigating the practicality and robustness of the major system reliability evaluation groups in challenging and harsh operating conditions

• Identifying the most effective system reliability evaluation group, resulting in superior performance outcomes

• Facilitating data-driven decision-making strategies by comparing the major system reliability evaluation groups, empowering analysts to choose the most accurate and appropriate method for their specific applications

• Demonstrating the applicability of the system reliability methods in various industries dealing with complex systems

The rest of this paper is organized as follows. The “Methods” section describes the study aims, proposed method, procedure, and boundaries for this study. Then, the “Methods” section investigates the major system reliability evaluation groups in a case study. Afterward, the achieved results are discussed in the “Discussion” section. Finally, conclusions and some remarkable findings are presented in the “Conclusions” section.

Methods

Reliability analysis is one of the most significant metrics in evaluating a system’s performance. It is a process that encompasses collecting and pre-processing datasets, selecting appropriate reliability techniques (e.g., mathematical, statistical, or simulation), estimating system reliability, and interpreting the results. This process provides an appropriate insight into the system failure patterns, potential failure modes, and system characteristics, enabling analysts to make informed decisions about the system's reliability improvement. There are two kinds of reliability analysis processes: structural and system reliability analyses [17,18,19]. This study revolves around the system reliability analysis process, particularly reliability methods based on TTF data analysis.

Figure 2 illustrates the proposed step-by-step procedure in this study. As shown in Fig. 2, this study is designed based on two phases, encompassing reliability estimation and comparison processes. The system reliability estimation is started by collecting data and performing data pre-processing. Then, the data distribution is checked. If the dataset has a known distribution function, the parametric reliability method can be considered for analyzing the process. Otherwise, the non-parametric or semi-parametric reliability methods can be employed. Indeed, the non-parametric and semi-parametric reliability methods are used when the dataset does not have a known distribution function or if the distribution is complex or multi-modal. Therefore, it is possible to estimate the system’s reliability using the available methods. In the second phase, multiple selection criteria are identified by experts, and then the best system reliability evaluation group is selected among the non-parametric, parametric, and semi-parametric reliability methods.

Fig. 2

The designed reliability estimation process for this study

Non-parametric reliability method

The non-parametric reliability method is focused on collecting and analyzing the TTF dataset without making assumptions about an underlying distribution function. This method revolves around descriptive statistics to analyze the TTF data. Researchers have developed various non-parametric reliability models, such as Kaplan-Meier [20] and Nelson-Aalen [21]. The non-parametric reliability method has a significant advantage over the other reliability methods (i.e., parametric and semi-parametric). In other words, it provides accurate outputs without assuming a specific probability distribution function. This procedure eliminates the risk of choosing an incorrect distribution and guarantees robustness in the analysis. However, it is crucial to mention that the non-parametric reliability method is restricted to the observed data, confining their ability to simulate results for other time intervals beyond the available data. Thus, it is necessary to consider this limitation when employing a non-parametric reliability method.

The Kaplan-Meier model is proposed as one of the most conspicuous non-parametric reliability models in analyzing the TTF data. The reliability diagram is drawn as a step function with discontinuities or jumps at the observed failure times. Also, the height and width of these steps vary depending on the reliability function estimations and failure time observations, respectively [20]. The Kaplan-Meier model formulates the reliability function as follows:

$$R({t}_{j})=\prod\nolimits_{j\left|{t}_{j}\le t\right.}\frac{{{n}_{j}-d}_{j}}{{n}_{j}}$$

(1)

where ${t}_{j}$ (j = 1, …, m) represents the failure times, m is the total number of data points, ${n}_{j}$ is the number of units at the failure risk just before time ${t}_{j}$, and ${d}_{j}$ is the number of failures at time ${t}_{j}$. If the observed data express the failure events, ${d}_{j}=1$. Otherwise, if the observed data describe the censored data, ${d}_{j}=0$.

In the Kaplan-Meier model, the first observation occurs at time t = 0. Thus, there is no failure event in the first observation (R(t₁) = 1), and then the reliability function goes to zero as a step function.

Parametric reliability method

The parametric reliability method is an effective technique for understanding system reliability. This method provides appropriate insights into the failure mechanisms, and the resulting model can assess the reliability parameters for the system’s lifetime. Also, the reliability is evaluated by fitting the standard distribution functions into TTF data. In this case, the estimated model can predict the reliability values beyond the range of the existing dataset.

The parametric reliability method is performed as follows. The trend of TTF data is first tested to identify the failure patterns of a system. The probability plotting method [22] is suggested to examine the data trend. This method is configured based on plotting the cumulative number of events against the cumulative TTFs [23]. The plot output is either a straight line or nonlinear. If the curve has an increasing (or decreasing) trend, the system is repairable, and thus, it is repaired after occurring a failure event. In a repairable system, the PLP is proposed as one of the most significant parametric reliability models in analyzing the failure intensity of a system. The failure intensity is defined as follows:

$$h\left(t\right)=\lambda .\beta .{t}^{\beta -1}$$

(2)

where $h\left(t\right)$ denotes the intensity function, t is the time between failures (TBFs), $\beta$ presents the Weibull parameter, and $\lambda$ is the model parameter.

The PLP is formulated using non-stationary techniques like the non-homogeneous Poisson process (NHPP) [24]. Indeed, a minimal repair process is performed on the system after occurring failure, and the system status returns to its status just before performing the repair action. In this model, the reliability function can be formulated as follows:

$$R\left(t\right)=\mathrm{exp}(-H(t))$$

(3)

where $R\left(t\right)$ is the reliability function, $H(t)$ is the cumulative intensity function ($H\left(t\right)={\int }_{0}^{t}h(\varnothing )d\varnothing$), and $h\left(\varnothing \right)$ is the intensity function.

Also, satisfying the identical and independent distribution (IID) conditions demonstrates that the system is non-repairable. These conditions are fulfilled by evaluating the data trend and dependency. In a non-repairable system, the dataset has a straight-line trend, and the data dependency is examined via the serial correlation test. This test is conducted by plotting the i^th incident time versus the (i-1)^th incident time. In this diagram, the dataset is independent if all the points are scattered in a single cluster; otherwise, the dataset is dependent, indicating the violation of IID conditions. In a non-repairable system, the RP model is suggested as one of the most remarkable parametric reliability models in predicting the failure behavior profile of the system.

Semi-parametric reliability method

The semi-parametric reliability method evaluates the TTF data when exogenous factors affect the system’s reliability. This study revolves around the PHM as one of the most practical semi-parametric reliability models for evaluating system reliability in a heterogeneous environment.

The PHM has two main elements: a baseline hazard function and a multiplicative term (covariates). This model is mathematically formulated by Eq. (4) [25, 26].

$$h\left(t|Z\right)={h}_{0}\left(t\right)\mathit{exp}\left({\beta }^{T}Z\right)$$

(4)

where $h\left(t|z\right)$ is the observed hazard function, and ${h}_{0}\left(t\right)$ is the baseline hazard function (dependent only on time), which occurs when the covariates have no influence on the failure profile ($Z=0$ or $\mathrm{exp}\left({\beta }^{T}Z\right)=1$). Also, $Z$ is a $t\times 1$ vector containing covariates. In addition, ${\beta }^{T}$ is a $1\times t$ vector of regression coefficients. These coefficients characterize the impact of covariates. The PHM is assumed to be proportional, demonstrating a constant hazard ratio (HR) between any two observations over time. This proportionality is tested as follows:

$$HR=\frac{{h}_{1}\left(t|{Z}_{1}\right)}{{h}_{2}\left(t|{Z}_{2}\right)}=\frac{{h}_{0}\left(t\right)\mathit{exp}\left({\beta }^{T}{Z}_{1}\right)}{{h}_{0}\left(t\right)\mathit{exp}\left({\beta }^{T}{Z}_{2}\right)}=\mathit{exp}[{\beta }^{T}\left({Z}_{1}-{Z}_{2}\right)]$$

(5)

where HR is the hazard ratio, and ${h}_{1}\left(t|{Z}_{1}\right)$ and ${h}_{2}\left(t|{Z}_{2}\right)$ are two different observations.

In the parametric PHM, the unknown parameters for the baseline hazard function and the coefficients of covariates are estimated via the log-likelihood function.

$$\mathrm{ln}({L}_{i})={ d}_{i}\mathrm\;{ln}\left[{H}^{\prime}\left({t}_{i}|Z\right)\right]-H({t}_{i}|Z)$$

(6)

where $\mathrm{ln}({L}_{i})$ is the log-likelihood function for the i^th failure event, ${H}{\prime}\left({t}_{i}|Z\right)$ is the derivative function of the cumulative observed hazard function ($H({t}_{i}|Z)$), and ${d}_{i}$ is the event indicator. If the observed event is failure, ${d}_{i}=1$; otherwise, ${d}_{i}=0$.

In this case, the Newton-Raphson technique is utilized to find the roots of the maximum likelihood estimation (MLE). This technique is formulated as follows:

$${\updelta }_{i+1}={\updelta }_{i}+(-{}^{g\left({\updelta }_{i}\right)}\!\left/ \!{}_{H({\updelta }_{i})}\right.)$$

(7)

where ${\updelta }_{i+1}$ represents the value of the new root, ${\updelta }_{i}$ denotes the root value of the i^th iteration, $g\left({\updelta }_{i}\right)$ describes the gradient vector, and $H({\updelta }_{i})$ characterizes the Hessian matrix.

Finally, it is possible to compute the system reliability function using the following equation:

$$R\left(t|Z\right)=\mathrm{exp}({H}_{0}(t)\mathrm{exp}({\beta }^{T}Z))$$

(8)

where $R\left(t|Z\right)$ is the observed reliability function, ${H}_{0}(t)$ is the cumulative baseline hazard function, $Z$ is a vector containing covariates, and ${\beta }^{T}$ is a vector of regression coefficients.

Results

Historical data analysis

The failure and maintenance data collection process plays a crucial role in assessing the performance and reliability of mining equipment. This process involved systematically gathering data from mining operations to obtain solid insights into failure modes, failure frequency and severity, breakdowns, maintenance activities, and the effectiveness of maintenance strategies. In this case, the data collection spanned one year and specifically was focused on the failure and maintenance data of a Komatsu dump truck with a capacity of 100 tons, which operated at Sungun mine, East Azerbaijan province, Northwest of Iran. The dump truck had accumulated approximately 15,000 h.

The collected dataset included the failure times, restoration or replacement times, type of failed sub-system, and environmental conditions. Table 1 provides a sample of the collected failure dataset for this study. This dataset includes TTFs for the dump truck sub-systems (i.e., Engine, Transmission, Hydraulics, Body and Chassis, and Gearbox), the severity of the failure incident, and average temperature. In Table 1, the value of one denotes the failed sub-systems in each failure observation. For instance, the first failure occurred after 14 operation hours at − 0.1 °C, and the failure incident was due to the failure in the Hydraulics sub-system. Also, the severity value was zero, indicating that this failure was a mild incident. Table 2 provides the preliminary analysis of these data. In this table, the number of data was 92, and the variables were categorized into two groups, encompassing binary and continuous variables. The binary variables are subject to two states of success and failure. The percentage of ones represents the percentage of failure occurrences in each variable. In the continuous variables, the mean, standard deviation, minimum, and maximum values were reported for each variable. After the preliminary analysis of the dataset, it was analyzed based on the three major system reliability evaluation groups.

Table 1 A sample of the collected failure dataset

Table 2 A statistical summary of the collected dataset

Non-parametric analysis of the collected dataset

The non-parametric reliability method was conducted by formulating the Kaplan-Meier model. Table 3 gives the results of the reliability estimation using the Kaplan-Meier model.

Table 3 Evaluating system reliability using the non-parametric method

Table 3 reports various valuable information, including system reliability at different times, standard error, and uncertainty (95% confidence interval). Also, the reliability value goes from 0.9565 to 0.0109 after 220 operation hours.

Parametric analysis of the collected dataset

The truck reliability was estimated through the parametric reliability method. In this regard, the TTF data trend was first examined using the probability plotting technique. Figure 3 depicts the trend test for the truck system. The results of Fig. 3 illustrated that this dataset did not have a trend. Then, the serial correlation test was utilized to investigate the TTF data dependency (Fig. 4). According to the results of these two figures, the dump truck system followed the IID conditions, demonstrating that the dump truck should be analyzed as a non-repairable system. Therefore, the RP model was employed to estimate the dump truck failure behavior.

Fig. 3

The trend test for truck system

Fig. 4

The serial correlation test for the truck system

After confirming the IID conditions for the existing dataset, a standard parametric distribution function was fitted to the data to find the best probability distribution. Multiple standard parametric distributions were analyzed for this purpose. Table 4 reports the estimated parameters and the log-likelihood values for four distribution functions. Among these distributions, the Log-normal distribution showed the best fit with a log-likelihood value of − 452. Also, the mean and the standard deviation for the Log-normal distribution are 3.761 and 0.77, respectively.

Table 4 Evaluating various standard parametric distribution functions for identifying the best fit

Then, the system reliability was estimated based on fitting the Log-normal distribution function to the dataset. The system reliability function was formulated by the log-normal distribution as follows:

$$R\left(t\right)=\mathrm{exp}\left[\mu +\frac{1}{2}{\sigma }^{2}\right]$$

(9)

where $R\left(t\right)$ is the reliability function obtained from the Log-normal distribution, and $\mu$ and $\sigma$ are the mean and standard deviation of the Log-normal distribution function, respectively.

Table 5 gives the reliability values estimated by the parametric method at various times. The uncertainty of the estimated reliability was also computed using the lower and upper bounds at a 95% confidence interval (Table 5).

Table 5 Estimating system reliability at various times using the parametric method

Additionally, the parametric method was utilized to estimate the reliability of each truck sub-system. In this procedure, the truck system was decomposed into five sub-systems. Then, the failure data for each sub-system were analyzed. Table 6 gives the best-fitted distribution and the reliability value for each sub-system. The reliability value was estimated at 100 operation hours. Among these sub-systems, the most reliable and unreliable sub-systems were Gearbox and Engine, respectively.

Table 6 Estimating the reliability of each truck sub-system using the parametric method

Semi-parametric analysis of the collected dataset

The truck system reliability was estimated using the semi-parametric method, particularly parametric PHM. The estimation process was performed by analyzing the TTFs and the binary and continuous variables.

According to Table 1, the hazard function was formulated as $h\left(t|Z\right)$. In this case, the Weibull distribution function was chosen as the most appropriate function for modeling the baseline hazard function. Then, the Weibull distribution parameters and the model's variables were computed using Eqs. (6) and (7). Afterward, the hazard function was calculated as follows:

$$\mathrm{ln}H\left(t|X\right)=-0.98+1.29\times \mathrm{ln}t+0.66\times Engine+0.18\times Transmisson+0.65\times Hydraulics+0.84\times Body\&Chassis+1.17\times Gearbox-1.19\times Severity-0.0058\times Temperature$$

(10)

where $\mathrm{ln}H\left(t|X\right)$ is the log cumulative hazard function, the Engine, Transmission, Hydraulics, Body & Chassis, Gearbox, and Severity are binary variables, and Temperature is a continuous variable.

Therefore, the reliability function was obtained to estimate the reliability values for the dump truck system. Table 7 gives the reliability results based on the semi-parametric method. This table reports the reliability function and the estimated uncertainty (95% confidence interval) to provide a proper estimate.

Table 7 Estimating the reliability function using the semi-parametric method

Discussion

The truck system has been evaluated using three major system reliability evaluation groups, including non-parametric, parametric, and semi-parametric methods. Figure 5 demonstrates the reliability curves derived from the semi-parametric (parametric PHM), parametric (RP or PLP), and non-parametric (Kaplan-Meier) methods.

Fig. 5

A comparison between the reliability curves in the non-parametric, parametric, and semi-parametric methods

As shown in Fig. 5, in the initial 30 operation hours, the non-parametric method estimates the reliability values higher than the other methods (i.e., parametric and semi-parametric methods). The parametric reliability curve coincides with the semi-parametric reliability curve in this interval. The reliability curves indicate different values during the 50–100 operation hours. However, these deviations are negligible. After this period, all reliability curves are approximately matched. Moreover, the reliability estimation curves illustrated that the dump truck system reliability dropped to 0.4 and 0.19 after 50 and 100 operation hours, respectively. This issue revealed the necessity of applying preventive maintenance plans before these operation hours to improve the system availability and prevent dump truck sudden failures.

Although these major system reliability evaluation groups fundamentally had different statistical procedures in the ranking and evaluation process, they nearly predicted similar results. However, it is essential to compare the efficiency and performance of these major system reliability evaluation groups to provide better insights into their functionality. For this purpose, multiple criteria were chosen to analyze and compare their performance. Table 8 compares these major system reliability evaluation groups based on several criteria.

Table 8 A comparison between the performance of the major reliability evaluation groups

According to Table 8, five performance criteria were considered to compare the non-parametric, parametric, and semi-parametric reliability methods, including Method Scope and Completeness, Data Availability and Abundance, Variable Categorization, Uncertainty Quantification and Analysis, and Extrapolation Capability and Predictive Power.

The method scope and completeness criterion demonstrated that the semi-parametric method efficiently estimated the influence of operational and environmental variables together with reliability and failure rate analyses, all in a one-step approach. But the parametric (or non-parametric) methods required a multi-step approach, decomposing the system into several sub-systems to evaluate their individual failures separately.

Data availability and abundance criterion confirmed the advantage of the semi-parametric method over the other methods. Indeed, the semi-parametric method lies in a one-step approach, allowing the utilization of the full dataset for the analysis process. However, the parametric (or non-parametric) method requires the decomposition of the system into multiple sub-systems, each analyzed separately. Consequently, data will be shared between sub-systems, potentially leading to data insufficiency for certain sub-systems.

Variable categorization was another criterion for choosing the best method. In this study, two different variables were considered: failure-related variables (e.g., transmission failure) and non-failure-related variables (e.g., rain and temperature). The parametric (or non-parametric) method could not examine and quantify the effect of non-failure-related variables, whereas the semi-parametric method could efficiently assess these variables and their effects.

Uncertainty quantification was also another criterion for comparing these three major reliability evaluation groups. The confidence interval for the non-parametric method was wider than those of the semi-parametric and parametric methods, with values of 0.28, 0.15, and 0.11, respectively. Therefore, the parametric and semi-parametric methods demonstrated better performance than the non-parametric method from the uncertainty perspective.

The fifth criterion was extrapolation capability and predictive power. Both the semi-parametric and parametric methods could simulate and predict reliability estimations beyond the data range. While the non-parametric method does not have the ability to extrapolate beyond the available data range.

According to these findings, it is concluded that the semi-parametric method provided superior performance compared to the other methods. Thus, this system reliability evaluation group can be used as the most robust and effective method for evaluating the reliability of complex systems that operate in harsh environments.

Conclusions

This study compared three major system reliability evaluation groups to identify the best method for evaluating the mining truck performance. For this purpose, the Kaplan-Meier, RP (or PLP), and parametric PHM were chosen as the most significant system reliability evaluation models to formulate the non-parametric, parametric, and semi-parametric methods, respectively. Also, an actual mine haulage operation dataset was collected to estimate the dump truck reliability. Then, the system reliability was estimated using all three major system reliability evaluation groups at different times. The reliability analysis curves illustrated that the dump truck reliability dropped to 0.4 and 0.19 after 50 and 100 operation hours, respectively. The findings revealed that although these major system reliability evaluation groups had different statistical procedures, the reliability values were almost similar. However, the semi-parametric method outperformed the other methods due to the less computational process and estimating more details. Therefore, it is recommended to consider this method for evaluating the reliability of complex systems like mining dump trucks, which operate in harsh and heterogeneous conditions.