A design of new wind power forecasting approach based on IVMD-WSA-IC-LSTM model

Abstract

The wind power forecasting (WPF) technology can reduce the adverse impact of wind power grid connection. Based on the characteristics of wind power data, an algorithm based on improved variational mode decomposition (IVMD) and long short-term memory (LSTM) Network is proposed to predict the wind power, and hyper parameter optimization search of LSTM using Whale Swarm Algorithm with Iterative Counter (WSA-IC). Firstly, through correlation analysis, the characteristics of 10 different wind power data are screened, and two kinds of data with large correlation with wind power are determined as input of the mode. Secondly, IVMD is used to calculate the maximum envelope kurtosis, determine the best decomposition parameters of the variational mode decomposition (VMD), and the original wind power and wind speed sequences are decomposed to obtain the IMF with different time scales. Finally, to address the problems of difficult optimization of hyper parameter and difficulty in obtaining optimal solutions for LSTM neural network modes, the WSA-IC algorithm is proposed to optimize its key hyper parameter, and the IVMD-WSA-IC-LSTM forecasting mode is established to obtain the short-term forecasting results of wind power. The algorithm is tested with the data of China Longyuan Power Group Corporation Limited. Compared with other common forecasting approaches using same data, the mean absolute error (MAE) of the forecasting approach is reduced to 0.007859, the mean square error (MSE) is reduced to 0.00011, and the determination coefficient is improved to 0.998828, which has higher forecasting accuracy.

Introduction

Wind power, as a new energy, has developed rapidly in recent years [1]. Wind power plays an important role in our country and is widely used. However, randomness, intermittent and uncertainty of wind power will adversely affect the stable and safe of power grid. Accurate wind power forecasting is beneficial to efficient operation, safety analysis, and energy trading of power grid [2].

In response to the problems of low accuracy and poor anti-interference ability of traditional power forecasting algorithms, researchers propose some artificial intelligence-based WPF algorithms. Xu et al. [3] proposes a new WPF approach by mining the data of numerical weather prediction (NWP). The WPF approach based on LSTM neural network often has better accuracy than some artificial intelligence approaches [4, 5]. LSTM can effectively solve the problem of RNN’s inability to capture long-term dependence, but it has high complexity, long training time and multiple hyperparameters difficult to select. Yang et al. [6] proposed a bidirectional long short-term memory (Bi-LSTM) to predict multiple attributes of a product. Delgado et al. [7] presents a recurrent neural network-based variant LSTM-based power forecasting. A new prediction approach for wind power is proposed in Sun et al. [8], in which LSTM network, wavelet decomposition (WT), and principal component analysis (PCA) are combined together. In Pu et al. [9], an ultra-short-term WPF mode based on PSO and LSTM neural network combination is proposed, which improves the precision of ultra-short-term WPF. However, particle swarm optimization (PSO) has some problems such as slow convergence rate and local optimal solution. The optimization algorithms such as Drosophila Optimization (DO), Whale Swarm Optimization (WSO), and Bat Swarm Optimization (BSO) have emerged. In the Peng et al. [10], an improved DO algorithm is used to propose a wind speed forecasting mode to obtain wind power data with higher forecasting accuracy by expanding the parameter search range and improving the convergence speed of the algorithm. However, the original data is characterized by the rapid variational of wave crests and troughs, which leads to some defects in the forecasting mode. For data problems, Shi et al. [11] uses wavelet decomposition (WD) and artificial neural network (ANN) for forecasting, which improves forecasting performance, but wavelet packet decomposition in the Zheng et al. [12] requires artificial wavelet function setting, and noise interference may occur if improper decomposition result is set. In the Cheng et al. [13], a short-term WPF mode based on the combination of empirical mode decomposition (EMD) [14] and radial basis function neural network (RBFNN) [15] was proposed to make the decomposed wind power component series with strong regularity and improve the forecasting accuracy. Wang et al. [16, 17] adopts empirical mode decomposition empirical model decomposition (EMD) method to better solve the problem of modal aliasing phenomenon and susceptibility to noise interference, but the total number of modes K is difficult to determine, affecting the accuracy the mode. These methods solve the nonlinear problem of wind power data forecasting to some extent, but there are some problems such as poor generalization ability of mode, long training time and easy to over-fit or under-fit. Konstantin et al. [18] proposes RNN mode based on LSTM to predict wind power, which can make more accurate forecasting by using long-term correlation in time series, and solve the defects that other neural networks cannot learn for a long time. The Deng et al. [19] proposed a method to optimize the design of key hyper parameter of LSTM neural networks by combining the cuckoo algorithm. In the Chang et al. [20], a sparrow search algorithm is proposed to optimize the learning rate and number of neurons in LSTM and CNN-LSTM networks. However, it is difficult to select the optimal hyper parameter of LSTM neural network mode, and intelligent optimization algorithms such as cuckoo bird algorithm and sparrow algorithm need to set niche parameters for different problems, so it is difficult to obtain global optimal solutions.

WSA (Whale Swarm Algorithm) is a meta heuristic algorithm, which belongs to Swarm intelligence algorithm [21]. WSA-IC(Whale Swarm Algorithm with Iterative Counter) does not need to introduce small habitat parameters and can effectively jump out of local optimal solutions by identifying extreme value points based on stability thresholds and fitness thresholds during the iterative process. An IVMD-WSA-IC-LSTM WPF algorithm is proposed. The maximum envelope kurtosis is calculated by IVMD to determine the optimal decomposition parameter K of VMD, and the WSA-IC is used to improve the multimodal optimization iteration rule to obtain the optimal hyper parameter and global optimal solution of LSTM to improve the WPF accuracy. Compared with other forecasting algorithms, the forecasting error is smaller, the average absolute error is lower, the coefficient of determination and the forecasting accuracy are higher.

The first part of the paper introduces the VMD algorithm, and improves in the second part. In the third part, the improved whale swarm optimization algorithm is introduced to optimize the hyper parameter of LSTM neural network, and the fourth part tests the algorithm.

Improve methods

VMD Algorithm

VMD takes the original signal data into a number of Eigen mode components IMF with different center frequencies with finite bandwidth [22]. By constructing and solving the variational problem, the optimal solution and its central frequency of each Eigen mode function are solved by iteration [23].

Firstly, the variational problem [24] is constructed, assuming that the original signal f is decomposed into k components, and the decomposition signal is guaranteed to be a modal component of finite bandwidth with central frequency. The constraint variational expression is shown in Eq. (1):

$$\left\{\begin{array}{ll}\mathrm{min}\left\{\sum\limits^K_{k=1}\left\|{\partial}_t\left[\left(\delta\left(t\right)+\frac{j}{\pi{t}}\right)*u_k\left(t\right)\right]e^{-jw_{k}t}\right\|^2_2\right\}\\ s.t.\sum\limits^K_{k=1}u_k=f\end{array}\right.$$

(1)

where $u_{k}$ is mode signal; f is time series; k is mode number; * refers to convolution operator; w_k refers to center frequency of kth mode component; δ(t) refers to Dirac function, indicating density; and $e^{{ - jw_{k} t}}$ is the phase of signal rotation with time.

Secondly, the variational problem is transformed into a non-constraint problem, to facilitate the optimal solution of the Variational constraint mode and obtain an augmented Lagrange expression [25].

$$L\left( {\left\{ {u_{k} } \right\},\left\{ {w_{k} } \right\},\lambda } \right) = \alpha \sum\limits_{k = 1}^{K} {\left\| {\partial_{t} \left[ {\left( {\delta \left( t \right) + \frac{j}{\pi t}} \right)u_{k} (t)} \right]e^{ - jwkt} } \right\|_{2}^{2} } + \left\| {f(t) - u_{k} \left( t \right)} \right\|_{2}^{2} + \left[ {\lambda (t),f(t),f(t) - \sum\limits_{k = 1}^{K} {u_{k} \left( t \right)} } \right]$$

(2)

where, α is the secondary penalty factor, λ is the Lagrangian multiplier.

Finally, by using alternate direction multiplier method, the sum is optimized and the “saddle point” of Eq. (2) is searched, which is the optimal solution of Eq. (1). Expressions of $u_{k}^{n + 1}$,$w_{k}^{n + 1}$, $\lambda_{k}^{n + 1}$ are obtained by alternately updating:

$$\hat{u}_{k}^{n + 1} (w) = \frac{{\hat{f}(w) - \sum\limits_{i \ne l} {\hat{u}_{i} (w) + \frac{{\hat{u}(w)}}{2}} }}{{1 + 2\alpha (w - w_{k} )^{2} }}$$

(3)

$$w_{k}^{n + 1} = \frac{{\int_{0}^{\infty } {w\left| {\hat{u}_{k} (w)} \right|^{2} dw} }}{{\int_{0}^{\infty } {\left| {\hat{u}_{k} (w)} \right|^{2} dw} }}$$

(4)

$$\hat{\lambda }^{n + 1} (w) = \hat{\lambda }^{n} (w) + \gamma \left( {\hat{f}(w) - \sum\limits_{k} {\hat{u}_{k} }^{n + 1} (w)} \right)$$

(5)

$$\sum\nolimits_{k} {\frac{{\left\| {\hat{u}_{k}^{n + 1} - \hat{u}_{k}^{n} } \right\|_{2}^{2} }}{{\left\| {\hat{u}_{k}^{n} } \right\|_{2}^{2} }}} < \varepsilon ,n < N?$$

(6)

where, $\varepsilon$ is the convergence criterion and N is the maximum number of iterations.

IVMD algorithm

Since VMD decomposition is to decompose the original data sequence from low frequency to high frequency one by one, the low frequency IMF sequence is more likely to reflect the big trend of data, when envelope kurtosis is maximum, the frequency of IMF sequence is the highest, at this time, no further VMD decomposition is necessary. Therefore, finding the maximum envelope kurtosis means finding the optimum number of layers K. The maximum envelope kurtosis solution process is as follows:

Assuming that the number of decomposition layers of VMD is K, the envelope of each IMF can be calculated from the following equation, i.e.:

$$xt_{k}^{i} = \left| {\frac{1}{\pi t} \cdot xt_{k}^{i} (t)} \right|\quad i = 1,2, \cdots ,k$$

(7)

where, $xt_{k}^{i}$ is the absolute value obtained by Hilbert transformation, and $xt_{k}^{i} (t)$ is the ith IMF when the number of decomposition layers K.

The ith IMF envelope kurtosis calculation formula is

$$ek(i) = \frac{{E\left( {xt_{k}^{i} - \mu \left( {xt_{k}^{i} } \right)} \right)^{4} }}{{\sigma \left( {xt_{k}^{i} } \right)^{4} }}$$

(8)

where, $\mu \left( {xt_{k}^{i} } \right)$ is the mean value of $xt_{k}^{i}$; $\sigma \left( {xt_{k}^{i} } \right)$ is the standard deviation of $xt_{k}^{i}$, $E\left( {xt_{k}^{i} - \mu \left( {xt_{k}^{i} } \right)} \right)^{4}$ is the fourth order central moment of $xt_{k}^{i}$.

The envelope kurtosis $ek_{i} (i = 1,2,...,k)$ of each IMF can be calculated from Eq. (9), therefore, the local maximum envelope kurtosis $ek_{k}^{\max }$ is

$$ek_{k}^{\max } = \max \{ ek_{1} ,ek_{2} , \ldots ,ek_{k} \}$$

(9)

The local maximum value $ek_{k}^{\max }$ when K = (2, …, k) is calculated respectively from Eq. (9), and the global maximum value of maximum envelope kurtosis $ek_{g}^{\max }$ is calculated by Eq. (10), and the trend diagram of maximum envelope kurtosis is drawn. The value of K corresponding to global maximum value is the optimal decomposition layer number of maximum VMD.

$$ek_{g}^{\max } = \max \{ ek_{1}^{\max } ,ek_{{_{2} }}^{\max } , \ldots ,ek_{{_{k} }}^{\max } \}$$

(10)

Improve Whale Swarm Algorithm

WSA is a new meta-heuristic algorithm, belonging to group intelligence algorithm [26,27,28,29,30]. It is used to solve the merit-seeking problem by imitating behaviors such as searching and hunting among whale populations using ultrasonic waves as information carrier. The position iterative formula of the whale swarm algorithm is:

$$\left\{ \begin{gathered} x_{i}^{t + 1} = x_{i}^{t} + \beta \left( {y_{i}^{t} - x_{i}^{t} } \right) \hfill \\ \beta = rand(0, \, \rho_{0} e^{{ - \eta \cdot d_{X,Y} }} ) \hfill \\ \end{gathered} \right.$$

(11)

where, η is the ultrasonic attenuation factor; d_xy is the distance between X and Y; ρ₀ represents the initial ultrasonic intensity; $y_{i}^{t}$ indicates the position of whale y_i at step t; $x_{i}^{t}$ and $x_{i}^{t + 1}$ is the position of whale x_i at step t and t + 1 respectively; and β is a random number of $[0, \, \rho_{0} e^{{ - \eta d_{X,Y} }} ]$.

WSA-IC improves the iteration rule of WSA and solves the problem of getting into local optimality easily without specifying different attenuation coefficient values for different problems, and without introducing any niche parameters. WSA-IC can identify and jump out the found extremum effectively during iterative process, which is helpful to find global optimal solution.

Improvement of WSA iteration rules

Improve WSA location update rule: generate a replica of that current whale, and if the replica whale does not move to a better position than the original whale under the guidance of its “better and nearest” whale, then the original whale remains in its original position, otherwise the original whale moves to the copy whale position. Without introducing arbitrary niche parameters [31], this strategy effectively generates multiple subgroups, which enhances the local optimization ability of the algorithm and helps to find multiple optimal solutions.

Identify and jump out of found extreme points

Two new parameters, namely stability threshold T_s and fitness threshold T_f, are introduced for identifying and jumping out of the found extreme points. Firstly, the preset T_s is used to determine whether a whale has reached a steady state (i.e., an extreme point is found), and an iteration counter c is set for each whale to record the number of iterations for the un-updated position of the whale. Secondly, the global optimal solution is identified by T_f. By checking the iterative counters of the whales that have not made position updates in each iteration, if the whale reaches the steady state ($X.c = T_{s}$), the size of T_f and $|f_{gbest} - f(X)|$ determine whether to update to the current global optimal solution set, then initialize the whale to jump out of the found extreme value point. Otherwise, the iteration counter is incremented by 1.

LSTM forecasting mode

LSTM network is an improvement of recurrent neural network (RNN), which effectively solves the gradient disappearance and explosion problems existing in the original RNN by adding a gating structure. LSTM is known as a gate-based cyclic neural network. The LSTM network is able to obtain important information for a specific period of time and can maintain a certain time interval. c Stores information longer than short-term memory, but much shorter than long-term memory. The structure of the LSTM cycle unit is shown in Fig. 1, and the calculation formula is as follows:

$$c_{t} = f_{t} \odot c_{t - 1} + i_{t} \odot \tilde{c}_{t}$$

(12)

$$h_{t} = o_{t} \odot \tanh \left( {c_{t} } \right)$$

(13)

$$\tilde{c}_{t} = \tanh (W_{c} x_{t} + U_{c} h_{t - 1} + b_{c} )$$

(14)

$$f_{t} = \sigma (W_{f} x_{t} + U_{f} h_{t - 1} + b_{f} )$$

(15)

$$i_{t} = \sigma (W_{i} x_{t} + U_{i} + h_{t - 1} + b_{i} )$$

(16)

$$o_{t} = \sigma (W_{o} x_{t} + U_{o} h_{t - 1} + b_{o} )$$

(17)

where $\odot$ is the product of vector element; c_t-1 is the memory cell at the previous time; $\tilde{c}_{t}$ is the candidate state, tanh is the activation function; W, U, and b are network parameters; σ() is the Sigmoid activation function, whose output interval is (0, 1); x_t is the input at the current time; h_t-1 is the external state at the previous time; W, U and b are network parameters.

Fig. 1

The structure of the LSTM cycle unit

IVMD-WSA-IC-LSTM forecasting mode

The accuracy of WPF will be affected by the setting of LSTM hyper parameter. First, a WSA-IC-LSTM forecasting algorithm mode is designed based on the learning rate l_r, the learning rate planning r_f, the number of hidden layer neurons U_h and the number of samples b_s of the LSTM mode as the hyper parameter of the WSA-IC algorithm for finding the optimal object, and the fitness value of MSE is taken as the target function. During the training process, the corresponding optimal hyper parameter with the smallest MSE is searched. Using WSA-IC for hyper parameter search can avoid the problem of randomness caused by artificial parameter adjustment, and the optimization effect is stable.

The flow of component forecasting based on WSA-IC-LSTM is shown in Fig. 2. The steps of WAS-IC-LSTM are as follows:

• Step 1: Initialize the whale population. Including whale position initialization, iteration counter set to zero;

• Step 2: Initialize the optimization parameter. initialize superparameters such as l_r, r_f, U_h1, U_h2, bs, etc.;

• Step 3: Input training set, train LSTM network and calculate training error MSE;

• Step 4: Judge whether the MSE meets the termination condition. If the termination condition is satisfied, save the optimal hyper parameter at this time and execute step 5. Otherwise, repeat step 2;

• Step 5: The optimal hyper parameter of LSTM is set for mode training;

• Step 6: Output LSTM forecasting in preparation for subsequent work.

Fig. 2

The flow of component forecasting based on WSA-IC-LSTM

Then a combined forecasting method based on IVMD-WSA-IC-LSTM is proposed:

• Step 1: Using IVMD to decompose the preprocessed data into K IMF components;

• Step 2: Normalize each component;

• Step 3: Initialize the boundary threshold, fitness threshold, stability threshold, distance threshold and maximum evaluation times of WSA-IC algorithm, and initialize the number of whale population of WSA-IC algorithm;

• Step 4: The forecasting mode of LSTM wind power is constructed, the parameter optimization range is determined, and a whale population is generated. The learning rate l_r, learning rate planning r_f, the number of neurons in hidden layer U_h1 and U_h2, the number of batch samples bs are taken as hyper parameter variables, and the parameter optimization range is determined, and each IMF component is divided into training set and test set;

• Step 5: Defining the fitness, using the mean square error MSE of the LSTM mode output values as the fitness value;

• Step 6: Each IMF component is sent to LSTM network training, superparameter optimization using WSA-IC algorithm is used to obtain the optimalhyper parameterat minimum time of MSE. The optimal superparameter of LSTM is set for training, and the predicted value of each component is output.

• Step 7: The forecasting results of each IMF component are superimposed to obtain a final power forecasting value.

Results and discussion

Algorithm implementation and testing

Hardware platform configuration: Windows 10 operating system, 16 G memory, GTX1650 super graphics card, using Python 3.8 for programming. Select the 245-day data of Baidu KDD CUP 2022 No. 1 blower, taking 10 min as an adoption point, with 35,280 sampling points in total. The forecasting mode adopts rolling multi-step forecasting mode. Since 10 min is taken as the sampling point, the time window (input sequence length) N of forecasting of wind power is set to 24 and the measurement step length (output sequence length) l is set as 10.

IVMD decomposition

The wind power active power and wind speed series decomposed by IVMD are the data of 245 days with 10 min as a sampling point of the above data set, and the maximum envelope kurtosis method is adopted for analysis. K is taken as 2–9. By calculating the local maximum envelope kurtosis under different values of K, the K value corresponding to the global maximum envelope kurtosis is found, as shown in Table 1.

Table 1 The local maximum envelope kurtosis under different K

According to Table 1, when K = 6, the maximum envelope kurtosis is the global maximum envelope kurtosis, so the optimum number of VMD layers K = 6 is determined.

Next, the input parameters such as penalty factor of VMD α, noise tolerance τ, direct current component parameter DC, convergence tolerance criterion tol, initialized center frequency parameter init, and the number of layers K of the VMD are set as shown in Table 2.

Table 2 The number of layers of the VMD

The VMD decomposition was performed on the power and wind speed series respectively, and for the convenience of observation, 2000 sampling points were intercepted to do the decomposition results schematically. The power IVMD decomposition results are shown in Fig. 3. Adopting the power sub-sequence (IMF1-IMF6) and wind speed sequence (IMF1-IMF6) after IVMD decomposition, it can be seen from the figure that there are obvious differences in the frequency of modes among each sub-sequence, which effectively avoids the problem of mode aliasing and end effect. The new sub-sequence is constructed by reconstituting the corresponding position, using the reconstructed sequence to predict the components, and finally each forecasting results are superimposed to obtain the optimized forecasting result.

Fig. 3

The power IVMD results

Analysis of new WPF mode

In order to verify the forecasting effect of forecasting algorithm, BP neural network forecasting algorithm, RNN forecasting algorithm, LSTM network forecasting algorithm, LSTM network forecasting algorithm based on WSA-IC optimization and IVMD-LSTM forecasting algorithm are introduced. Among them, BP forecasting algorithm, RNN forecasting algorithm, LSTM forecasting algorithm and IVMD-LSTM forecasting algorithm set the same hyper parameter. The IVMD-LSTM algorithm and IVMD-WSA-IC-LSTM algorithm set the same VMD decomposition times, and the parameter settings are shown in the following table:

The IVMD-WSA-IC-LSTM algorithm mode consists of 6 sub-modes. Hyperparameters optimization is performed through WSA-IC algorithm. The optimal hyper parameter found is shown in Table 3.

Table 3 The optimal hyper parameter

Comparison and analysis of single forecasting algorithms

The forecasting effects of BP, RNN, LSTM, and WSA-IC-LSTM are compared. The hyper parameter of BP, RNN and LSTM forecasting algorithms are set artificially, and the WSA-IC-LSTM forecasting algorithm finds the optimal hyper parameter through WSA-IC. The hyper parameter settings for the BP, RNN, LSTM, and WSA-IC-LSTM forecasting algorithm modes are shown in Table 4:

Table 4 The optimal hyper parameters

The error analysis indicators for the forecasting effects of the four algorithm modes are shown in Table 5. Here, mean absolute error (MAE) refers to the average value of the absolute error between the real value and the predicted value, mean squared error (MSE) is the expected value of the residual sum of squares, R² represents the numerical feature of the relationship between a variable and other random variables.

$$MSE = \frac{1}{n}\sum\limits_{i = 1}^{n} {\left( {y_{i} - \hat{y}} \right)_{i}^{2} }$$

(18)

$$MAE = \frac{1}{n}\sum\limits_{i = 1}^{n} {\left| {y_{i} - \hat{y}_{i} } \right|}$$

(19)

$$SSR = \sum\limits_{i = 1}^{n} {\left( {\hat{y}_{i} - \overline{y}_{i} } \right)^{2} }$$

(20)

$$SST = \sum\limits_{i = 1}^{n} {\left( {y_{i} - \overline{y}_{i} } \right)^{2} }$$

(21)

$$R^{2} = \frac{SSR}{{SST}} = 1 - \frac{SSE}{{SST}}$$

(22)

where n is the number of samples, $y_{i}$ is the test value of wind power, and $\hat{y}_{i}$ is the predicted value of wind power.

Table 5 The comparison diagrams of the forecasting results

And Fig. 4 shows the comparison diagrams of the forecasting results of the BP, RNN, LSTM and WSA-IC algorithm modes with 130 sampling points.

Fig. 4

The error analysis indicators for the forecasting effects of the four algorithm modes

The effect of LSTM algorithm forecasting mode is significantly better than BP and RNN algorithm forecasting effect. Second, the WSA-IC-LSTM forecasting algorithm mode outperforms with the LSTM forecasting algorithm mode.

Conclusions

In view of the higher precision of WPF in power system, an ultrashort-term wind power forecasting method based on IVMD-WSA-IC-LSTM is proposed in this paper. And the following conclusions are drawn through comparative experiments:

• (1) IVMD determines the number of decomposition layers of VMD, and can accurately acquire the intrinsic characteristics between wind power data. Compared with LSTM, IVMD-LSTM forecasting mode has better forecasting effect, which verifies the feasibility of original wind power sequence based on IVMD decomposition processing.

• (2) The WSA-IC-LSTM forecasting mode is based on the LSTM forecasting, and the WSA-IC algorithm is added for hyperparameter search to find the smallest loss function MSE, corresponding to the learning rate l_r, the learning rate attenuation r_f, the number of hidden layer units U_h1 and U_h2 and the number of batch samples bs. By comparing the forecasting results of the two modes, it is found that the forecasting effect of WSA-IC-LSTM is better than LSTM. The validity of hyper parameter optimization based on intelligent optimization algorithm (WSA-IC) is verified.

• (3) Further in-depth comparison analysis of the forecasting effects of the IVMD-WSA-IC-LSTM forecasting mode and the IVMD-LSTM forecasting mode, combined with the above findings, led to the conclusion that the mode presented in this paper has the highest forecasting accuracy. The predicted MAE was reduced to 0.007859, MSE decreased to 0.00011, and R² increased to 0.998828. The feasibility is verified.