Multivariate Wind Speed Forecasting with LSTMs for Wind Farm Performance Estimation

One of the most important renewable energy sources today is wind power generation. However, this energy source depends on the flow of air in the area where the wind generators are installed, and as with many other renewable resources, this is a natural resource whose magnitude is not directly controlled by man. The prediction of wind speed then becomes a key problem when we want to project the energy performance of a wind farm. The behaviour of the wind, as a climatic variable, can be estimated from various atmospheric parameters such as temperature, humidity and air pressure. In this paper we propose a multivariate wind speed estimation model from the history of these atmospheric parameters using a deep neural network. The performance of the model is then evaluated against the same historical data, which produces a fairly small error. The code is implemented in Keras deep learning library with TensorFlow backend. Keyword Atmospheric parameters, Deep learning, Forecasting, Kinematics, Multivariate model, Wind speed


II. PROBLEM FORMULATION
Jepírachi means northeastern winds (winds coming from the northeast in the direction of Cabo de la Vela) in Wayuunaiki, the native Wayuu language of the indigenous population of the Guajira (Colombia). This is also the name given to the first wind farm for the generation of wind energy built in the country. The wind farm is located in the northeastern region of the Colombian Atlantic Coast, between the towns of Cabo de la Vela and Puerto Bolivar, in the vicinity of Bahía Portete, in the municipality of Uribia. It has an installed capacity of 19.5 MW of nominal power, with 15 wind turbines of 1.3 MW each, subjected to the trade winds that blow almost all year round in this part of the peninsula, at an average of 9.8 meters per second. The machines are distributed in two rows of eight and seven machines respectively, in an area approximately one kilometer long parallel to the beach and 1.2 kilometres wide to the north of the Kasiwolinrancheria and to the west of the Arutkajuirancheria.
This pilot facility also functions as an experimental research center of the EmpresasPúblicas de Medellín (EPM). Numerous research projects have been proposed on this plant, its operation and development. In our case, we seek to establish the projection and future performance in terms of electricity generation capacity. As part of this work we are developing models of wind behaviour and prediction in the area. From this idea, we propose the use of machine learning strategies for the construction of these models from the historical climatic behaviour.
We are interested in making a multivariable model using a Long Short-Term Memory (LSTM) network. From previous research, and the nature of historical climate data, it is logical to think about using recurrent networks for the development of the model. This persistence of information or memory brings the behaviour of the network to a sequence, a characteristic present in climate data that we want to replicate to estimate the wind speed.
LSTMs are a very special type of recurrent neural networks. In general they work much better than traditional recurrent networks, particularly when it comes to long-term dependencies, in fact, they were specifically designed to avoid this problem. In short, LSTMs are networks capable of remembering information for long periods of time, and they do so naturally. They do this because instead of using a single connection layer for the inputs, she has four interacting with each other in a special way.
Using the LSTM we seek to find a behaviour model for wind speed in the area of operation of the Jepírachi wind farm that contemplates the history of several climatic parameters, including: temperature, humidity, precipitation and air pressure.
III. METHODOLOGY The Long Short-Term Memory (LSTM) recurrent neural networks are able to almost seamlessly model problems with multiple input variables. For this reason this type of networks become a great tool in time series forecasting, particularly when the problem is characterized by dependence on multiple variables. This is also a great advantage over classical linear methods.

A. Wind Speed Forecasting
In the estimate we are using a weather dataset. These data correspond to daily readings for ten years at the Rioacha airport Almirante Padilla (Guajira, Colombia). The data includes date, temperature, wind speed and weather. The complete feature list in the raw data is as follows:  No: Row number We use this data to estimate the future behaviour of the wind. That is, given some weather conditions in previous days, what will be the wind speed in the next few days. This dataset can also be used to estimate other variables, however the particular interest of this research is the future estimation of the electrical performance of the Jepírachi wind park.

B. Data Preparation
The raw dataset must first be prepared for use with Pandas. In particular, we must first consolidate the date information in a single column in order to use it later as an index. The original raw dataset is shown in Fig. 1. The script in Fig. 2 loads the raw dataset and parses the date information as the Pandas DataFrame index. Then we remove the "No" column and specify the names for each column. We keep the same names defined before, and print the first 10 resulting rows to verify the result (Fig. 3). The processed dataset is saved again with a new name (raw_2b.csv), but maintaining its format. To facilitate the analysis of behaviour, we made plots of each series within the dataset (Fig. 4). Again the curves have been organized in the same order, from top to bottom: wind, tmax, tmin, humi, prec and pres. The figure shows the repetitive cyclic behaviour of the data, some dependence that relates them, and some atypical data. Fig. 4. Behaviour of each column of data within the dataset. From top to bottom: wind, maximum temperature, minimum temperature, humidity, precipitation and pressure

C. Multivariate LSTM Forecast Model
In order to make the model fit it is first necessary to formulate the prediction as a supervised learning problem. As an initial test we used the data to predict the behaviour of the wind (wind speed) on the current day (instant t) according to the values of wind and weather of the previous day (instant t-1). This is only one option that allows to establish a solution, and that coincides with the objective of establishing the daily performance of the wind farm. Other possible formulation alternatives (and maybe more interesting) could be to make the prediction but considering a longer data history (one month, one year, for example), or project some other weather parameter.
To process the data, we first normalize all input variables in the range 0 to 1. Fig. 5 shows the first five rows of the transformed dataset and the prediction of the wind variable. We show the six input variables at instant t-1 (input values), and the output variable at instant t (wind speed on the current day). From this point the model behaviour analysis is performed, examining wind speed predictions using as input variables data from more than one day in the past. The objective is to exploit the time backpropagation of the LSTM network for the projection of the time sequence. The LSTM-based model has multiple input variables, and the dataset is used for both training and model evaluation. We fit the model on the first eight years of data, and then evaluate it on the remaining three years of data. In this sense we divide the dataset into training and test sets. Then, we divide these two sets into input and output variables. Finally, the inputs are reshaped into the 3D format expected by LSTMs, i. e., [samples, timesteps, features]. The script in Fig. 6 takes care of these tasks. The final format of the inputs is (Eq. 1): ((2920, 1, 6), (2920,), (621, 1, 6), (621,)) (1) That is, 2920 days of data for training, and 621 days of data for testing. Fig. 6. Splitting data in training and testing, and into input and output variables We defined the LSTM network with 200 neurons in the first hidden layer, and one neuron in the output layer to predict wind speed. The input shape will be one time step with six features. We use the Mean Absolute Error (MAE) loss function and the efficient Adam version of stochastic gradient descent. The model will be fit for 100 training epochs with a batch size of 72. The script in Fig. 7 takes care of these tasks. The raw dataset (in total 3542 daily readings corresponding to the last 10 years, from 1 January 2009 to 12 September 2018) correspond to measurements made at the Rioacha airport Almirante Padilla (Guajira, Colombia). The site is located 13 m above sea level to the southwest of the city. We use the model described in the previous section to test the entire dataset. We combine the forecast with the test dataset and invert the scaling. We also invert scaling on the test dataset with the expected wind speed values. With forecasts and actual values in their original scale, we can calculate an error score for the model. In this case, we calculate the Root Mean Squared Error (RMSE) that gives error in the same units as the variable itself. The script in Fig. 8 takes care of these tasks. In the end, the model's RMSE was 4.223 km/h. Given the speed values, this is a fairly small error for the model. We also plot the curves of train and test loss from the LSTM during training (Fig. 9).

V. CONCLUSION
In this paper we developed a multivariate model for the estimation of wind speed using LSTM networks. The model was trained and validated using weather data from the last 10 years recorded at the Rioacha airport Almirante Padilla (Guajira, Colombia). The wind prediction is performed as an input for the performance estimation of the Jepírachi wind farm located in this city. The model was validated with the same training data (18% of the data) and the results were submitted to the Root Mean Squared Error (RMSE) calculation. The RMSE value of the model was 4.223 km/h, a value that guarantees a good performance of the model. ACKNOWLEDGMENT This work was supported by the Universidad Distrital Francisco José de Caldas, in part through CIDC, and partly by the Facultad Tecnológica. The views expressed in this paper are not necessarily endorsed by Universidad Distrital. The authors thank the research group ARMOS for the evaluation carried out on prototypes of ideas and strategies.