Application of LSTM neural network based UAV attitude control in attitude estimation

. From now on, Unmanned Aerial Vehicle (UAV) technologies have become more mature, and UAV has been skillfully used in various regions under this mature technology. However, during UAV flight, they often encounter interference from external factors such as airflow, wind and temperature changes, which pose challenges to their stability and flight accuracy. This paper proposes an algorithm what is based on LSTM (Long, Short-Term Memory network), aiming that the flight attitude is affected when it is subject to external interference, while still ensuring the robustness of its flight performance in this state. In this paper, the optimization method of anti-jamming adaptive adjustment based on LSTM. In the meantime, a UAV dynamic model and neural network control based on dynamic modeling are established. Then article established the LSTM rule and designed the PID controller of LSTM. And to prove the effectiveness of the designed PID controller by pilot experiments with Matlab / simulated link simulation and flight experiments. Finally, step size response, autonomous tracking, robustness and anti-interference were tested by experimental simulations.


Introduction
The full name of the UAV is Unmanned Aerial Vehicle.The UAV is an unmanned aerial vehicle moving based on the setting program or the real-time remote control [1].In the civilian field, Unmanned Aerial Vehicles, as a nonlinear underdrive coupling system, have the characteristics of small size, small space required for takeoff and landing, simple operability and good stability.It can play a special role in aerial photography, medical treatment, rescue, construction, etc.
In this area, the attitude control of UAV is very important.Nowadays, common UAV control systems include PID(proportional integral derivative),NN(Neural Network),DL(deep learning), backstepping method, neural network method, etc., of which PID control is widely used UAVs due to its low computational cost.forinstance, The combination of TD (the tracking differentiator), ESO (the extended state observer), and NPID (the nonlinear PID) developed with PID can effectively improve the effectiveness of the controller.The automatic disturbance rejection controller (ADRC), thus forming a new "automatic disturbance rejection control" technology.The new controller has the characteristics of simple algorithm and easy parameter adjustment.PID controller is a very classic control algorithm [2] [9], which is a linear controller,When using the PID controller, the state of the machine can be adjusted according to the desired error to achieve the expectation At the same time, the conventional PID control system has problems in the precise control aspect.Because the PID controller is a linear controller, while in reality, it is mostly nonlinear.With linearity, the accuracy will decrease.In terms of the fine control.There are also ways of improvement.Common improvement methods include adding a filter in the PID algorithm, or a compensation algorithm.
Recursive [3] neural network is a simple extension of neural network, neurons will be arranged in one layer by layer, and each neuron is not connected to the same layer and the next layer, only to the upper layer, the neurons will contact the output of the previous layer, and as input, transmitted to the next layer, there is no feedback between each layer.Recursive neural network is one of the welldeveloped, extensive artificial neural networks.
Unlike ordinary feedforward neural networks, RNN (Recurrent Neural Network) has recurrent connections, taking the hidden state of the current input and the previous moment as input at each time step, while outputting the hidden state of the current time step.This repeated hidden state transfer allows the RNN to retain historical information while processing sequence data and to be able to model the entire sequence.By using the same parameters at each time step, the RNN can process the arbitrary length of each input sequence.However, RNN has problem such as gradient disappearance or gradient explosion.In view of this problem, some tries to change the structure of RNN, so that the improved neural networks such as LSTM (Long,Short-Term Memory network) can become a development direction of RNN.
In short, RNN can process sequence data, model and predict sequence data through circular connections and hidden state transfer mechanisms.The application of RNN is often associated with natural language processing.
By understanding the distinction between the neural network model and the traditional neural network, [11] can find the difference between the two.The traditional network layer is fully connected, and the nodes in the layer are not connected.This structure leads the network to be fully used to process sequence data.RNN, on the other hand, adopts a circular structure that allows information to be transmitted between neurons within the layer, which can capture the timing information well.The network structure of the RNN is shown in the following figure.Specifically, The hidden layer is different from the ordinary network layer.In a typical neural network, the input of a hidden layer is a combination of the output from the previous layer and the output from the remaining hidden layers.The output of each layer is passed as input to the next layer, and this process continues until the final output layer is reached.This allows the network to learn and extract higherlevel features and patterns from the input data.The most important feature of RNN is parameter sharing at each step.The RNN has the following advantages over the other algorithms.The RNN has a memory mechanism that can influence the current output based on past input information.This allows the RNN to better handle sequence data such as time series, text, etc.The RNN is able to accept input sequences of variable length.Unlike traditional feedforward neural networks, the hidden layer states of the RNN will be unfolded along the time steps and remain shared between each time step.The RNN is able to consider the contextual information.By passing the previous time step to the current time step, the RNN can better understand and represent the context relations in natural language data such as text or speech.While traditional feedforward neural networks are prone to the problem of gradient disappearance or explosion when dealing with long-term dependencies, RNN through cyclic connections is able to trace trends in time, so as to better deal with long-term dependencies.Due to its advantages, RNN has been successfully applied in machine translation, speech recognition, language model, sentiment analysis, stock prediction and many other fields.In conclusion, the advantage of RNN is its ability to process sequence data, consider contextual information, solve long-term dependency problems, and has achieved good results in many practical applications.

Problem formulation
1.1.1.network model.UAV angular offset is three-dimensional, nonlinear angular variation.With the growth of the number of input features, the efficiency of network model decreases in exponential form, which is very unfavorable to the application in this aspect.In the case of targeted disturbance resistance.Basicly, Sufficiently complex, superimposed neural network can significantly improve the accuracy of the algorithm used.
Most neural network models can divide three parts: input layer,hidden layer,output layer.Between these, only the number of layers in the hidden layer is uncertain.The layers number of the hidden layer is determined according to the complexity of the problem and the accuracy of the algorithm.In some simple problems, the hidden layer can not exist, the whole model only input layer and output layer; in some complex problems, the hidden layer can exist many, according to the difficulty of the problem.Nodes in each layer of such models can be called neurons.Specifically, the neurons are located in the input layer, which corresponds to the features of the training data.While the neurons in the hidden and output layers can be represented by the activation function.Activation function There are many types.
Neural networks can make decisions by learning and adapting to input data in different environments.For UAVs attitude control, this means that they can obtain environmental information by sensing and processing the sensor data, and make intelligent decisions based on that information.Compared with traditional UAV control methods, neural networks have more powerful adaptability and generalization ability, and can deal with complex and changeable environments and tasks.Neural networks can improve the control performance of UAVs through large-scale data training.By using a deep neural network structure, we can train the model using the powerful computational power and parallel processing.This allows the UAV to learn complex perception-action mapping and make accurate and efficient control decisions in a real-time environment.
Moreover, due to the end-to-end learning ability of neural networks, the perception, decision making and execution of UAVs can be integrated into a unified system.This feature makes the robot not need to rely on pre-defined control rules or expert knowledge.Instead, robots can complete the task by learning from the data, overcoming the limitations of traditional control methods.The neural networks are scalable and adaptable in terms of UAV attitude control.By changing the network structure and parameter settings, we can customize the controller design for different tasks and environments.This allows the UAV to adapt to needs in different scenarios and quickly adapt to new tasks and environments.
In conclusion, neural networks have the advantages of strong adaptability, data-driven, end-to-end learning, and scalability in UAV attitude control.These characteristics make the neural network become a powerful tool to realize the intelligent UAV attitude control, and promote the development and application of robot technology.Interference during flight will lead to flight instability, Angle deviation and other conditions, etc.This paper studies the correction of Angle deviation.Corresponding to separate categories, we wanted to train a neural network model that determines the feedback intensity according to the size of the flight angle change during UAV interference.Through the mathematical simulation to obtain the sample data.

learning algorithm
Learning algorithms are algorithms that can make the data learning continuously to improve the computation accuracy.Learning algorithm is a network model built to imitating human thinking mode and neural network, so as to analyze and learn.It interprets data by simulating human brain, such as images, sounds and text, which can use learning-based algorithms.Learning-based algorithms can integrate simple low-level representations to form more abstract highlevel representations to classify properties or features, and thus represent them through distributed features of the data.Deep learning structures can have a multi-layer perceptron with multiple hidden layers.When using this algorithm, the efficient and accurate learning method is the advantage compared with other algorithms.At the same time, the learning method of changing the algorithm also has the characteristics of using unsupervised or semi-supervised features instead of manual feature learning, as well as a more efficient hierarchical feature extraction algorithm.
Learning-based algorithms have many advantages in UAV control.First, the learning algorithm can analyze and understand the UAV flight data to extract features and patterns critical to the control task.This enables the algorithm to adapt adaptively to changes in different environments and tasks, enabling more efficient and precise flight control.
Secondly, learning algorithms can continuously improve the performance of UAV by accumulating and learning experience.Through trial and learning, the algorithm can gradually improve its accuracy and stability, so that the control ability of UAV can be rapidly improved in different tasks and complex environments.
Moreover, learning algorithms can also overcome the challenges posed by non-linear and unknown environmental factors.By building complex nonlinear models or using deep learning techniques, learning algorithms can better understand and adapt the interactions between the UAV and the environment, and predict and handle a variety of nonlinear effects.This allows UV to make more flexible and intelligent decisions when faced with complex environments.
Finally, in the field of autonomous flight, learning algorithms can also provide decision-making assistance and autonomous flight functions for UV Vs through machine learning methods.Based on the existing learning experience, learning algorithms can help UAV quickly adapt to the changing environment and situations in different tasks, and make independent decisions to achieve specific goals.
In conclusion, the advantages of learning-based algorithms in UAV control are reflected in their ability to adapt to data, make progress through accumulating experience, overcome nonlinear and unknown environmental challenges, as well as provide auxiliary decision-making and autonomous flight functions.These features make learning-based algorithms an important means to improve the performance and intelligence of UAV control.

model of Unmanned Aerial Vehicle
UAV is a vehicle with four rotor, with X distribution and cross distribution.The X-type quadrotor is used in this paper.The quadrotor UAV can only achieve various movements by changing the rotational speed of the motor with the morphology of the propeller blade.The physical model of a quadrotor UAV can be viewed as a centrosymmetric The UAV (The UAV studied in this article and the established mathematical model of UAV are all quadrotor) is a vehicle with four rotor whose flight is affected by various forces and moments.For accurate and effective control, this paper establish a suitable mathematical model to describe its kinematic and kinetic properties.This paper describes its pose using Euler angle, where Euler angle includes roll angle, pitch angle and yaw angle.Furthermore, we need to define position coordinates and linear velocity to describe their position and velocity states.The movement of the quadrotor drone is driven by four electric motors, which do the thrust by controlling the speed of the rotation.Each rotating motor generates a pulling force up, and it also causes a torque.The balance of these forces and torque and the interaction between the three rotation axes will determine the movement of the quadrotor drone.In order to establish a mathematical model of the quadrotor UAV, the mass and inertia matrix of the aircraft, as well as the aerodynamic parameters.By using Newton's law, you can write equations of motion for a quadrotor drone in three directions.These equations will include the derivatives of the linear velocity, position, and attitude, and can further improve this mathematical model to more precisely describe the movement behavior of the quadrotor UAV under different environmental conditions.By modeling the external perturbations and various environmental factors in the model, we are able to better predict the operation of the aircraft and carry out more advanced control algorithms.In summary,the mathematical model of the quadrotor UAV involves the description of the kinematics and dynamics, which needs to consider its attitude, position, and the influence of the dynamical system.

Dynical system modeling of UAV.
There are triaxial position, velocity, and acceleration for line motion; triaxial attitude angle for angular motion [5]; this is a total of 15 states.By Newton's second law is: Here, F and acceleration A are both expressed under the geographical system (F represents generated the four rotorstotal lift), with further expansion of the upper formula available R represents the conversion matrix from the airframe coordinate to the geographic coordinate system.Since i f , 4 ... 1 i = represents the lift of the four rotors along the negative direction of the Z axis of the body, it needs to be converted to the geographic system The model representation of line motion can be obtained by further simplifying the (1)(2)(3) formula The modeling process of linear motion is very simple, but multiple rotors can not only move but also rotate, and this rotation process requires the knowledge of the rotation of the rigid body.
How the object rotates is related to the torque produced by the rotor.The torque and force F and distance l have the following formulas: In the upper formula, F is the the rotors total lift, represent the control torque produced by the rotor action in the three attitude channels, respectively.
According to the Euler equation Where ε is the angular acceleration, ω is the angular velocity,M is the attitude channel control moment; The relationship between Euler angular velocity and body angular velocity is The disturbance of the drone is reflected at small angles,so the article can think that W=I.The four propeller torques of the quadrotor UAV offset each other, with a total torque of 0  .For a given input, only the values of the language variable near the input point have large membership values, and the membership of the language variable value away from the input point is either small or 0. There is only a small number of nodes output non-0 in A, while the output of most nodes is 0.The number of nodes in the fourth layer is the same as that in the three layers,

Structural design
Based on the above for the description of neural network, and the application of uav resistance environment, can get research problem: in considering the 3D motion of drones and drones will face a variety of environmental problems, how to apply Recurrent neural network to drone resistance level and successfully improve its stability.
Uav interference can be divided into the following parts: environmental interference, fuselage interference, and reference interference.Environmental interference includes but is not limited to storm, rainstorm, air humidity change, pressure change, airflow disturbance, fuselage interference includes but is not limited to wing damage, underpower, and rotor control.Reference interference includes but is not limited to using uav to tracking the moving object.UAV is divided into two kinds of disturbance, linear displacement disturbance, and angular perturbation.Any perturbation can be simulated with a superposition of the two.

LSTM neural network design
Feedback recovery by linear displacement when generating line perturbations.In generating the angular disturbance, the propeller x-type is the surface, with the center as the axis.With the axis as the reference frame, specific feedback is performed to recover when the x four corners produce a relative offset.Finally, add the change vector of the two cases to complete the feedback recovery after perturbation.
At the same time, the difference between transient disturbances and persistent perturbations will be an important problem affecting the stability after recovery

UAV Attitude and LSTM NN
In this paper,UAV attitude calculation scheme based on LSTM is designed, and The computational flow of this design is presented on the Fig 7 .In this way, through understanding the data of the time of the first K, the nonlinear complex relation of postures is fitted ,It is shown that it fits the nonlinear complex The input training was done via IMU sensor data by data analysis in Fig. 7. Overall, GYRO, ACC, and MAG are three terms with nine inputs.Training of the output was done via complementary filtering algorithm.Over all, the roll, pitch, yaw, totally are three terms of nine outputs.Since the error distance of the predicted value can be determined by dividing the mean error,this method can be used for the prediction performance.The smaller the mean squared error, the smaller the prediction error, the following is the calculation method,  [10] This experiment uses MATLAB as the simulation software, and the framework of deep learning uses the configuration of Ubond + Pytho + Keras, respectively.

Simulation Results and Analysis
In order to make the experimental data realistic, this paper developed a UAV platform to collect data for experimental feedback.Figure 8 shows the sensor data of accelerometer, gyroscope.More than 8,000 simulation experiments yielded relatively stable data.

Conclusion
Through the data logic received by sensors, a neural network control method for controlling the UAV immunity feedback attitude.Experiments show that for UAV attitude control and resistance to interference, the introduction of neural network will make UAV attitude control can be compared with the traditional control higher precision control, UAV attitude control under the complex calculation can show higher robustness, in the face of different types of interference have good performance.In future research, UAV attitude control in the extreme environment.At the same time, we need to consider the improvement problems including the universality of the algorithm, and try to expand the research object from quadrotor UAV to most UAV.

2. 1 . 1 .
Structure of Unmanned Aerial VehicleThe diagonal propeller turns the same[1][6][9], and the adjacent propeller turns the opposite, to offset the torque.The blades are streamlined in design.It can bring about an upward lift when rotating.Assuming the propeller angle, the blade deflection angle cannot be changed, but only the multiple-direction motion can be changed by changing the propeller steering and rotation speed.

Figure 6 . 2 .
Figure 6.Neural Network.Take the common Neural Network[7][8][9], for example.Input layer is the first layer,and the input value is input to the next.In the second layer,each node can correspond to the same layer of the previous set of nodes.The number of nodes i m is relative to the

4 ,..
as the same time,It normalized the applicability of each rule The fifth layer is the output layer, enabling a Proceedings of the 2023 International Conference on Machine Learning and Automation DOI: 10.54254/2755-2721/42/20230701It can be deduced that.
Proceedings of the 2023 International Conference on Machine Learning and Automation DOI: 10.54254/2755-2721/42/20230701 relations of the poses.At the same time,The study predicted thatthe data of the time of the K+1,and the postures are calculated.

Figure 8 .
Figure 8.The sensor data from IMU.