Model of Autonomous Positioning Through Associating Environment Memory Information

Du Jia,Wu Dewei,Zhou Yang

1.College of Information and Navigation,Air Eorce Engineering University,Xi′an 710077,P.R.China;

2.Xi′an Communications Institute,Xi′an 710065,P.R.China

Model of Autonomous Positioning Through Associating Environment Memory Information

Du Jia1,2*,Wu Dewei1,Zhou Yang1

1.College of Information and Navigation,Air Eorce Engineering University,Xi′an 710077,P.R.China;

2.Xi′an Communications Institute,Xi′an 710065,P.R.China

A model of autonomous positioning through associating environment memory information is presented for unmanned combat aerial vehicle(UCAV).The representation strategy of environment by constructing place cells is used to produce the memory information,and the landmarks in memory are retrieved through perceiving and processing the environment.During UCAV′s flight,the landmarks are obtained in real-time and are matched with the landmarks in memory.Then,the idea of ranging positioning is adopted to calculate UCAV′s location based on the corresponding relationship between current obtained landmarks and the memorized landmarks.Simulation shows that the proposed model can realize autonomous positioning in the memorized environment,and the positioning performance is well when the sensor has a high precision.

autonomous positioning;environment memory;environment representation;landmarks′positioning; feature point

0 Introduction

The positioning is the foundation of the navigation process of unmanned combat aerial vehicle (UCAV).Considering UCAV′s intelligence and autonomous capability,we hope UCAV can realize positioning by itself[1].Usually,methods of vehicle′s positioning can be categorized into two ways when there is no information fusion.One way is based on the perception of self-motion. The other way is based on the perception of external environment information.The first way will produce accumulative positioning error when implementing positioning alone[2-4].Eor the second way,the positioning is implemented discretely, and the positioning results at different moments are irrelevant.Therefore,the second way can behave well when the environment reference is available during positioning.Besides,human′s positioning mode also largely depends on the memory of the environment.If the vehicle can know its location by combining the external reference with the memory,the intelligence and autonomous ability of positioning will be improved greatly.

Considering the attribute of UCAV,three basic problems should be solved when the positioning is implemented based on external environment information.Eirst,which way is used to describe the environment?Currently,the vision sensor is the main equipment to perceive the environment,and the visual information can be expressed by images in the application.Therefore, the description of the environment can be achieved by describing images[5-6].Second,which way is used to memorize the description and obtain the representation of the environment?Eor this issue,we need consider whether the storage mode is suitable for searching,adding and updating efficiently.At present,the algorithms with learning and memorizing ability can be used tosolve this problem,such as autonomous development algorithm[7]and artificial neural network[8]. Third,which way is used to calculate UCAV′s location based on the perceived environment and the memory?Eor the above mentioned problems, we have carried out some studies on the representation of environment and the positioning of landmarks[9-10],but the positioning of UCAV has not been solved.Therefore,the research is deepened here,and a model of autonomous positioning through associating environment memory information is presented.

*Corresponding author,E-mail address:dujia0227@126.com.

How to cite this article:Du Jia,Wu Dewei,Zhou Yang.Model of autonomous positioning through associating environ

ment memory information[J].Trans.Nanjing Univ.Aero.Astro.,2016,33(5):584-594.

http://dx.doi.org/10.16356/j.1005-1120.2016.05.584

1 Autonomous Positioning Model

The proposed autonomous positioning model is divided into three parts.The representation of environment,the positioning of landmarks and the positioning of UCAV.The representation of environment is to memorize the perceived environment.The positioning of landmarks is to obtain landmarks′location in memory,which is implemented directly by retrieving the environment representation knowledge.The positioning of UCAV is to calculate UCAV′s location.

The main idea of the model is summarized as follows:UCAV constructs the inside representation to memorize the environment.During UCAV′s flight in the memorized environment,it perceives the environment by vision sensor in real-time,and the corresponding landmarks in the environment are obtained.Then,the landmarks′location in sensor coordinate system is calculated based on the imaging mechanism of vision sensor. Simultaneously,the landmarks′location in global coordinate system is obtained by retrieving the memory.Therefore,the sensor′s location in global coordinate system can be calculated based on the landmarks′location in different coordinate system.At last,some transformations between the different coordinate systems are implemented to obtain UCAV′s location.

1.1 Environment representation

Through long-term research,animal neurologists find that rodents′hippocampus plays a key role in animal navigation.In hippocampus,the firing activity of biological place cells exhibits strong location selectivity.A single place cell can represent a specific location in the environment, and the firing activity of the whole place cells can describe and represent the entire environment[11-12].This biological characteristic can be applied to UCAV′s positioning to represent the environment,and the memory about the environment can be produced by constructing place cells. As a result,the environment representation knowledge is produced.

Fig.1 shows the process of representing the environment by constructing place cells.It makes use of the robustness of feature points extracted by speeding up robust features(SURE)algorithm[13].Besides,the memory ability of incremental hierarchical discriminant regression (IHDR)[7,14]is adopted.The basic process of environment representation is as follows:Eirst, SURE algorithm is used to extract the robust feature points in the perceived image.Then,the feature points are regarded as the landmarks,and the landmarks set are formed by combining landmarks′description vectors with landmarks′location.Next,the landmarks set are used as the input of IHDR,and they are clustered into different states after the processing of IHDR.Einally,a single state is regarded as a place cell to represent a location and the environment representation knowledge is expressed by the whole place cells.

This environment representation model is analyzed in Ref.[9]in detail,which points out that this model is effective and the produced knowledge can identify the environment by analyzing the corresponding relation between the real-time landmarks and the place cells.

1.2 Landmarks′positioning

UCAV can obtain a number of landmarks when it flies in the memorized environment.If current landmarks are associated with some information in the environment representation knowledge,the landmarks′location in memory can be retrieved directly.Because of the robustness offeature points,some landmarks can be described well even though the image captured by vision sensors changes along with scaling,rotation and noise pollution.Therefore,the landmarks′description vector is used to retrieve the environment representation knowledge during landmarks′positioning.

Fig.1 Process of representing environment by constructing place cells

Fig.2 shows the process of landmarks′positioning.Eirst,SURE algorithm is used to extract the feature points in the perceived image,and the landmark′s description vector x′tis obtained. Then,x′tis used to retrieve the environment representation knowledge,and the description vector ¯xiand the location¯yiin the memory are obtained (In the retrieval process,the distance between landmark′s description vector and the clusters of IHDR is calculated to search the cluster with shortest distance.Then¯xiand¯yiof the searched cluster is exported).Einally,x′tand¯xiare purified by ratio method,and all purified landmarks are marked as the landmarks that can be positioned.Simultaneously,the corresponding¯yiis used as the location of landmark i.Through the above steps,the landmarks′positioning is finished.The detail process is analyzed in Ref.[10], which points out that the positioning accuracy of landmarks is high in small noise pollution.Here we do not discuss it in detail.

Fig.2 Landmarks′positioning process

1.3 UCAV′s positioning

Fig.3 shows the process of UCAV′s positioning.The image is perceived by vision sensor (e.g.,binocular vision system)in real-time,and the landmarks with 64-dimension description vectors are obtained by extracting feature points. Then,the landmarks′description vector is used to retrieve the environment representation knowledge.As a result,the landmarks′location in global coordinate system is obtained,denoted by (xnmi,ynmi,znmi).Simultaneously,the landmarks′location in sensor coordinate system is calculated through the imaging mechanism of vision sensor, denoted by(xsmi,ysmi,zsmi).The core formula of the imaging mechanism is as follows

Fig.3 UCAV′s positioning process

where S is the scale factor and[umi,vmi]Tthe landmarks′location in the image.fuand fvare the focal length,[u0,v0]Tis known as the reference point,which is the intersection point between image plane and the optical axis,u0and v0are the internal parameters of the vision sensor. Since[umi,vmi]Tis obtained by extracting feature points,(xsmi,ysmi,zsmi)can be obtained directly by the sensor.

Then,the idea of ranging positioning is adopted to calculate UCAV′s location in global coordinate system based on the(xnmi,ynmi,znmi)and (xsmi,ysmi,zsmi).The detail is as follows:Let(xns, yns,zns)be the vision sensor′s location in global coordinate system.Let(xnu,ynu,znu)be UCAV′s location in global coordinate system.Because the distance between the landmarks and the vision sensor is fixed,we can obtain

(xns-xnmi)2+(yns-ynmi)2+(zns-znmi)2=

xs2mi+ys2mi+zs2mii=1,2,…,N (2) where N is the landmarks′number.As a result, the number of nonlinear equations is also N.

Next,the positioning method of Caffery[15]is used to solve the equations.

where

Usually,the vision sensor is fixed on the vehicle,so the relationship between vision sensor′s location and UCAV′s location can be obtained directly.Let pbsbbe the vision sensor′s location in the body coordinate system.Besides,let Cnbbe the direction cosine matrix from the body coordinate system to global coordinate system.Therefore,UCAV′s location in the global coordinate system can be obtained

Through the above process,UCAV′s location can be calculated when more than three landmarks are obtained.

2 Results and Analysis

Eirst,the representation of environment and the positioning of landmarks are analyzed by simulation.Second,the performance of autonomous positioning through associating environment memory information is discussed.

2.1 Analysis of environment representation and landmarks′positioning

(1)Extract feature points in initial perceived image by SURE algorithm.The initial perceived image is shown in Fig.4.Its size is 326 pixel× 400 pixel.The extracted feature points are shown in Fig.5.The number of feature points is 656. Then,construct place cells according to the model mentioned in Section 1.1,so the representation of environment is achieved,and UCAV can memorize the experienced environment.In the simulation,the parameters of IHDR are set as follows: The dimension of input space is 64.The dimension of output space is 2.The number of nodes qsis 20.Amnesic parameters t1,t2,b and m are set to 5,20,1 and 100,respectively.The detail content is introduced in Ref.[10].

(2)Initial perceived image is polluted by Gaussian noise,and the pollution image is regarded as the test image.Gaussian noise′s average is set to 0,and its variance is divided into 0.01, 0.03,and 0.08.Then,the feature points in the test image are extracted by SURE algorithm toobtain the test landmarks.Simultaneously,the landmarks′location in the test image is regarded as the accurate reference location.

Fig.4 Initial perceived image

Fig.5 Eeature points in perceived image

(3)The description vector of test landmarks is used to retrieve the representation information to obtain the correspondingandin memory.

(4)According to the landmarks′description vector in test image and the retrieved description vector in memory,the test landmarks are purified by ratio method.

(5)The purified landmarks′location is used as positioning result,and the landmarks′positioning is finished.

(6)Compute the error between positioning result and reference location,and compare the error with the threshold of location error(The setting of location error′s threshold is to evaluate the excellent or worse of the positioning results.If the positioning error is lower than location error′s threshold,the positioning result is considered exact.In the paper,the location error′s threshold is set to 1.414).The positioning accuracy rate is analyzed.The positioning accuracy rate is the ratio that the number of purified landmarks whose positioning error is lower than location error′s threshold relative to the total number of purified landmarks.

Fig.6 shows the corresponding relationship between the positioning result measured by the proposed model and the reference location obtained by SURE algorithm.In the simulation, the threshold of ratio method is 0.6.In Fig.6, the reference locations are shown as the points in the left of the lines,and the positioning results are shown as the points in the right of the lines. Simulation validates that the proposed model can realize the memorizing of perceived environment and the positioning of the landmarks.

Fig.6 Corresponding relationship between positioning result and reference location

2.2 Performance of autonomous positioning through associating environment memory information

In this section,we simplify the simulation process,including:(1)The barycenter of UCAV is consistent with the origin of sensor coordinate system;(2)the capture of real-time image andthe extraction of feature points are elided,and the randomly produced points in the horizon are regarded as the landmarks;(3)during UCAV′s flight,the landmarks′number that UCAV can perceive is more than 3,and the perceived landmarks can be found in the environment representation knowledge.

Based on the above simplifications,the location of UCAV on the horizon is denoted by(xnu, ynu),which is calculated by Eq.(3).The flight height is denoted by znu,and it is obtained based on Eq.(2).The flight process in the simulation includes level flying,climbing and down.The flight trajectory and the distribution of landmarks are shown in Fig.7.The flight speed is 40 m/s. The total fight time is 175 s.The positioning period is set to 1 s.

Fig.7 Elight trajectory and landmarks

The simulation contents include:(1)The positioning performance in the different sensor precision is analyzed at certain location;(2)the positioning performance in the different landmarks′number is analyzed at certain location;(3)during the whole flight process,the positioning result in different sensor precision is analyzed.Performance index includes:Positioning error,positioning error′s standard deviation,positioning precision and positioning precision′s standard deviation.They are defined as follows:

Positioning errorwhere(^xnu,^ynu,^znu)is UCAV′s location in global coordinate system,and it is obtained by the proposed model.(xnu,ynu,znu)is the true location of UCAV in global coordinate system.

Positioning error′s standard deviation

whereΔR={Δr1,Δr2,…,ΔrN}.

2.2.1 Positioning result at certain location

Let(300,2 000,105)be the true location of UCAV.Eorty landmarks are used to estimate UCAV′s location.Assume that the landmarks′location error in global coordinate system satisfies Gaussian distribution.Its mean is 0,and its standard deviation is 5.Fig.8 shows the positioning results of one thousand experiments.Simulation results show that:(1)The proposed model can realize UCAV′s positioning,and the positioning error in each direction is similar to Gaussian distribution.(2)The error in height direction is larger than that in horizon direction,because this positioning calculation is based on the invariability of the distance between the landmarks and sensor.This distance is mainly decided by the flight altitude.Therefore,when the landmarks′location is wrong,it will directly lead to the error of distance between the landmarks and the sensor, which ultimately affects UCAV′s positioning result,especially for the height value.(3)In order to improve the positioning performance,other equipment can be used to decrease the height error,e.g.,the radio altimeter.

Fig.8 Positioning error of UCAV and its distribution in different directions

Fig.9 shows the positioning precision when the standard deviation of landmarks′location error is changed.Simulation results show that the value of positioning precision increases when the standard deviation of landmark′s location error increases.Conversely,if the standard deviation of landmarks′location error is small,that is to say,the sensor′s performance is better,the positioning performance of UCAV will be promoted. Eor example,when the standard deviation of landmarks′location error is lower than 5 m,the average of UCAV′s positioning precision is lower than 10 m,and the standard deviation of positioning precision is lower than 5 m.

Next,the positioning precision of UCAV is analyzed when the number of landmarks is adjus-ted.The true location of UCAV is also(300, 2 000,105).The landmarks′location error in sensor coordinate system satisfies Gaussian distribution.Its mean is 0,and its standard deviation is 10.Landmarks′number changes from 3 to 100.At each condition,one thousand experiments are implemented,and their average is used as the final result.Fig.10 shows the simulation result.The conclusions can be summarized as: (1)When the selected landmarks is less than 10, the value of positioning precision is big,and its descending trend is evident with the increase of the landmarks′number.(2)The positioning performance is better when the landmarks′number is from 20 to 40.(3)The change of landmarks′number has no obvious influence on the improvement of positioning performance when the landmarks′number is more than 40.Therefore,the landmarks′number from 20 to 40 is appropriate when the positioning precision and computational cost are considered.

Fig.9 Positioning precision with change of standard deviation of landmarks′location error

Fig.10 Positioning performance in different landmarks′number

2.2.2 Positioning results during flight

In the simulation,the landmarks′location error in sensor coordinate system satisfies Gaussian distribution.Its mean is 0,and its standard deviation is divided into 2,5,7,10,and 15. During UCAV′s positioning,forty landmarks are selected to calculate UCAV′s location.In the same condition,ten experiments are carried out and the average of ten experiments is used as the final result.Fig.11 shows the positioning results during the flight when the standard deviation of landmarks′location error is 5 m.Table 1 presents the detail positioning results at different landmarks′location error.Simulation results show that:(1)UCAV can achieve autonomous positioning and its positioning performance is enhanced with the improvement of the sensor′s precision.(2)The error in horizon direction is small,but the error in height direction is big. Therefore,some other approaches should be adopted to modify the error in height direction. (3)The sensor plays an important role in UCAV′s positioning,and the positioning performance can be enhanced with development of sensor technology.

Fig.11 Positioning results during flight(Set standard deviation of landmarks′location error to be 5 m)

Table 1 Positioning performance at different landmarks′location error

3 Conclusions

An autonomous positioning model is presented based on the representation of environment and the retrieval of landmarks in memory.Simulation results show that the proposed model is feasible.UCAV can realize autonomous positioning in memorized environment.The average of UCAV′s positioning precision is lower than 10 m when the standard deviation of landmarks′location error is lower than 5 m.Besides,considering UCAV′s positioning precision and the computational cost,the landmarks′number from 20 to 40 is appropriate for providing a good positioning reference.However,in the paper,we mainly discuss the realization and the positioning error of the proposed model,which is lack of the analysis of the computational cost and real-time performance.Therefore,this research need be further deepened.

Acknowledgement

This work was supported by the National Natural Science Eoundation of China(No.61273048).

[1] WU Dewei,TAI Nengjian,QI Junyi.A new research progress of UCAV intelligent navigation based on cognitive theory[J].Journal of Air Eorce Engineering University(Natural Science Edition),2011, 12(4):52-57.(in Chinese)

[2] LUO Yuan,EU Youli,CHENG Tiefeng.Simultaneous localization and mapping implementation based on the improved Rao-Blackwellized particle filter[J]. Control Theory&Applications,2015,32(2):267-272.(in Chinese)

[3] JARADAT M A,ABDEL-HAEEZ M E,SAADEDDIN K,et al.Intelligent fault detection and fusion for INS/GPS navigation system[C]//9th International Symposium on Mechatronics and Its Applications.Amman,Jordan:[s.n.],2013:1-5.

[4] XIONG Zhi,PAN Jialiang,LIN Aijun,et al.SINS/ GPS/CNS integrated navigation system federal PE algorithm in launch inertial coordinate system[J]. Journal of Nanjing University of Aeronautics&Astronautics,2015,47(3):319-323.(in Chinese)

[5] XU Xiandong,HONG Bingrong,GUAN Yi,et al. A comparison of several feature points methods used in mobile robot visual navigation[J].Journal of Huazhong University of Science and Technology(Natural Science Edition),2011,39(11):200-203.(in Chinese)

[6] BAO J T,SONG A G,TANG H R,et al.Navigation method for reconnaissance robot based on vision object tracking[J].Journal of Southeast University (Natural Science Edition),2012,42(3):399-405.

[7] WENG J,HWANG W S.Incremental hierarchical discriminant regression[J].IEEE Transactions on Neural Networks,2007,18(2):397-415.

[8] WANG Y H,CHEN S D,HAN Z P,et al.Modeling and algorithm application of weapon assignment system[J].Transactions of Nanjing University of Aeronautics and Astronautics,2014,31(6):693-700.

[9] ZHOU Yang,WU Dewei,TAI Nengjian,et al.A method of constructing place cells in UCAV terrain space environment perception[J].Journal of Air Eorce Engineering University(Natural Science Edition),2014,15(2):62-66.(in Chinese)

[10]WU Dewei,ZHOU Yang,DU Jia,et al.Method to select and position landmark in UCAV environment perception[J].Systems Engineering and Electronics, 2014,36(10):2048-2052.(in Chinese)

[11]ARLEO A,SMERALDI E,GERSTNER W.Cognitive navigation based on nonuniform gabor space sampling,unsupervised growing networks,and reinforcement learning[J].IEEE Transactions on Neural Networks,2004,15(3):639-652.

[12]GIOVANNANGELI C,GAUSSIER P,DESILLES G.Robust mapless outdoor vision-based navigation [C]//2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.Beijing,China: IEEE,2006:3293-3230.

[13]BAY H,TUVTELLARS T,GOOL L V.SURE: Speeded up robust feature[C]//9th European Conference on Computer Vision.Graz,Austria:University of Liubljanna,2006:404-417.

[14]WENG J,HWANG W S.Incremental hierarchical discriminant regression for online image classification [C]//6th International Conference on Document A-nalysis and Recognition.Seattle,USA:IEEE,2001: 476-480.

[15]CAEEERY J.Wireless location in CDMA cellular radio systems[M].Boston:Kluwer Academic Publisheers,1999.

Ms.Du Jia received M.S.degree from Xi′an Communications Institute in 2008.She is currently a Ph.D.candidate in Air Eorce Engineering University.Her research is focused on the intelligent and autonomous navigation of aircraft.

Prof.Wu Dewei received M.S.degree from Air Eorce Engineering University in 2000,and Ph.D.degree from Northwestern Polytechnical University in 2005.He is currently a full professor of Air Eorce Engineering University. His research is focused on the theory,technology and application of aircraft′s navigation.

Mr.Zhou Yang received M.S.degree from Air Eorce Engineering University in 2014.He is currently a Ph.D.candidate in Air Eorce Engineering University.His research is focused on the intelligent and autonomous navigation of aircraft.

(Executive Editor:Xu Chengting)

TP212.9;V249 Document code:A Article ID:1005-1120(2016)05-0584-11

(Received 17 May 2015;revised 21 October 2015;accepted 27 November 2015)