A Survey of Underwater Multi-Robot Systems

Ziye Zhou, Jincun Liu, and Junzhi Yu,

Abstract—As a cross-cutting field between ocean development and multi-robot system (MRS), the underwater multi-robot system (UMRS) has gained increasing attention from researchers and engineers in recent decades. In this paper, we present a comprehensive survey of cooperation issues, one of the key components of UMRS, from the perspective of the emergence of new functions. More specifically, we categorize the cooperation in terms of task-space, motion-space, measurement-space, as well as their combination. Further, we analyze the architecture of UMRS from three aspects, i.e., the performance of the individual underwater robot, the new functions of underwater robots, and the technical approaches of MRS. To conclude, we have discussed related promising directions for future research. This survey provides valuable insight into the reasonable utilization of UMRS to attain diverse underwater tasks in complex ocean application scenarios.

I. INTRODUCTION

THE developments of underwater robots have significantly influenced ocean exploration and exploitation in recent years. Underwater robots can withstand the high pressure in deep-sea environments, where it is dangerous for humans to stay and execute tasks such as exploration, sampling,reconnaissance, target search. Although researchers have made significant progresses by using underwater robot technology, a single robot is hardly competent to carry out these tasks because of the limitation of working efficiency in a vast ocean. Therefore, researchers are eager to apply the underwater multi-robot system (UMRS) to accomplish tasks with higher efficiency and lower cost.

From the beginning research to the state of the art of multirobot system (MRS), cooperation is one of the most commonly discussed topics. The research on the scheme and method of MRS began from around 1985 [1]–[3]. As an extension of MRS, the first batches of UMRS researches were investigated and applied to the planning, cooperative sampling, and cooperative navigation [4]–[6]. Since the beginning of the 21st century, with the development of MRS theory and the improvement of autonomous underwater vehicle (AUV) technology, UMRS has been studied and applied more and more widely. In [7], [8], the cooperative estimation approaches were utilized for tracking and following a shark tagged with an acoustic transmitter. Inspired by biology, researchers involved in the project termed collective cognitive robotics (CoCoRo) and the subsequent project called submarine cultures performed long-term robotic exploration of unconventional environmental niches(subCULTron). They developed several prototypes for bioinspired AUVs, such as artificial lily pad, artificial mussel,and artificial fish [9]. Moreover, the swarm intelligence of the heterogeneous groups of AUVs was explored for underwater monitoring, search [10], exploration [11], and data collection of hydro-geology or marine biology [9]. Multi-domain cooperation is a development trend of UMRS, which combines the advantages of the aspects of multi-domain robots in monitoring [12], navigation [13], [14], and interdisciplinary experiment [15]. Therefore, the fundamental purposes of these MRS applications are regarded as improving the capability or reducing the cost of the robot system through cooperation. In other words, cooperation is the commonality of different MRSs.

However, researchers seldom discussed the abstract concepts of cooperation, but dealt with the specific tasks of MRS. One of the reasons for the phenomenon is that cooperation is so discernible and easily distinguished from non-cooperation that researchers do not need to discuss the abstract definition of cooperation. Actually, exploring the concept of cooperation is meaningful for understanding,design, and development of MRS. Leitner discussed the cooperation according to the definition in the Oxford English Dictionary and the cooperation was divided into active cooperation and passive cooperation [16]. In [17], the cooperation of MRS is divided according to aware or unaware, then aware cooperation is subdivided into strong coordination, weak coordination, and non-coordination. In some studies, cooperation is always regarded as the opposite side to competition. However, most competition rules were designed for the examinations of the performance of MRS,such as the Robo Cup and Federation of International Robotsoccer Association (FIRA) Cup. In fact, we have not found any UMRS designed for internal competition. In [18], a competitive architecture for underwater robot is proposed and compared with a fixed reactive defending strategy by a 2vs2 competition. Shao and Wang [19] designed a platform of polo-like games for robotic fish competition. In order to test the performance of vision-based robotic fish, Zhenget al.[20]proposed a set of game rules for the international underwater robot competition (IURC). In brief, the purpose of the competition between two or more MRSs are to screen out more effective mechanisms for cooperation.

This paper presents a novel taxonomy and comprehensive survey to UMRS from the perspective of cooperation. The motivation is to provide researchers with a novel perspective on the classification of UMRS. Especially, compared with previous review papers [16], [17], [21]–[24], more emphasis is placed on the cooperation of MRS, which can be classified by task-space (TS), motion-space (MTS), measurement-space(MMS), and their combinations, regardless of whether the form of cooperation is passive or active, strong or weak.These concepts will be elaborated later in Section II.

The remainder of this paper is structured as follows. Section II presents a novel taxonomy of MRS and an architecture of UMRS. We categorize the cooperation of MRS into three spaces, TS, MTS, and MMS. The cooperations of UMRS in TS, MTS, and MMS are detailed in Section III, Section IV,and Section V, respectively. Further, there are cooperations located in both MTS and MMS, which are detailed in Section VI.Some critical issues and future developments of MRS are discussed in Section VII. Finally, some concluding remarks are given in Section VIII.

II. A TAXONOMY FOR UMRS

The communication delay and low rate make the structure of underwater communication quite different from other environments. Thus, the main difference between UMRS and MRS in other environments is the limitation of underwater communication. In practical applications, the organization and cooperation methods of UMRS and MRS are similar. As a part of MRS, UMRS shares the same classification as MRS in other environments from the perspective of cooperation. Thus,we surveyed several overviews for MRS and its sub-research areas. Farinelli divided the MRS into four levels: cooperation level, knowledge level, coordination level, and organization level [17]. At the cooperation level, cooperative systems are distinguished from non-cooperative systems. According to the existence of awareness of other robots, cooperative systems can be divided into the aware systems and unaware systems.Then, according to the coordination mechanisms, the aware systems are divided into strong coordinate, weak coordinate,and non-coordinate. Differently from Farinelli’s taxonomy,another categorization of MRS proposed in [21], MRS was divided into six application domains involving surveillance and search, foraging and flocking, formation and exploration,cooperative manipulation, team heterogeneity, and adversarial environment. Bayindir divided the swarm robots into five axes, which are modeling, behavior design, communication,analytical studies, and some problems [24]. Yanet al. [22]explored the properties of MRS and discussed several meaningful issues such as the coordination mechanism,resource conflict, communication, motion planning, and decision-making structure of MRS. Champion and Joordens[23] enumerated algorithms of underwater swarm. Huanget al. [25] divided the patrolling tasks into two categories,which are regular and adversarial patrolling. This paper refers to the categorizations in these articles, and puts forward a novel taxonomy for the cooperation of MRS. Considering a single robot is the basic component of MRS, before going into MRS classification, the motion and communication sensing characteristics of the underwater robot will be first introduced.

A. Underwater Robots

Compared with the robots operating on land and in the air,underwater robots are confronted with a more severe environment. Owing to the uncertainty of the hydrodynamic force, the dynamic model of a robot is approximate.Meanwhile, underwater communication and perception are limited because of the exponential decaying rate of electromagnetic signals in the water. Therefore, researchers are supposed to consider these particularities in practical applications.

1) Kinematics and Dynamics Models:To facilitate the description of the AUV motion, as illustrated in Fig. 1, three types of frames of reference are used. The earth-fixed frame of reference {U} and the body-fixed frame of reference {B} are utilized to describe the motion of AUV. The Serret-Frenet frame of reference {F} are used to simplify the control of AUV[26]. In terms of the propulsion mode, AUV can be divided into propeller propulsion AUV and bionic propulsion AUV.Bionic propulsion modes include the fish-like propulsion,jellyfish propulsion [27], [28], and bio-inspired jet propulsion[29]. Considering the propeller propulsion AUVs are more widely implemented in the studies of UMRS, the 6 degrees of freedom (DOF) kinematics and dynamics model for thei-th AUV in a group consisting ofnrobots can be expressed below.

Fig. 1. Coordinate frames of the AUV.

where ζi=[xi,yi,zi,ϕi,θi,ψi]T,i∈{1,2,...,n} is the position and orientation vector ofi-th AUV in the earth fixed frame of reference. νi=[ui,vi,wi,pi,qi,ri]Tis the velocity vector ofi-th AUV in the body-fixed frame of reference.Fi=[Xi,Yi,Zi,Ki,Mi,Ni]Trepresents the force and moment vector ofi-th AUV in the earth fixed frame of reference.Ji(ζi) denotes the transformation matrix from the earth fixed frame of reference to the body-fixed frame of reference.Ci(νi) andDi(νi)represent the hydrodynamic damping and lift matrix respectively. The vector of gravitational forces and moments is defined asgi(ζi). A more specific model of AUV can be found in [30].

TABLE I COMPARISON OF UNDERWATER COMMUNICATION TECHNOLOGIES

2) Underwater Communication:Underwater communication is quite different from communication in other environments, which limits the realization of MRS function.There are generally four types of underwater communication methods: acoustics, radio frequency (RF), optics, and magnetics [31], [32]. Acoustic methods are the most widely implemented in underwater communication, which can maintain communication from dozens/hundreds meters[33]–[35] to several/dozens kilometers [36]–[38]. However,due to the underwater background noise and the attenuation of the acoustics signal, the acoustic communication rate usually does not exceed several tens of kbps. In addition, acoustic communication has a relatively high delay than RF because of the low speed of sound propagation. The radio frequency method has a faster communication speed and transmits smoothly through the air/water interface, which is convenient for the application of MRS in shallow water [31]. Since the electromagnetic waves are absorbed by water, the RF waves can only propagate a few meters with no more than 10 Mbps[39], [40]. Optical communications can achieve a high communication rate in tens of meters, but it requires a high transparency of water. The suspensions and living organisms in seawater will significantly reduce the communication range and quality, thus researchers employ blue or green ray modems because of theirs low absorption rates in water. The data rate of underwater optical communication is usually between 1–10 Mbps, which is positively correlated with power consumption and negatively correlated with range, and the farthest is generally no more than 100 m [41]–[44].Magnetic induction communication is a novel kind and has been examined in laboratory settings [45], [46], which is more stable and predictable due to the almost same magnetic permeability of the water and the air [32]. The comparison of these communication methods is tabulated in Table I.

B. A Taxonomy for the Cooperation of MRS

As mentioned previously, several surveys classified MRS according to different standards such as the application domains [21], algorithms [23], intelligent levels [17], order of dynamic model [47], and some interested problems [22], [24].In this paper, cooperation is assumed as the basic property of MRS, and we try to investigate the character of cooperation.However, the definition of cooperation in MRS field is somewhat vague [16]. Contrary to language fuzziness,researchers can clearly distinguish cooperation from noncooperation, exactly as the categorization in [21]. Before defining cooperation, a question should be firstly interpreted:what makesnrobots combine as a system, not a set? We can consider the question from the advantages of MRS. First,MRS can accomplish tasks thatndetached robots are inadequate. Second, MRS can accomplish the same task with higher efficiency or quality than the same number of detached robots. To summarize, the opinion may be an agreement in the field of MRS: MRS does not satisfy the superposition principle. Through cooperation, MRS has realized more different functions than the simple addition of multiple robots.Thus, the emergence of new functions may be the key to distinguish cooperation from non-cooperation. Based on this,cooperation is defined here.

Definition:Considering a setSnis composed ofnrobots,the functions of thei-th (i∈1,2,...,n) robot are defined as a setfi, and the functions ofSnare denoted as a setFn. Then,Cnrepresents the difference betweenFnand, which is called “emergent functions”. IfCnis not empty and the performance of MRS is enhanced, the MRS is considered as cooperative. IfCnis not empty and the performance of MRS is not enhanced, the MRS is a non-cooperative system. IfCnis empty,Snis not considered as a system.

For instance, compared withnuncooperative robots, the emergent function of a cooperative positioning task is the robust, highly precise, and low-cost positioning of robots.

With the definition of cooperation, cooperation and the corresponding MRS can be classified. Since the emergent functions are the results of cooperation, the classification of the emergent functions can be regarded as the classification of cooperation. In particular, the emergent functions of MRS are associated with the functions of a single robot, which provides the basis of the classification. As is well known, the functions of a single robot can be divided into three categories:completing high-level tasks, moving/executing, and measuring/sensing. For thei-th robot, the functions for completing high-level tasks are denoted asfT−i, such as grasping and pushing. The functions for moving and executing are represented byfMT−i, such as moving. The functions for measuring arefMM−i, such as the measurement of a scalar field. The functions of robotican be expressed as follows:

1) Motion-Space:All robots discussed in this paper are entities with the moving ability in the three-dimensional (3D)space. The positions, attitudes, velocities, forces, and other relevant parameters of all robots in MRS constitute the motion-space. The results of cooperation in motion-space are considered as the functions which are emerged by the motion or actuation ofnrobots, and usually involve the coordination of position and velocity. The methods to realize the emergent functionsare to deal with the parameters of multiple robots in motion-space. For instance, the consensus protocols of formation control consider the differences of position and velocity, while the path planning algorithms minimize the total travel consumption.

2) Measurement-Space:In general, robots may load multiple sensors to form a mobile sensor network to measure the states of themselves and their targets. The measurements,noises, and other relevant measurement parameters of all robots’ sensors constitute the measurement-space. The result of cooperation in measurement-space is considered as the emergent functions produced by the measurements and related parameters ofnrobots. By using the methods and strategies of state estimation, MRS obtains a better measurement such as accuracy and interference immunity after the cooperation.

Since the measurement model generally contains the motion parameters of the robot, the cooperation in measurementspace always associates with the motion-space. However,cooperative navigation is still considered as the cooperation in measurement-space because the measurement results do not affect the motion control. While, in cooperative sampling,monitoring, and other tasks, there are many interactive effects between motion and measurement. Therefore, they are the cooperations in both motion-space and measurement-space.

3) Task-Space:Similarly to the strategy-space of game theory, the task-spaceis defined as the composition of all the performable tasks of all robots in MRS. It should be noted that the task-space is at a higher level than motion-space and measurement-space. Indeed, the task-space is abstract, so it must be mapped to the other two spaces to realize tasks. The mapping indicates that the cooperation is in task-space and motion-space, or task-space and measurement-space. More precisely, all cooperation is related to the task-space. For convenience, when we consider the cooperation in task-space and motion-space or measurement-space, the task-space can be omitted.

Now, the cooperation of MRS is divided into three spaces.Cooperation can take place in one space or span in multiple spaces. According to the above classification, Table II is tabulated for reference. Furthermore, based on the taxonomy,a complete framework for MRS is illustrated in Fig. 2.Considering the different communication and sensing structures, the structures of MRS can be classified into centralized, distributed, and hybrid ones. If a single robot processes the information from other robots and makes decisions, the structure of MRS is centralized. If each robot processes the information and makes decisions by itself, the structure of MRS is distributed. The hybrid structure is a combination of a distributed system and a centralized system.In order to describe the information flow and the structure of MRS, graph theory is widely used, especially in formation control and cooperative navigation.

Fig. 2. The taxonomy based architecture of MRS.

C. Graph Theory

Graph theory is commonly used to describe the communication network of robot. The topological network composed ofnrobots is usually described by graphG(V,E,A),whereV={1,2,...,n} represents the vertex set of graphs corresponding to the robot set. The edge setE⊆V×Vdenotes the information connection relationship between robots, andA=[aij] indicates the connection matrix. When robotican receive the information from robotj,aij>0,otherwiseai j=0. Generally, the communications between robots are not always bidirectional. Thus, the graphG(V,E,A)is a directed graph. A complete directed graph is a directed graph in which each pair of robots are connected by a bidirectional edge, so there is a directed path from any robot to any other robot in the network. The in-degree of robotiis denoted asthe matrixD=diag{deg1,...,degn}is called degree matrix of graphG(V,E,A). Then, the Laplacian matrix ofG(V,E,A) is defined asL=D−A.Laplacian matrixLhas at least one single zero eigenvalue,and the corresponding eigenvector can be denoted asl=[1,1,...,1].

III. COOPERATION IN TASK-SPACE

Task is an abstract concept, which is more difficult to quantify than specific parameters such as velocity and position. Researchers need to define each task to construct the task-space. Then, MRS assigns tasks through negotiations or strategies, and each robot will complete the corresponding tasks. Therefore, the cooperation in the task-space is primarily considered as the task allocation. Considering the complex task can be divided into simple subtasks, task-space is hierarchical to some extent. For instance, a robot pushes a box, which is conspicuously a task,TSpush. However, it may be a sub-task of the maze taskTSmaze, the robot passes through a maze stuck with obstacles. The hierarchy of taskspace will be reflected in the task allocation strategy. The methods of task allocation in MRS field are mainly divided into two categories: behavior-based method and market-based method.

A. Behavior-Based Methods

The behavior-based method decomposes MRS into several subsystems with task implementation behaviors, to be competent for the tasks that need to be completed in the underwater working environment [48]. In order to accomplish complex tasks, the role-behavior-action architectures orsimilar architectures named by other terms are usually adopted[49]–[53]. Since behavior-based methods only respond to predefined states, researchers are supposed to consider all kinds of scenes faced by robots comprehensively and define corresponding actions.

TABLE II THE CLASSIFICATION OF COOPERATION AND CORRESPONDING METHODS

Take the role-behavior-action framework as an example. At the beginning of cooperation, each robot is assigned a role with different behaviors. Such behaviors are usually divided into two categories: reactive behaviors and deliberative behaviors. Among them, reactive behavior shows a direct response to environmental stimuli. That is, the current action is determined to accord to the region or state where the robot is. For instance, in [50], different actions are selected according to different regions where the robotic fish is located. The deliberative behaviors involve some kind of cooperation intention. In the water polo case of [49], robots are assigned the role and utilize a fuzzy reinforcement learning approach to conduct the deliberated mechanism to generate sub-goals and the corresponding actions. Each action means a specific gait sequence of the robotic fish. The application of role-behavior-action architecture can make multi-robotic fish perform some complex tasks.

The behavior-based methods are also applied to formation control. Similarly to the structure of role-behavior-action, a four-layer structure was proposed in [54] with the cooperative layer, monitoring and planning layers, behavior layer, and execution layer. In the cooperative layer, leaders and followers exchange information and plan tasks through acoustic communication in an uncertain environment. In the monitoring layer and planning layer, tasks are decomposed into behavior sequences. In the behavior layer, the leader fuses the obstacle avoidance and target seeking behaviors by fuzzy rules, and the follower fuses obstacle avoidance and formation keeping behaviors. The fused behaviors will be executed via fuzzy reasoning. The behavior-based methods need less computing and communication resources and achieve a good real-time performance.

B. Market-Based Methods

Compared with the behavior-based methods, the marketbased methods can provide higher quality solutions with extensive planning. In the market economy, self-interested individuals and groups maximize their profits by trading goods and services. The market exchanges lead to the shared development of the entire system. Inspired by market economies, each robot has a specific cost function for various resources (e.g., time, network bandwidth, computing power,sensors, etc.) to quantify the robot’s preference for the use of such resources, calculating bids for various tasks. For a centralized system, there is an auctioneer robot broadcasting the task to the auctioneer. Each robot bids to the auctioneer according to the cost function. Then the auctioneer minimizes a global objective function and assigns tasks to robots with the lowest execution cost. In the box-pushing task in [52], robotic fishes cannot simultaneously complete two or more tasks.Therefore, only the robotic fishes that have not been assigned tasks can bid on the newly announced tasks. Considering inaccurate bids, redistribution/re-auctioning tasks can usually adjust unreasonable schemes to improve the quality of solutions. For the distributed system, there is no central auctioneer. In [55], when an AUV intends to execute a task,the virtual market will be established, and the AUV becomes the auctioneer. This kind of distributed method is more robust for the underwater environment. Compared with the behaviorbased methods, the market-based methods can provide higher quality solutions. Therefore, market-based methods are the most commonly used methods in task allocation.

IV. COOPERATION IN MOTION-SPACE

The cooperation in motion-space is the interaction and coordination between robot motions. Robots in MRS do not only consider themselves but also the motion of other robots during the cooperation in motion-space, which requires corresponding mathematical tools to tackle these motion parameters.

A. Formation Control

As the typical cooperation in motion-space, formation control is an active research field in MRS research and is one of the most basic applications of MRS. The purpose of formation control is to maintain a formation in a certain 3D space by various methods under a specific environment. In Cao’s classification, formation control is divided into formation producing and formation tracking according to the presence or absence of group reference [56]. Formation producing is the formation control without a group reference,robots self-organize to generate some desired through several rules, such as the flocking algorithm [57]–[60]. Formation tracking, that is, formation control with a group reference, the robot follows the constraint of group reference and has a fixed relative positional relationship to form a pre-defined formation [61]–[64]. The formation control can also be divided into two types according to whether or not there is a leader: leaderless and leader-follower.

Robots in formation control require the position and velocity information of their neighbors, adopt some control strategy to make all robots have the consistency of position and velocity. Considering the collision avoidance of robots, a certain method is used to maintain the desired formation characteristics or shapes. Generally speaking, the consensus algorithm is employed to ensure the convergence of the robot’s position and velocity, and the potential function or other methods are used to avoid the collision between robots. The prevailing approach for formation control is the consensus theory. The following parts will first introduce the basic mathematical tools in the formation control and then explain some practical challenges in the formation control.

1) Consensus Theory and Potential Function:The MRS can mathematically be modeled as a second-order integral model:

wherepirepresents the coordinate ofi- th robot,qiis the velocity ofi- th robot,uidenotes the control input fori-th robot. A typical consensus protocol is formulated as

whereai jrepresents the connection relation of communication between thei- th robot and thej-th robot, andA=[aij] is called the connection matrix. γ0and γ1are the consensus weights of position and velocity, respectively.

Consensus protocol aims at an agreement of the motion states for the robots connected by a directed graphG. There are some variants of the consensus protocols [65]. However,as mentioned priorly, it is necessary to keep a certain distance and orientation between robots. A simple and effective method is to use the artificial potential function to calculate the attraction force or repulsion force between underwater robots to solve the problem. Letand define a non-negative function

whenrij=d, a minimum value ofUi(pi,pj) is obtained, and whenrij→0 orrij>>d,Ui(pi,pj)→+∞. Here,dis the desired distance between the robots. The force between thei-th robot andj-th robot is

which means, the potential force maintains a fixed desired spacingdof the robots in the group. The direction ofFi jdepends on the relationship betweenrijandd. Whenrijd, the forceFi jbecomes attractive and increases byrijincreasing. Then, the net force of thei-th robot is

whereNis the neighbor set of thei-th robot. The control inputuiis given by

In fact, not all formation control problems are solved as a consensus problem. Some researchers expanded the controller for a single robot into a distributed controller for MRS. For example, Hou and Cheah proposed an adaptive PD-based controller for shape control and ushered in a multi-layer region control concept to limit the oscillatory movements of AUVs [66]. Similarly, Weiet al.[67] proposed a distributed Lyapunov-based model predictive controller (DLMPC) to track a 2D virtual-structure of AUVs under the disturbances of unknown ocean current and combine the artificial potential fields-based cost function to achieve the inter-AUV collision avoidance. Yanget al.[68] decoupled the shape dynamic model and employed the Jacobi shape theory to maintain the formation shape and track the desired trajectory of the formation center by using a robust controller. The decomposition of the shape dynamics simplified the design process of the controller. Compared with the consensus control, these methods are more simple, robust, and intuitive for the formation control. McColgan and McGookin designed a self-organizing coordination algorithm for the schooling of robotic fish by calculating the distance to neighbors and determining the behavioral zone [69]. The self-organizing formation is more flexible than a fixed shape formation, and different rules result in different structures of the shape.

Besides the consistency and collision avoidance of MRS,more complex factors should be considered in the practical formation control, such as the time delay of communication,dynamic uncertainty, disturbance of ocean current, convergence performance, resilient control, and other factors.

2) Time Delay:The underwater communication of robot is limited by two parts. The one is high time delay and low transmission rate of underwater acoustic communication. The other is the data packet dropout of underwater communication. As a rule, the data packet dropout is regarded as a special time delay problem because the transmission protocol guarantees retransmission after the dropout. Therefore,researchers mainly study the effect of time delay on the MRS formation consensus.

Time delay can be divided into communication delay and input delay [56]. Among them, communication delay refers to the time required for the information sent by robotjto be transmitted to roboti. Input delay refers to the total time required for processing, decision-making, and execution of robotiafter receiving the information. Considering delay, the MRS consensus protocol (5) becomes

Using this protocol, Yanet al.[70] considered the input delay and added a nonlinear additional uncertainty to the dynamic model. In fact, for the convenience of formation control, most UMRSs merely consider communication delays.Some are time-varying [70], [71], while others are fixed [68],[72]. One of the approaches to obtain the communication delay is to time-stamp each packet when it was sent [72].Thus, the delay information can be obtained from a synchronized system. Based on the optimization theory like the quadratic programming, Suryendu and Subudhi [72]designed an adaptive controller with the state delay of the leader AUV. The optimization based method has effectively compensated the time-varying delay in the open water experiment [72]. In [70], [71], the Lyapunov-Krasovskii functional theory and algebraic graph theory were utilized to construct and prove the sufficient consensus conditions for the time-delay scenario of UMRS.

3) Disturbance and Uncertainty:The impact of environmental factors such as ocean current will bring unknown disturbance to AUVs, which will reduce the stability of formation. The uncertainties of the model parameters are also considered as the inner disturbances. Typically, these disturbances are modeled as a nonlinear additional function,which is random or has some kinds of dynamic models with an item of the control input.

where τ(pi(t),qi(t),t) is the nonlinear disturbance.qc(t)denotes the ocean current. It is worth noting that the termqc(t)is not optional unless the ocean current is known [73]. In most cases, the ocean current is unknown and is treated as an external disturbance. To solve the problems of formation control under disturbance, uncertainty, and noise, a multitude of methods have been proposed. The observer methods are commonly utilized. In [67], an extended state observer (ESO)was applied to estimate the unknown current disturbances,which is more constructive and computationally tractable. In[74], a fixed-time disturbance observer was designed to estimate the compound disturbance, which can achieve exact estimation within a finite time upper bounded by a positive constant independent of initial estimation error. A fuzzy logic system was employed in [75] to approximate the uncertain dynamics under the unknown time-varying ocean disturbances. Saboori and Khorasani [76] proposed a state feedback method to solve theH∞and the weightedH∞consensus problems for disturbance signals with the bounded root-mean-square (RMS) disturbance. Cuiet al.[77] used distributed adaptive technique and constructed a nonsingular fast terminal sliding mode controller to estimate external disturbances and internal uncertainties.

4) Attacks:Attacks, violent current impacts, unexpected collisions, and other accidents will affect the formation performance. They may lead to the fault of the formation system, which is different from the slight disturbances discussed in the previous section. Large disturbances may lead to node failures or link failures of the formation system. The error caused by these failures may spread to the whole formation system. Thus, the formation system is supposed to utilize resilient control approaches under the large disturbance condition or attack, and should be reconfigurable when the node robot faults by disturbance.

The node failures contain disturbances or attacks on sensors and actuators. The link failures contain some attacks such as false data injection (FDI) and denial of service (DoS) attack.These failures are modeled and solved differently. For node failures, the attack on sensors of robotiis modeled in [65] as

wherexi(t) is the state of the roboti, which represents the position for a single-integrator kinematic model or represents the position and velocity for a double-integrator dynamic model.is the attack signal injected into the sensors ofi-th robot. The attack on actuators of robotiis modeled as

whereui(t) is the control input,is the attack signal injected into the actuators ofi-th robot.denotes the distorted control input.

Similarly to the equation (13), the FDI attack model is

where ψq,ij(t) represents the communicating information from robotjto roboti, δa,ij(t) is the attack signals injected from robotjto roboti.

The DoS attacks deactivate the data receiving or sending of a robot during a period of time. The DoS attacks are modeled in [78]:

Notice that, when the attack happens to roboti, αi(tk)=1;otherwise, αi(tk)=0 . α is the attack rate.d(tk) is the consecutive attack duration.

It should also be noted that the mentioned attack model is formally similar to the disturbance model proposed in above part, and the approaches also have similarities. They both need to detect and identify the abnormalities and to compensate them by some methods, such as [65], [79].Mustafa and Modares [65] proposed an expected state predictor to predict the normal behavior of each robot, then designed a distributed adaptive compensator to compensate for the discrepancy between the actual state and its predicted normal one. The compensator can ensure the consensus of the formation under a time-varying attack [65].

The differences between attacks and disturbances are that the disturbances are usually unavoidable while attacks can be defended beforehand. In addition, the influences of attacks are more harmful than disturbances. These differences imply that more flexible approaches should be developed, such as reconfigure control approaches or formation switching methods. The control reconfiguration methods detect and isolate the faults and reconfigure the controllers to accommodate the faults in the formation of MRS. In [80],[81], the faults detection and isolation used local information because the AUV merely shared the information with its neighbors. Gallehdariet al.[81] reconfigured the control gains to enforce the output of the faulted AUV imitated that of the healthy AUV. For the leaderless and leader-follower architectures in [80], the optimal control law was obtained by solving local Hamilton-Jacobi-Bellman equations when the actuators of AUV faulted. In [82], the hybrid event-triggering strategies (HETSs) were proposed for the DoS attacks. For the finite deactivated time, the HETSs adjust the triggering frequency during DoS attacks to maintain the formation. For the infinite deactivated time, the DoS attacks are considered the loss of the robot. Then, the MRS may choose the topology switching method [71].

5) Convergence Performance:The finite-time convergence and fixed-time convergence are the extended studies of the consensus convergence of multi-agents. Finite-time problem denotes that the system converges in a finite time, and the fixed-time problem is that the system can converge in a certain time. The finite-time control and fixed-time control provide faster response speed, higher tracking accuracy, and better anti-interference capability. For more detail on the theorems on finite time and fixed time, please refer to [83],[84].

In the leaderless and leader-follower multi-AUV formation area, Li and Wang [85] proposed a distributed collision avoidance controller by the feedback of the distributed finitetime observer, which was proposed for the followers to estimate the leader’s velocity. Cuiet al.[86] designed a formation-keeping method combined on nonsingular fast terminal sliding mode and neural network approximation, and proved that the tracking errors could converge with strong robustness along the sliding surface in finite time by using the finite-time stability analysis. Considering the communication limitation, a fixed-time control approach with an eventtriggered mechanism was proposed in [87] to reduce the energy consumption and communication burden, and the fixed-time approach can also obtain a higher accuracy performance and a faster convergence speed.

Besides the finite-time and fixed-time convergence, the convergence rate is also a research hotspot. Xinget al.[88]evaluated the effect of the parameter of the consensus protocol and the connectivity of topology on the convergence rate.They also proved that adding undirected or directed edges in the topology formed with followers and leaders or increasing the weights on the communication links will accelerate the convergence rate [88].

B. Path Planning

Path planning for UMRS is regarded as a kind of cooperation in motion-space. In general, path planning algorithms optimize the length or energy consumption for AUVs. Differently from formation control, the path planning for UMRS rarely requires the consensus of robot velocity or position during the task. In fact, path planning for UMRS is mainly applied for the rendezvous of multiple start points and single target points [89], [90], or the travel of multiple target points [91]–[93].

For rendezvous tasks, the path planning problem adds a constraint that all the AUVs reach the rendezvous point as simultaneously as possible. A common rendezvous process for the beforehand determined rendezvous points is to estimate the proper rendezvous time so that all the AUVs can reach the rendezvous point at the same time, then consider the influence of known or unknown ocean currents and uncertain factors to plan paths distributed. Thus, the robot can reach the convergence point as close as possible. In the case of the online rendezvous, the rendezvous point is searched according to some factors such as the relative distance and ocean current. In [89], a dynamic programming method was adopted to solve the convergence problem of time-varying current fields. Yaoet al.[94] divided the rendezvous task into two steps. The first step was to find the maximal voyage time by the single-AUV path planning in an ocean current and obstacle environment. The second step was to obtain the approximately equal voyage time by the multi-AUV path replanning. The improved biased min-consensus (BMC)method is applied in the two steps of path planning [94]. For the non-specified rendezvous point case, Zenget al.[90]proposed a distributed shell-space decomposition scheme combined with a spline-based quantum particle swarm optimization path planner for multiple autonomous marine vehicles to search for a rendezvous point and ensure exact simultaneous arrival at the rendezvous point.

The planning for traveling of multiple target points is usually divided into two steps: voyage assignment and path planning. Voyage assignment is based on one or several certain indicators such as the total energy consumption of the system [91], the minimum total distance, the minimum average energy consumption or average distance, the minimum mean square deviation of energy consumption and distance. Then, endurance and travel ability are considered. In such a situation, the traveling task is usually converted into an optimization problem to allocate target points for every robot.For example, in a dynamic target environment, considering the shortest system journey and the maximum safe journey of the robot, the self-organizing map network method is adopted to allocate the target points [92], [93]. Benefiting from the efficiency of differential evolution method for continuous optimization and the good performance of ant colony optimization on both set-based and order-based optimization problems, Yuet al.[91] used a combined method composed of the ant colony optimization algorithm and the differential evolution for surface point location and voyage planning. In[95], the motion planning problem with energy, navigation,and dynamics constraints was solved by the direct multiple shooting method. The energy consumption is modeled as

The path or trajectory planning of multi-objective usually uses some simple single robot path planning algorithms to yield the optimal path. The most popular methods are A*algorithm and its variants [91]. In addition, considering the problem of multiple AUVs with multiple targets in a constant current environment, a velocity synthesis approach is proposed in [93] for the 3D workspace to offset the ocean current’s effect. Chowet al.[96] considered the AUV dynamics and the dynamic ocean currents, and modified the Dubins path to a non-constant radius of curvature to minimize the total travel time.

C. Cooperative Manipulation and Transportation

Underwater cooperative manipulation refers to the movement, rotation, disassembly, and installation of a certain entity by multiple underwater robots. The state of AUV will affect task allocation. Thus, cooperative manipulation is the cooperation between task-space and motion-space. The application of underwater cooperative manipulation includes:box pushing [51], [52], water polo game [18], [49], [50], [53],underwater recovery of a blackbox [97], target grasping and transportation [98], [99], etc.

According to the classification of [100], a subtask of underwater operation and transportation is usually divided into four successive phases: navigation, grasping,transportation, and deployment. For some specific AUVs and tasks, the four phases may vary. For some tasks of AUVs without actuators, such as box pushing tasks or water polo games of robotic fishes, the grasping phase becomes finding a suitable push point after the navigation phase [52]. Then, they provide the thrust according to the cooperative algorithm to realize transportation. During this process, the robot’s motion is not only affected by the motion state of other robots but also takes into account the current state of the box.

Considering that the constraint of communication, the ability of information exchange for underwater cooperative operation is limited. AUVs exchange key information such as the velocity of toolframe to save communication resources.According to the requirements of tasks, on the premise of ensuring the safety of AUVs and objectives, physical constraints are considered and the priority list of control targets is determined furthermore [100].

V. COOPERATION IN MEASUREMENT-SPACE

As mentioned above, the measurement-space is composed of sensor measurement values. Inertial navigation system(INS), compass, and Doppler velocity log (DVL) are usually used to measure the motion state of AUV. Cooperative positioning and cooperative navigation are cooperation based on these measurements. Similarly, conductivity-temperaturedepth sensors or other scalar field sensors measure some scalars in the underwater environment, and the cooperation based on these measurements is cooperative sampling and exploration. Sonar can scan underwater terrain and identify targets according to echoes, and underwater cameras can also realize these functions in areas with diving and clear water quality. Therefore, the cooperative underwater terrain construction, cooperative target searching, and monitoring are cooperations based on these measurements.

Cooperative navigation and positioning are considered as cooperation in measurement-space because the measurement results do not affect the motion control of robots. This section introduces this part. The other tasks, such as cooperative sampling and cooperative monitoring, are considered the cooperation in both the movement-space and the measurement-space due to the interaction between motion parameters and measurement parameters, which will be illustrated in Section VI.

It should be noted that in addition to the move and force execution, robots need to position, navigate, and detect during the cooperative manipulation task. However, the current researches in cooperative manipulation areas are mostly focused on force and position coordination without cooperation in sensing and positioning. For the refined cooperative manipulation, cooperative detection and perception should be incorporated. In this context, cooperative manipulation involves cooperation in both movement-space and measurement-space.

Considering the limitation of cost, it is unrealistic to configure high-precision INS and DVL for all robots in UMRS. In order to reduce the cost, only one or a few AUVs are equipped with high-precision sensors. A typical cooperative navigation framework is: the high-precision sensors (such as high-precision INS, DVL, and GPS)equipped parent AUV communicates with its child neighbors which only a few sensors are equipped, broadcasts its position to improve the positioning accuracy of child [101]–[104]. In addition, this information sharing can restore the navigation function of some individuals in UMRS to a certain extent when their navigation sensors fail. In short, cooperative navigation can improve the navigation accuracy and robustness of MRS.

Cooperative navigation and positioning are problems of data fusion and state estimation. Each AUV communicates and exchanges data with others, processes the state information according to the sensor measurement, and generates an optimal state estimation according to a certain standard by combining the local measurement state with the received data.

Fusion architecture can be centralized, distributed, or hierarchical. Theoretically, the centralized architectures obtain a more accurate estimation of AUV’s state. However,distributed architectures are widely implemented in cooperative navigation due to the constraint of underwater communication. The state estimation problem can be formulated as follows:

wherefandgrepresent the equation of motion and observation respectively,ukdenotes the control input,wkis the noise ofuk,nkrepresents the noise of observation. Since the depth information can be obtained by depth meter, the 3D navigation can generally be simplified to the two-dimensional problem, thus reducing the complexity of collaborative navigation and positioning.

According to the linear property,fandgare divided into linear or nonlinear (NL) systems. The noiseswkandnkcan further be divided into Gaussian or non-Gaussian (NG) noises according to whether they obey the Gaussian distribution.Similarly to the methods employed in integrated navigation and multi-sensor information fusion, collaborative navigation is also divided into filtering-based methods and optimizationbased methods.

A. Filter-Based Methods

Filter-based methods of cooperative navigation and positioning mainly include the Kalman filter, particle filter,and their variants. The standard Kalman filter and its variants are based on a maximum posterior estimation, they are the equations of linear minimum mean square deviation and unbiased estimation with recursive operation through two processes of state prediction and state update. These methods can solve most of the state estimation problems from linear-Gaussian (LG) to nonlinear non-Gaussian (NLNG) systems.

The system in [101] is an nonlinear and Gaussian (NLG)system, its motion equation is nonlinear. For the highly nonlinear state transition and measurement model cases, the covariance is propagated through linearization of the nonlinear model, which is a disadvantage for the extended Kalman filter (EKF). Thus, an unscented Kalman filter (UKF)is utilized for the prediction step, while the measurement equation is linear, and EKF is employed for the measurement update step. In [102], the particle filter is used to process similar systems, and satisfactory results have been achieved in the post-processing of sea trial data. In [101], considering the influence of unknown water flow on motion, the water flow velocity is added to the state equation. In particular, error propagation needs to be avoided in the distributed estimation.It is verified that AUVs communicate with all neighbors,which will lead to rapid error divergence [101].

B. Optimization-Based Methods

The idea of state estimation based on filtering is to estimate the next moment with the value of the previous moment,while optimization is to regard all states as variables, the motion equation and the observation equation as constraints between variables, and construct an error function. At present,this part of the research is mainly based on the cooperative navigation of the factor graph, which is different from the recursive form of filtering method. The optimization method does not follow the Markov assumption and is based on a set estimation of measured values at different times. The optimization method considers the constraints in the whole trajectory, and the cumulative error increases slowly.

For NLG systems with nonlinear state equations and measurement equations, Fanet al.[103] transformed the global function estimation problem into a local function and product estimation problem. An improved factor graph and a sum-product-based cooperative positioning algorithm have been established. The effectiveness of the algorithm is verified by surface experiments. Considering the influence of the complex working environment and sensor performance, a cooperative positioning system has the problem of inaccurate statistical characteristics of system noise [103]. In addition,due to the influence of complex underwater acoustic communication environment, underwater acoustic ranging information is often interfered with by abnormal measurement noise, and its distribution often shows the characteristics of heavy tail distribution. A cooperative positioning algorithm based on factor graph and maximum correlation is proposed by Fan, and the cost function based on maximum correlation entropy can effectively reduce the influence of outliers on system accuracy [104]. Wuet al.[105] acquired the ocean current by acoustic Doppler current profiler (ADCP) and the position of the target by the equipped underwater acoustic beacon, and a cooperative current estimation and a target estimation model were designed, which is described by the factor graph.

C. Other Methods

The above-mentioned methods demand a complete communication system for each AUV to share navigation information. However, considering the cost and division of roles, some low-cost methods are proposed. Fischellet al.[106] developed a novel acoustic system for underwater swarming. AUVs employ a single-transducer to sense the Doppler shift and the attenuation of sound. The Doppler shift of sound is utilized to control vehicle heading, and the difference in attenuation can estimate the range between followers and the leader.

VI. COOPERATION IN BOTH MOTION-SPACE AND MEASUREMENT-SPACE

Considering the limited endurance of a single AUV in practical applications, researchers prefer using MRS for high performance and efficiency in the scalar field sampling [107],[108], target search [109]–[111], ship hull inspection [112],exploration, monitoring, surveillance, and hunting tasks [113],[114]. These tasks are the comprehensive applications of MRS. Thus, these cooperations are more complex and occur in the motion-space and measurement-space. Cooperation should not be classified according to the task category,because even for the same tasks, researchers may analyze them from different perspectives and solve problems in different approaches. For instance, some collaborative sampling tasks can improve the accuracy by the state estimation methods [107], while others can improve the robustness of the system by designing the sampling spatial structure [108]. Similarly to the classification above, the methods to solve the cooperation in the motion-space and measurement-space can be divided into three kinds: strategy,motion, and measurement. Among them, each kind is somewhat similar to the methods described in Sections III–V.

A. Strategy Kind

Since the complexity of the cooperation in both motionspace and measurement-space, it is necessary to design a set of strategies to accomplish the task and solve the possible accident, especially in underwater environments.

For the survey task, Yoon and Qiao [115] proposed a synchronization-based survey (SBS) scheme with a set of rendezvous strategies for multiple AUVs to consider communication limitations and data exchange. A task reallocation strategy was further designed to tolerate the condition of limited communication and AUV fails. Finally,three variants of SBS were compared, which were different in the efficiency of data aggregation and control signal dissemination. Researchers have different requirements for different search tasks. Linet al.[110] established a hierarchical architecture for underwater cooperative search with the distributed control policy. The architecture divides AUVs into data collecting AUV and data storing AUV and assures the connection of all AUVs and the Unmanned surface vehicles (USVs) which are the control decision center. By using software-defined networking, the architecture realized the network synchronization, navigation, data transformation,and cooperative control for multi-AUV [110]. Considering the unstable and unreliable communication, Yordanovaet al.[116] proposed a strategy to maximize the probability to bring AUVs together in the search process to ensure the exchange of data. The MDP is utilized to balance convergence and search[116].

Differently from the search and survey tasks, each AUV is responsible for a certain area during the surveillance task.Ferriet al.[55] proposed a distributed market-based task allocation algorithm that AUVs bid periodically. Considering the limitation of communication, several scenarios with different success rates of communication are simulated. The robustness is proved because of the intermittent communication and task reallocation mechanism [55].

For the cooperative searching, Zapata-Riveraet al.[117]compared the average energy consumption of three strategies for the cooperative searching, a leader approach, clustering approach, and belief, desire, and intention (BDI) approach. In the leader case, an AUV will become a leader when it finds the target firstly, the leader will inform its neighbors and AUVs in other zones. In the clustering case, if any AUV finds the target, it will inform the cluster head (CH), then the CH informs the mothership and other AUVs, which is the most energy-efficient approach. Compared with the CH approach,the BDI approach spends similar time and energy finding the target [117]. In [118], a multi-phase method was proposed to achieve an energy-efficient and fault-tolerant V-formation for multiple monitoring system and underwater navigation robot-II (MONSUN-II) AUVs. The low energy level AUVs will surface and alternate the location with high energy level AUVs, which will increase the endurance time of UMRS.

B. Motion Kind

Considering the inhomogeneous scalar field, or the communication-constrained network [119], the position of AUV affects measurements. Therefore, AUV prefers the area where it can achieve a better measurement in the cooperative sampling, searching, monitoring, and data collection. For target searching, AUV calculates the possible location of targets to maximize the probability of catching a target.Generally, the method of motion kind is the methods discussed in Section IV, which combines some constraints in the measurement-space. Some of these can be viewed as a path planning problem with a certain constraint. In [108], for high efficiency to generate the model of the scalar field, multi-AUVs are used and a selection basis function Kalman filter(SBF-KF) is proposed to estimate the scalar field model. To collect more information during the sampling, the mutual information between observations and estimations is represented as the uncertainty of the scalar field model, then combined with the rapidly-exploring random trees* (RRT*)algorithm to determine the next sampling position of the underwater vehicle.

For the cooperative search task, the 3D region is separated into several 2D regions by height. Lv and Zhu [109]considered the multi-AUV search at a 3D multi-target dynamic scenario and proposed a bioinspired neural network for searching and avoiding obstacles. Caoet al.[120]proposed a deep reinforcement learning approach for underwater searching, and the AUVs searched in different directions to avoid overlap. The experiment verified that the method had a higher statistical mean objective function value than traditional methods [120]. A reinforcement learning method called proximal policy optimization was utilized in[121] to find a time-optimal set of paths. In [122], the efficiency problem of cooperative surveillance was converted to the evaluation of the coverage degree and the target detection probability. A mathematical model was provided by considering static target and mobile target, respectively in a random scanning scenario. Considering the target exposure time, the relationship between AUV properties (sensing range and velocity of AUV, the speed of target) and target detection probability is formulated. And the theoretical analysis of the probability of achieving k-coverage is verified by random simulation.

For the hunting task of inhomogeneous multi-AUVs, Chen and Zhu [113] designed a hunting algorithm with the sequence: searching for evaders, forming a dynamic hunting alliance, and planning by a bio-inspired neural network. They verified the success rate of the algorithm in the 3D grid map,which has a lower success rate than the 2D environment because of the complexity of 3D hunting.

C. Measurement Kind

Considering the communication constraint and energy consumption, an intermittent belief propagation-based dead reckoning (IBPDR) framework was proposed to improve the overall localization accuracy by cooperation among AUVs[123]. When the cooperation does not occur, AUV will use the particle-based current-aided filter for self-localization to reduce communication costs. To ensure the convergence of the filter, or to obtain more accurate fusion information, the positions of AUVs should satisfy a certain relationship for the estimation. Measurement kind methods are commonly the variants of methods discussed in Section V which combined with some constraints by motion-space. For instance, Zhang and Leonard [107] described the formation shape and center by using Jacobi vectors and designed a cooperative Kalman filter for MRS to explore a static planar scalar field. To minimize the estimation error of the Kalman filter, the controller adjusts the formation shape adaptively online. For the cooperative target search and perception, a bistatic localization method was proposed in [124] to estimate the position of the target by exploiting the spatial and temporal diversity of measurement. Considering environmental interference, a transfer-reinforcement learning framework was employed in [125] for collaborative target recognition and recognized the same underwater target from different angles.Considering the failure of communication connectivity,Hollingeret al.[111] designed a minimum fusion rule for the coordination search of multi-AUVs and verified the possibility to achieve low average capture time under limited communication.

VII. FUTURE STUDIES

Although a great progress has been made in the researches on UMRS, there are still some problems needed to be solved.

A. Special Robot Design

To improve the performance of UMRS, researchers firstly consider improving AUVs in various tasks. However, the enhancement of AUV performance, especially in underwater communication and sensing, hardly makes a breakthrough over a short period of time. In this situation, two kinds of research ideas are competent for the field of UMRS. The idea of most researchers is to coordinate general AUVs with various methods to improve the overall performance of UMRS. Another idea is to design several types of heterogeneous and special robots with complementary functions according to the task requirements. The functions of AUV are relatively simple, but UMRS may make full use of individual abilities and play a role in tasks [9]–[11], [106].Researchers should comprehensively consider the task requirements and the abilities of underwater robots such as motion, communication, sensing, endurance and operation,design heterogeneous robots and corresponding coordination methods, and maximize the cooperation between them to make the overall functions of UMRS meet the task requirements. It is expected that the MRS composed of the special AUVs will play a greater role in ocean exploration and underwater operations before a breakthrough performance of AUV is made.

B. Experiment and Simulation

Most existing researches on UMRS are still at the simulation stage and lack experimental verification. The major reason is that the high cost of field experiments and highperformance AUV. There are two possible alternatives, i.e.,cost reduction and establishment of a universal simulation platform for UMRS.

Specifically, to reduce the experiment and system cost, a practical choice is to utilize miniaturized and highly maneuverable AUVs for experiments in the water pool or shallow water environment [18], [108]. Because of the high maneuverability brought by the bionic structure of robotic fish, the field required for the experiment is smaller than that of torpedo AUV. Therefore, many kinds of experiments can be performed in the pool. Another method is to adopt low-cost but effective underwater communication system to reduce the UMRS cost [106], [126].

A complete and universal simulation platform for underwater multi-robot simulation or semi-physical will accelerate the research progress. Researchers can build their dynamics models and control algorithms in the platform, and further employ the provided environment plug-ins to simulate wind, waves, and other environmental impacts. Thus, it facilitates the comparison between different studies under the same standard. At present, there are some similar systems in the field of aerial MRS, such as RflySim and XTDrone.Researchers can establish a dynamic model through Simulink,utilize the plug-ins to set up the environment, and connect the robot controller into the simulation system to conduct semiphysical simulation experiments. Different methods can then be compared with each other under the same standard, which can verify the effectiveness of the proposed algorithms and methods to a certain extent.

It should be emphasized that the necessity of experiments will never be replaced by simulation. Due to the high time cost and high economic cost of the outdoor experiments, the simulation platform is a compromise to some extent. An accurate and reliable simulation platform will advance the development iteration and make some predictions on the UMRS model, which is also the significance of the simulation platform. Eventually, the performance of UMRS will be tested by experiments.

C. Quantification of Emergent Function

In this paper, the concept of “emergent function” is mentioned, and UMRS is classified according to its category.However, the understanding of emergent function is still insufficient and needs further research. One of the difficulties is the quantification of emergent function. Although the emergent functions are tightly associated with the ability of a single AUV, the quantification of emergent function is different from the quantification of UMRS performance. The reason is that the performance of UMRS is an absolute quantity, while the emergent function is a relative quantity.Emergent functions are not only the expansions of AUV functions but also some new abilities. Thus, researchers in the UMRS field are more concerned about cooperative methods that can bring out more relative performance improvement and new abilities, which require further investigation and analysis of emergent functions. This quantification may lay a basis for the comparison of different MRSs. In addition, when the UMRS faces environmental change, robot failure, and complex dynamic situations, it can optimize strategies to maximize emergent functions by the quantification of the emergent functions, and switch tasks and roles for each robot to adapt to different unexpected situations.

D. Multi-Domain Cooperation

Multi-domain cooperation is emerging as a promising direction towards cooperation among various domains. The constraint of underwater communication limits the efficiency of information transmission of MRS, which is one of the main factors restricting the development of MRS. Unmanned surface vehicle (USV) and unmanned aerial vehicles (UAV)can complement the disadvantages of underwater robots, and assist AUV in information transmission, task scheduling, and precise positioning by radio communications and global satellite navigation systems. Consequently, high-speed communication across media, decentralized task allocation,and cross-domain coordination are the key technologies of multi-domain cooperation. In [127], the surface sinks, AUVs,and underwater sensor nodes composed of a data collection system to monitor underwater pipelines. The simulation showed that the multi-domain system composed at the variable speed AUV case has a higher average delivery ratio and lower average delay than the constant speed AUV case.The framework reduced the data retransmission by the nodes across the linear network, which leads to a reduction in energy consumption. Multi-domain cooperation will take a critical part in the wide-area target search, cooperative hunting,cooperative monitoring, cooperative sampling, and other missions.

E. Human-Machine Cooperation

At present, during the operation of UMRS, humans are often independent of the system and responsible for the initialization and maintenance of the system. In the tasks of underwater salvage, underwater rescue, underwater maintenance, and underwater scientific research, divers may cooperate with UMRS to complete the tasks together to complement each other. Thus, UMRS equipped with a variety of devices to reduce diver loads, such as communication modules, side-scan sonar and multi-beam sonar, and ultrashort baseline positioning system, can be exploited to enhance the diver’s flexibility and automation level. Besides, the life detection and support system can also be connected with UMRS to assess the physical condition of the diver. When dangers or accidents are detected, UMRS will alarm and assist divers to escape the dangerous situation. Therefore, the role of divers in the task should be taken into account when designing the MRS for human-machine cooperation. The system should be flexible and applicable, and more importantly, compliant control methods should be adopted when AUV approaches divers to guarantee the safety of divers.

The human-machine cooperation system may face more complicated and uncertain situations, and can be regarded as the group role assignment problem (GRAP). Therefore,researchers need to establish an efficient and flexible system,which can be constructed by role-based collaboration methods. The E-CARGO model (environments, classes,agents, roles, groups, and objects) is a typical framework to build a collaboration system, it considers the different types of messages in the communication among agents. The ECARGO model and its variants demonstrate the robustness,efficiency, and feasibility of providing role-based models for GRAP, and the role can be flexibly modified and adjusted[128]. The condensed E-CARGO model can adapt to the changing requirements and provide continuous solutions that minimize task switching among agents [129]. Even for a large set of instances of the problem and the conflict condition, the proposed solutions by the E-CARGO model performed effectively [130], [131].

VIII. CONCLUDING REMARKS

Achieving favorable cooperation is essential for a UMRS.This paper has reviewed the progress in UMRS, especially the methods for UMRS to perform various tasks in underwater environments. Specifically, we put forward a taxonomy of MRS according to the types of cooperation, and introduce primary methods for the cooperation of UMRS, such as task allocation, formation control, path planning, cooperative manipulation, cooperative navigation and positioning,cooperative sampling, cooperative monitoring, cooperative search, and cooperative hunting for underwater robots.Subsequently, we discussed some critical issues and future study directions to promote continuous improvement and innovation in the area of UMRS.

As an integrated technology, the UMRS in future practical applications should make trade-offs in several key factors typically involving cost, efficiency, stability, and redundancy.Currently, the research direction turns to more theoretical and more multilayer of UMRS. From a practical theoretical perspective, advancing the theoretical research of the emergent functions of UMRS is a significant and rewarding direction of effort. In particular, multi-domain cooperation will combine the advantages of UMRS and MRS in other environments and find wide application in diversified scenarios.