Pose-free assembly retrieval based on spatial-contact skeleton

Jianfeng YU, Jiazhen PANG, Jie ZHANG, Yuan LI

School of Mechanical Engineering, Northwestern Polytechnical University, Xi’an 710072, China

KEYWORDS Assembly retrieval;Computer-aided design(CAD);Rotation connection;Similarity measure;Spatial-contact skeleton

Abstract In complicated product industry such as aircraft manufacturing,an assembly model contains abundant engineering information for use in design,manufacture,and maintenance.Assembly retrieval can be used to find relevant models for knowledge reuse.However,an assembly with rotatable joints may have many poses,which brings difficulty to assembly retrieval,since there is no pose principle for assembly design. Therefore, focused on rotatable joints in assembly, a skeleton-based descriptor for pose-free assembly retrieval is proposed. The centroid points of part surfaces and contact faces in an assembly are extracted to construct a spatial-contact skeleton. The skeletonbased distance is proposed to measure the distance between two surface points, which is invariant to the rotatable joints. The distribution of skeleton distances between two parts is used to describe the pair.Considering a part paired with all other parts in the assembly,the set of part pairs is used to represent a part, and the modified Hausdorff distance is used to measure the dissimilarity between parts for assembly retrieval. Experiments are conducted to compare the accuracy of the proposed descriptor to holistic and structureless descriptors. The proposed method is shown to retrieve assemblies with similar parts and structures regardless of their rotatable joints.

1. Introduction

In aircraft manufacturing, an enormous number of computeraided design (CAD) models containing design knowledge are stored in database.The assembly model is the main form of CAD model in these libraries, since it embodies the shape information of parts as well as the connection information of the structure.Assembly retrieval from an existing model is an efficient way to locate similar assembly models and reuse their relevant knowledge.Based on the model database,significant embedded knowledge in an assembly can be reused in design, manufacture, and maintenance by assembly retrieval.A designer does not require a complete understanding of a model to express a retrieval intention, which improves design efficiency and lowers study costs.For example,in an engineering application, a designer can search assemblies similar to a model to stimulate thinking on a design and improve it.

A CAD assembly is designed to perform specific kinematic functions with a set of parts. Some parts form a rotation connection to provide the flexibility to complete a task, but this means an assembly model may have different poses in the digital design environment.There is no initial pose principle for a 3D model, so similar assemblies may be designed with completely different poses, which causes inconvenience when retrieving similar assemblies.

The rotation connection is what leads to different poses of an assembly. However, not all assemblies can provide the needed connection information, since the parts in a product model are often simply gathered in hierarchies of subassemblies, and only the absolute positions are stored.Therefore,we define the contact face by analyzing the point intersection between parts, so as to obtain the contact relationship. The connection pair can be seen as two parts with a contact face.We studied Bai’s 2D skeleton for an image methodto construct a 3D skeleton for a pose-free descriptor.

In the proposed method,by extracting the contact face and a part’s spatial position from a digital model, the spatialcontact skeleton is constructed based on the contact, spatial vertices,and contact relationship edges.A description method for a part pair is proposed based on the skeleton, which is invariant to rotation between connection pairs. Considering a part paired with all other parts in an assembly,the set of part pairs is used to represent a part. The modified Hausdorff distance is applied to the descriptor to match parts between assemblies.The assembly similarity is calculated by optimizing all dissimilarities between the matched parts. Compared to current assembly retrieval methods, the proposed method is a novel way to realize pose-free retrieval.

The rest of this paper is organized as follows. Section 2 reviews related work. Section 3 introduces our approach to building the spatial-contact skeleton and constructing an assembly descriptor. In section 4, we demonstrate a method to compute assembly similarity based on the modified Hausdorff distance and the zero-one programming algorithm. We discuss our experiments in section 5, and provide our conclusions in section 6.

2. Literature review

2.1. Part retrieval

Based on the research on 3D model, single-part retrieval has been widely studied. Tsai and Chang.used shape and engineering information respectively to analyze the similarity between parts. Hong et al.chose a two-stage method based on the overall shape and detailed features to measure similarity.Li et al.used 2D images of parts to construct the pyramid histogram of oriented gradients (PHOG) descriptor and compute the similarities with weights confirmed by support vector machine(SVM).Based on the 3D convolution neuron network for 3D model description,Zhang et al.proposed the FeatureNet to describe the 3D information for machine feature recognition.Schleich et al.sampled points on a part’s surface to construct the skin model shape, which considers the mechanical feature in shape description.

From a topological perspective, Gao and Shahbuilt a descriptor based on a manufacturing face adjacency graph to recognize and classify a part’s features.Some researchers have considered the partial reuse of parts. Based on the topology graph,feature dependency directed acyclic graphs(FDAGs),hierarchical indices,and face adjacency graphs (FAGs)have been used for partial part retrieval.Sunil et al.proposed the hybrid framework to describe a part based on graph and rule. The framework considers the adjacent and volumetric feature interactions in the B-rep model for automatic recognition.

The shape and topology are two main comparison objects for single-part retrieval,since they respectively represent visual and functional similarity. The pose of a part is less often considered because most researchers study retrieval for a rigid part model.

2.2. Assembly retrieval

In assembly retrieval, set- and graph-based methods are commonly used for assembly description.

In set-based assembly retrieval,part matching for similarity measurement is usually based on a bipartite graph. Using the description method for a part, similarity calculation algorithms based on shape information have been proposed, such as the vector space model (VSM),earth mover’s distance(EMD),and modified Hausdorff distance (MHD).Though these algorithms ignored contact between parts, they were adopted to realize a flexible or fuzzy mode of retrieval regardless of pose.Chen et al.proposed a method for a hierarchical descriptor including the degree of freedom and layout of parts.Matching information was used as a filtering condition, and the similarity was calculated based on the part. Han et al.evaluated the semantic similarity between multi-source semantics information of parts with WordNet and domain ontology.A semantic matching model was built based on a weighted bipartite to compute the similarity of assemblies. To consider multiple attributes,it involves the assignment of weights to different attributes.It requires an understanding of the relationship between design intention and weight assignment;otherwise, to find an effective weight combination would require many attempts. Katayama and Satopresented a matching method for 3D CAD assembly models which considers the shape from different views. Each assembly is represented by a set of projections for similarity calculation.

Considering structure reuse, graph-based algorithms have been proposed in assembly retrieval. Deshmukh et al.took the relationship between a mate as the attribute of edge and the contents of the part as the attribute of node to build a comprehensive descriptor,which is used for the assembly retrieval.Zhang et al.took the contact face as node and the connection relationship as edge to construct a generic face adjacency graph (GFAG) to represent structure information. Wang et al.proposed a graph descriptor that independently describes parts and mating relationships of an assembly model.Parts are represented by vectors of shape distributions, and contacts are quantified by the average distributions of two contact parts. Han et al.described an assembly by an attribute adjacency graph (AAG) from multiple attributes and used an improved spectral clustering algorithm to cluster assembly models for assembly retrieval. A node in the graph-based descriptor contains part information, and an edge contains structure information, which precludes weight assignment.Lupinetti et al.proposed the Enriched Assembly Model(EAM) descriptor for retrieval which has structural, assembly interface, shape and statistic layers. The EAM matching is translated into a sub-isomorphism problem for similarity calculation. Compared with the set-based retrieval methods, a graph-based method considers the structure information,however,the graph searching is time consuming when an assembly has many parts.

It can be seen that shape information and topology information are separately considered in current retrieval studies.In the research of model spatial description, the 3D model skeleton methods, such as the medial axis (MA),is used to represent the topology information in space, but this method has difficulty embodying the contact relationship between parts for assembly retrieval. In 2D skeleton research,Bai and Lateckiused a set of vertices and edges to represent an image’s skeleton for retrieval. Inspired by this method, we study the feature of an assembly to construct a 3D skeleton for pose-free assembly retrieval.

3. Assembly descriptor based on spatial-contact skeleton

Focus on the parts and connection relationships in assembly,we introduce a spatial-contact skeleton to build a pose-free descriptor.

3.1. Extraction of assembly feature

Fig. 1 An assembly MAssembly and its parts mparti （i=1...7）.

3.2. Establishment of a spatial-contact skeleton

To describe the part’s spatial connection relationship in an assembly, a spatial-contact skeleton is defined as follows:

[Definition 1] The spatial-contact skeleton (SCS) is a 3D skeleton which has a part vertex set (V), contact vertex set(V), and contact matrix (E). An SCS can be expressed as

Fig. 2 Multiple contact faces between two parts.

3.3. Assembly description based on spatial-contact skeleton

The distance between two surface points can reflect the shape information of a model. Based on the two-point surface distance, Osada et al.proposed the shape distribution to describe a 3D model shape for similarity comparison. However, the two-point distance is influenced by its pose, since an assembly may have rotatable joints. We propose an assembly descriptor based on the spatial-contact skeleton for posefree retrieval.

The spatial-contact skeleton is essential to ignore the influence of the rotatable joints because the distance between two vertices of parts remains the same when the joint rotates.Based on this feature, we define skeleton distance based on the SCS between two random points on the model surface,as follows.

Fig. 3 Spatial-contact skeleton of an assembly.

Fig. 4 ds-p between surface point pa and part vertex vpartu .

Fig. 5 Skeleton distance (dskeleton) between pa and pb.

4. Assembly retrieval method

An assembly is represented by a skeleton-based descriptor Dwhich is a matrix of vectors.The size of matrix is decided by the number of parts and it is difficult to compare two matrices with different sizes directly.We use a two-step method to calculate the similarity between parts and then calculate the assembly similarity based on part matching for retrieval.

4.1. Part similarity calculation

Fig.6 Illustration of dskeleton distribution between part u and part v.

4.2. Part matching

If assembly A and B are similar,there will be a one-to-one part matching relationship between parts in A and parts in B.Therefore, the overall dissimilarity between two assemblies is calculated under the optimal part matching relationship. Based on DSM,the comparison of assemblies is turned into an 0-1 integer programming optimization problem:Find a matching matrix to minimize the sum of dissimilarity between two sets of parts.The model of optimization problem is shown in Eq.(13):

Fig. 7 Retrieval procedure.

5. Experiments and discussion

Experiments were conducted to verify the feasibility and validity of the proposed retrieval method.In the experiments,Catia Advanced Meshing Tools was used to mesh the surface of model.MATLAB 2019a was used to build assembly descriptor and calculate the similarity. The computer’s processor was an Intel Xeon E5-2640 at 2.50 GHz with 16 GB memory.

5.1. Experiments on assembly comparison

5.2. Experiments on assembly retrieval

To verify the effectiveness of the method, an experimental assembly model library was built, containing 171 assembly models in 18 categories, with 1217 parts in total.

We carried out five groups of experiments on a) a gate valve; b) a wheel; c) a robot; d) a seat valve; and e) a clamp.After inputting the searched assembly model, we arranged the overall dissimilarity values in ascending order and kept the top six results. Fig. 9 shows the retrieval results obtained from each group (results do not contain the query assembly itself).

The first retrieval result of group a)shows that as the overall dissimilarity increased, the assembly became less similar,which aligns with subjective opinion. The parts in the fourth and fifth assemblies were different in shape, whereas the contact relationships in the sixth assembly were different; thus they scored higher in overall dissimilarity. This indicated that the shape and structure influenced the measurement of similarity.

The results in group b) show that when an input assembly consisted of a small number of parts, it had fewer available results due to fewer constraints. As a result, it was acceptable that some assemblies of the same type did not score at the top of the list, as the sixth assembly displays. In this case, further research results are required to discover a correlative assembly product.

Group c) shows that the Ddescriptor had superior performance in distinguishing the dissimilarity of assembly shapes. The overall dissimilarity results between assemblies increased, whereas the similarity of the shape decreased.

Group d) shows that the proposed method was useful for sorting the similarity of the assembly. The top six results were similar to the input assembly, and the first assembly was most similar to the input, whereas the sixth assembly was the most dissimilar. It is worth noting that the first and second assemblies consisted of the same parts, but the second assembly was more similar to the input because it had the same structure. This shows that the proposed method can be used to determine the dissimilarity of structures.

In group e), the overall dissimilarity figures were relatively large,and this is because the change in the other similar assembly bodies was smaller than that of the whole;hence,it had little influence on the scale change.In the fifth group,the change part was the base part, which had a great influence on the determination scale, so the descriptor was changed, and the distance between parts increased, resulting in high overall dissimilarity.

In the database with 171 assembly models, five assemblies have 6, 5, 7, 6, 8 parts and it took 3.72, 3.14, 4.23, 3.74, 5.32seconds in the above retrievals. The average time of similarity analysis between two assemblies was 0.00368 seconds. The increment of parts in an assembly increases the size of assembly descriptor Dso it also increases the retrieval time in similarity analysis.

Table 1 Contact relationship in the assembly A.

Table 2 Shortest distance (mm) between part vertices dp-p（）.

Table 3 Descriptor (histogram representation).

Fig. 8 Two-part pair sets to be compared.

5.3. Discussion of pose-free retrieval

An assembly may have different poses due to it has rotation joints. This poses a problem for describing an assembly in retrieval.

The Ddescriptor can solve this problem, since the parts are connected by a spatial-contact skeleton that is invariable to the vertices’rotation.An experiment was carried out to compare the D2 descriptorwith the proposed Ddescriptor on an assembly with two poses. The results are shown in Fig. 10.

The result indicates that the D2 descriptor of the assembly with a different pose was quite different from that of the original assembly, since D2 represents the overall shape feature.However, the overall dissimilarity between Dof the two assemblies with different poses was 0. It means that the assembly with a different pose remained the same Dwith itself. The experiment illustrated that this method can ignore the influence of pose to support pose-free assembly retrieval.

Fig. 9 Retrieval experiment results.

5.4. Discussion of retrieval accuracy

Accuracy experiments were carried out based on the rate of recall and precision. Comparison between different methods,including D2 descriptor,VSM,EMD,and MHDare displayed.

For each method, we applied the algorithm to the same database (section 5.2). Every assembly was retrieved, and the similarities between assemblies were calculated. The number of most similar assemblies was taken as the intermediate variable, and its corresponding recall and precision rates were related.We obtained the relationship between recall and precision, as shown in Fig. 11.

The curves in Fig. 11 illustrate recall and precision rate,where the upper curve outperforms the lower curve. The proposed method and other methods performed better than the original D2 descriptor, because they consider the part matching in retrieval. The Ddescriptor got decent results among these methods. The curve of proposed descriptor and curve VSM are very close, but the former had more precision when the recall rate was low. It is practical in assembly retrieval because similar results can be obtained quickly.It is noticeable that the proposed method does not reach the retrieval accuracy level of MHD, because the curve reflects the ability of retrieving all assemblies in a category. The MHD method only considers the part shapes in an assembly and it can find all assemblies with similar parts in a category easily. The proposed method additionally considers the contact information between parts, so some assemblies that have the same parts as the input assembly but have a different structure are denied in retrieval,which leads to a low precision and recall rate.For example,in the set valve category shown in Fig.9,the parts in the second similar assembly are the same as that in the input seat valve, but it only gets a dissimilarity result of 1.9489 because it has a different structure from the input. For the MHD method,the second similar assembly gets a dissimilarity result of 0 despite it has a different structure,which means the MHD method cannot figure out the difference in structure.Therefore,the proposed method can filter out some assemblies with different structures, and finally provides assemblies with similar part shapes and a similar structure for reuse.

Fig. 10 D2 and DSkeleton descriptors for assemblies with different poses.

Fig. 11 Comparison of recall and precision.

6. Conclusion

We proposed a pose-free assembly descriptor for retrieval based on a spatial-contact skeleton. Part shapes and contact relationships were considered in the form of a distribution of skeleton distances. The modified Hausdorff distance was adopted to compute the similarity between descriptors,thereby achieving overall assembly retrieval. Experiments were conducted to demonstrate the feasibility of the method, whose characteristics can be summarized as follows.

(1) The proposed Ddescriptor represents the assembly shape and contact information based on the spatial-contact skeleton. It is invariant to the rotatable joints to realize pose-free retrieval.

(2) Based on the proposed descriptor, an assembly is represented by a matrix of vectors and the parts of it are represented by a set of vectors. The descriptor can be generated in advance for online assembly retrieval.

(3) Compared with methods including D2,VSM,EMD,and MHDin assembly retrieval experiments,it was superior to the holistic descriptor D2 but inferior to the MHD method.However,the proposed descriptor considers the connection information compared with MHD method, which has more significance in assembly reuse.

The proposed descriptor solves the rotation issue in posefree assembly retrieval to support the mode reuse in aircraft design.However,it does not take other connections like translation into consideration. In future work, other types of connections will be studied to realize a more flexible assembly retrieval.

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

s

This study was co-supported by the National Natural Science Foundation of China(No.51475371)and the Key R&D project in Shaanxi Province (No. 2019ZDLGY0201).