人类行为解释与转换系统的多通道进化框架研究

拉维·莱美， 阿卡里氏·阿帕德，尼噶

(1.埃塞特大学 电子通信学院，印度 博帕尔 462041； 2. 塞格尔应用技术研究所, 印度 博帕尔462041)

人类行为解释与转换系统的多通道进化框架研究

拉维·莱美1， 阿卡里氏·阿帕德2，尼噶1

(1.埃塞特大学 电子通信学院，印度 博帕尔 462041； 2. 塞格尔应用技术研究所, 印度 博帕尔462041)

摘要:人类行为解释系统已经成为不同学科都感兴趣的研究课题.人类行为的本质是多通道的，本文通过大量的文献分析，总结了研究人类行为解释系统的各种方法，进而提出一种人类行为解释系统的框架.该框架的主要内容包括数据表达、数据流、背景、定义数据集及其表达、数据分析、解释、应用开发、系统工程及可拓学方法等.总结了上述有助于人类行为解释系统进化的框架.在结论中，提出了一个三阶段进化框架，包括根据应用需求训练系统、系统的运行、对预期行为的转化.

关键词:内容对象； 框架； 人类行为解释； 系统工程； FACS； 基于模型的系统工程； 可拓学理论； 系统调节

The learning is as human behavior is multimodal the content comes from various inputs like images, video, audio, text etc i.e. content has multimedia and there is need to use the concept of CO (Content Objects) to address a broad range multimodal search and retrieval, multimodal interactions and find ways to define this container related to physical object, abstraction, event or a concept which is having multiple dimensions. There are two aspects dealt in this paper.

(1)Framework System Evolution: This involves system engineering, problem solving approaches, standards which need to be considered.

(2)Human Behavior Interpretation Systems: This involves, face recognition, gesture recognition, context analysis, psychology, personality, cultural issues, prior exposure, experience, learning.

The authors carried out exhaustive literature survey and some important findings are tabulated in this paper giving the relevant references and the learning from them for evolution of frameworks. Literature review findings on Extenic and their applications was also done and elaborated in the paper.Finally the process for evolving the framework is given in conclusion which is the future work of the authors.

1Review of Literature and Learning

1.1Framework System Development Study Findings

The authors carried out extensive literature review to find issues in Framework Development.

Zahariadis[1]have stated about authoring of content objects using XML. The Content Objects (COs) are Objects that have representation of a specific instance of either a physical object or a physical entity (an entity that has physical existence, e.g. an human being) or an abstraction (a general concept formed by extracting common features e.g: learning engagement, work efficiency),an event or a concept (human beings learning, humans at work, humans moving on street), which might have multiple views (many images, videos, audio files, text i.e multimedia; real-world and user-related information i.e context, characteristics of the subject). Content Objects thus have a polymorphic container, which may have media, rules, behavior, relations and characteristics or any combination of the above. The Authoring Tool have to define the content object with different types of media items, real-world information (location, context, weather, time, etc.) and user-related information (emotional, expressive characteristics, prior experience, prior learning etc) and produce a rich media representation which is a Content Object. P Daras, et al[2]have described RUCoD (Rich Unified Content Description in their paper " Introducing a Unified Framework for Content Object Description" and "The formal description of a CO is its RUCoD file, which is an XML-based document specifying descriptors for all the above input types. Vincenzo Croce[3]have described the RUCoD content object descriptor initially designed to serve needs of I-Search Framework and extended for Project CUBRIK. The learning for the research problem is that the human behavior is multimodal. The content comes from various inputs like face expressions, gestures captured from images, video, audio, text etc i.e. multimedia. There is possibility of using XML for defining CO (Content Objects) emerging from the multimodal nature of human behavior and allow creation of multi-layered structure, integrating intrinsic or latent properties of the content, dynamic properties, non-verbal expressions, emotional as well as real-world descriptors.

Stefan Kopp, et al[4]have examined four existing languages: BEAT, MURML, APML and RRL and have mentioned that the aim for the representation languages is to have independence of a particular application or domain, have independence of the employed graphics and sound player and to have a clear-cut separation between function-related versus process-related specification of behavior. They have proposed a three stage model called SAIBA (Situation, Agent, Intention, Behavior, Animation) with three stages namely intent planning, behavior planning and behavior realization. A Function Markup Language (FML), describing intent without referring to physical behavior, mediates between the first two stages and a Behavior Markup Language (BML) describing desired physical realization, mediates between the last two stages. The learning is that a representative language have been used which are independent of application employed encompassing event i.e. context, agent, intension and behavior to describe the functional and behavior. There is possibility of defining human behavior with functional and behavior with representative markup languages

System Engineering approach have involved comprehending the complex problem with graphical representations to communicate system′s functional and data requirements[5]. The graphical representations have included Functional flow block diagram (FFBD). These have been used to show the functions that a system have to perform and the order in which they have to be enabled and involves set of control constructs), Data flow diagrams (these have been used to show required data flow between the functions of a system and may involve data repository, data sources etc)N-squared chart (These have been used to showsN×Nmatrix for n functions, for each function outputs are located in the row of that function and all inputs are in column of the function, thus have been able to show both function and data) Case Diagram have been utilized to show human initiated functionality), sequence diagrams have described interaction among physical components in terms of an exchange of messages over time and used to specify data flow between subset of system components. Enhanced Functional Flow Block Diagram(EFFBD) have been used to display functions, control flows specifications and are complete enough providing dynamic, static validation. The learning is from systems approach is that the framework should have various representations depicting function flow, data flow, matrices, and sequence diagrams, enhanced function flow providing dynamic and static characteristics of the system framework. Representation is essential.

Cai Wen, Yang Chunyan[6]have advocated the use of Extenics to help dealing with contradictory problems with aid of computers. The solutions evolved have used basic elements including matter, affair and relation to express all things, all matter, different relations, problems and the process of solving contradictory problems. The learning is that Conjugate Extenics Analysis is a good process and can be adopted for evolving framework for contradictory problems like human behavior interpretation.

Jeff A Estefan[7]have described Model-Based Systems Engineering (MBSE) methodologies used in industry and characterized as the collection of related processes, methods, and tools. These have been used to support the discipline of systems engineering in a "model-based" or "model-driven" context. The learning is that for framework development, there is need of representing the processes, methods, tools.

William C. Elm, et al[8], "Integrating Cognitive Systems Engineering (CSE)throughout the Systems Engineering Process " have stated that the practice of CSE does not compete with SE (System Engineering) but, instead, have suggested that current SE practices need to ensure that the technology components are engineered with the users′ cognitive needs in mind. They have further V model of system engineering involving stages like concept of operations, high level requirements, detailed requirements, high level design, detailed design, implementation, integration and testing, subsystem verification, system verification, operation and maintenance.

The learning is that the framework should involve stages keeping in mind the cognitive needs and define detailed operations, requirements, design implementation, integration and testing at system level. At the subsystem level verification, system verification, operation and maintenance are vital. Successful systems design is possible only when the human is considered an integral component of the overall System and framework is evolved accordingly.

James N Martin[9]have defined a methodology as a collection of related processes, methods, and tools. It is essentially the application of related processes, methods, and tools to the identified Research problem. The purpose of each major System Engineering process model standard have been summarized as follows[10-14]:

(1)ISO/IEC 15288:Establish a common framework for describing the lifecycle of systems.

(2)ANSI/EIA 632:Provide an integrated set of fundamental processes to aid a developer in the engineering or re-engineering of a system.

(3)IEEE 1220： Provide a standard for managing a system.

The learning is that the in framework the possibility of using system designing using standards can be considered.

1.2Human Behavior Interpretation Study Findings

Rupinder Saini, Narinder Rana[15]have mentioned about FER (Facial Expression Recognition) as an ever green research field for Computer Vision, Artificial Intelligent and Automation. The Eigen face approach, principal component analysis (PCA), Gabor wavelet, principal component analysis with singular value decomposition etc have been directly or/and indirectly used to recognize human expression in several situations. To achieve accurate recognition two or more techniques have be used and combined.

The learning is there is need to combine various techniques and the success of technique is dependent on pre-processing of the images/ data used.

Beat Fasel, Juergen Luettin[16]have clearly stated that Facial expression recognition should not be confused with human emotion recognition.

The learning is that facial expression recognition is classification of facial motion and facial feature abstraction and human emotion recognition is multimodal. Hence, for evolution of framework for human behavior interpretation and transformation one will have to consider multimodal factors such as emotional voice, pose, gestures, gaze direction, facial expressions.situation, and contextual information. Hence Holistic approach is required involving multimodal entities.

Mandeep Kaur, Rajeev Vashisht[17]in their research on Comparative study of PCA for classification of emotion using Singular Value Decomposition and PCA on different parameters have found that Different techniques can be used with differing accuracy.

The learning is that the comparison of the various classification method can be done on parameters like singular values, dimensional reduction, recognition rate, principal components, noise removal, memory requirements, mathematical form, computational time, numerical properties in order to make a choice of technique.

Beat Fasel, Juergen Luettin[16]have carried out an exhaustive literature survey covering prominent automatic facial expression analysis methods and systems, Facial Motion, deformation extraction and aspects like face normalization, facial expression dynamics, facial expression intensity etc. The learning is that data from such detailed surveys, detailed processes can be adopted to evolve the framework.

Automatic classification of human behavior involves

(1)Understanding of bodily motion[18-20];

(2)Understanding of gestures and signs[21];

(3)Analysis of facial expressions[22],

(4)Interpretation of affective signals[23].

(5)Social signal processing deals with interactions between humans[24]integrating verbal cues with non-verbal behavioral cues

Analysis have been done in work places equipped with sensors termed as Ambient intelligence and it deals with smarter environments to make the space responsive to changes in these behaviors[25].

The spatio-temporal analysis of dynamics of human actions, observed through different sensory modalities, have allowed inference and customization[26-27].The temporal behavior which is related to time may be on micro scale (yawn, hand gesture, eye blink) or macro scale(daily activities like eating, using computer,reading, listening, writing, sleeping) have also been analyzed.

J B Cortes, F M Gatti[28]have analyzed different somatotypes. They have found that endomorphic individuals (round, fat, and soft) tend to be perceived as more talkative and sympathetic, but also more dependent on others; Mesomorphic individuals (bony, muscular, and athletic) have tendency to be perceived as more self-reliant, more mature in behaviour and stronger; ectomorphic individuals (tall, thin, and fragile) have shown tendency to be perceived as more tense, more nervous, more pessimistic and inclined to be difficult. The learning is that the have also been used for behavior analysis to elicit the attribution of certain personality traits

For non-verbal behaviours and the related behavioral cues[29-32]have carried exhaustive research. The associated behavior cues with technologies like computer vision and biometry in automatic detections include the following

(1)Physical Appearance (height, attractiveness, body shape)

(2)Gestures and postures (Hand gestures, posture, walking)

(3)Face and eye behavior (facial expressions, gaze, focus of attention)

(4)Space and Environment (distance, seating arrangement)

(5)The vocal behavior includes cues like prosody (set of speech variables, including rhythm, speed, pitch, and relative emphasis, that distinguish vocal patterns.), turn talking, vocal outbursts, and silence can be analyzed with speech analysis

Petty and Cacioppo[33]have postulated two different routes for changing the attitudes and the behaviors of a person, namely, a central route and a peripheral route in elaboration likelihood model (ELM). The central route has emphasis on attention and elaboration of arguments, and coherence, logic, and clarity of arguments. The peripheral route has used secondary attributes like the attractiveness familiarity, credibility of a source.

Fogg[34-36]have shown a behavior grid. The horizontal axis shows type of behavior change which includes performing new behavior, performing existing behavior, increase / decrease behavior or stop behavior. The vertical axis captures time, duration and schedule related aspects.It has been shown that shortest change is a one-time performance of a behavior, and the longest change is a change of habit and behavior is always performed. The learning is that such grids can be used for capturing Behavior Change for transformation applied.

Carlos Busso, et al[37]have showed that in the case of unimodal systems, some pairs of emotions are usually misclassified. These confusions could be resolved by the use of another modality. Hence, they have shown that the performance of the bimodal emotion classifier was higher than each of the unimodal system. The learning is that multimodal systems would give better performance

1.3Literature Review on Extenic Research and Applications of Extenics Theory

Wu Peixu, Liu Jianqun[38]have put forward a new design method for multi-functional products based on the three existing Extension design methods and have proved that the method of designing multi-functional products is feasible and effective

Yu Zhiwei, Li Xingsen[39]through a case study of share-learning teaching method have illustrated a model describing a system with element theory to avoid the difference and in-consistency of the basic-element modeling of relation among elements to normalize and conform the element modeling by adding, deleting, revising the basic-element and their characteristics.

Ye Yongwei, et al[40]have proposed solution using Matter-element input and output model for equipment monitoring parameters and fault types for the heating system. The parameter samples were taken into training, and a comparative simulation experiment was made for the result. The experiments have revealed that the extension neural network has a simpler construct and can respond faster compared with the traditional neural network.

Luo Liangwei, Yang Chunyan[41]have established a configuration model of product design to solve the low-carbon, green logistics packaging problems of flexible design in the field of ceramic logistics packaging based on the extension design theory.It sets design space, analyzes design objects and transcribes them into gene extension module for extension transformation based on the combination of extension theory and genetic engineering.It can express the characteristics by the implicit and explicit primitive module combinations and get a new design and favorable evaluation by establishing correlation function.Finally, by the aid of computer a new design model have been optimized with its effectiveness and feasibility verified.

LI Zhi-yong, Gao Feng[42]have worked on Oracle bone inscriptions (OBI) basic information arrangement which is the bottleneck of OBI information processing, and the OBI language automatic modeling is the meaningful method to resolve this problem. They have used an automatic modeling technique of extension model to support OBI information processing.A semantic database composed of OBI and modern Chinese have been built based on HowNet.They have used three concept tables to train the OBI text which are matter-element concept table, affair-element concept table and relation-element concept table using HowNet elements.The base-element model with a certain degree of maturation is extracted from the OBI text, and have used the base-element Extenics transformation rules it can realize extension reasoning.The experiment results have shown that the proposed OBI language model is helpful to the modern Chinese research.They have shown that it can provide solutions for the semantic deduction of unidentified OBI characters and the text content integration of broken Oracle bones in OBI information processing.

Chen Guofang, Yu Zhaoxuan, Zhou Pan[43]have evolved a model based on matter-element extension theory, matter-element extension model was established by matter-element construction and correlation solution for the Goaf safety evaluation which is a typical multi-factor and non-compatible problem. Representative indexes have been selected from three aspects of rock quality, environmental condition and goaf occurrence to built evaluation indexes system of goaf safety. The original model was improved by normalizing matter-element data via subordinate function and revising evaluation indexes′ weights via improved method to obtain improved matter-element extension model. uSing Gold mine as example, the result of the improved matter-element extension model accords better with the actual conditions so authors have shown that it is reasonable and reliable to apply it to goaf safety evaluation.

Li Wei, et al[44]have shown a new risk evaluation model for large-span tunnel based on the theory of Extenics. Moreover, the index weights of evaluation have been determined using correlation function method. They have used hierarchy-extension theory for large-span tunnel collapse risk evaluation and have demonstrated the accuracy and reliability of analysis and have successfully applied to the practical risk assessment of Shirenzigou Tunnel. The established model has provided a reference for the advanced forecasting large-span tunnel collapse risk in the engineering construction by using theory of Extenics.

Xiao Huimin[45]have demonstrated the necessity and feasibility of introduction of creative imitation innovation in the secondary innovation for technology innovation. They have taken vibrating screen of petroleum machinery industry for example and have described the entire innovation process by using extension innovation method. With the extension transfer, an innovative product they have obtained satisfactory innovative products and improved the utilization of emerging technologies, and proved it to be scientific and effective.

Yu Zhiwei, Li Xingsen[39]have stated that a system with element theory has been put forward to avoid the difference and in-consistency of the basic-element modeling. The affair-element modeling, matter-element modeling, relation-element modeling and basic-element model revising are included in the model.Also the modeling of relation among elements have been proposed to normalize and conform the element modeling by adding, deleting, revising the basic-element and their characteristics. They have shown a case study of share-learning teaching method to illustrate the process.

Lu Bairong, et al[46]have shown the parking guidance system evaluation model is based on a variety of factors and fuzzy complex analysis methods. They have experimented with a complex matter-element model based on Extenic, relational function, membership function, fuzzy matter element analysis, evaluation indicators, measured values and the evaluation level. The relation entropy method have been used to determine the indicator weight and the evaluation values of the parking guidance system.

Sun Xiangjun[47]have shown that for the contradiction problem of command decision one adopts matter-element extension transformation, analyses consistency of decision-making system,transforms the contradiction problem into consistency problem,evaluates consumption and effect of scenario according to evaluation rules. They have shown that the simulation experiment of formation air defense validates rationality of the model, and the extension model provides strategy for contradiction problem of command decision.

The learning is that Extenic was successfully used for number of problems in real world situations successfully.

Thus finding from the survey is that Extenic theory aspects have been used extensively for contradiction problem of command decision, parking guidance system evaluation model,and can be successfully used for contradictory problem of measurement and interpretation of human behavior and subsequently apply transformation for evolving the proposed framework.

2Evolving Multimodal Framework

From the Literature review the method used to evolve framework was

(1)Enlisting the multimodal characteristic of human behavior which include

①Physical Appearance (height, attractiveness, body shape)

②Gestures and postures (bodily motion,Hand gestures, cultural signs, posture, walking)

③Face and eye behavior (facial expressions based on FACS, gaze, focus of attention)

④Space and Environment (distance, seating arrangement)

⑤The vocal behavior includes cues like prosody (set of speech variables, including rhythm, speed, pitch, and relative emphasis, that distinguish vocal patterns.), turn talking, vocal outbursts, and silence can be analyzed with speech analysis

⑥Other cues and data can be text analysis, verbal recording analysis

⑦Background Experience, cultural background, financial background, friend circle

⑧Social media cues can be taken from face book, twitter and other content

⑨Further academic performance analysis, personality analysis, counseling reports, health reports all contribute to describe the subjects

⑩Subject data on people involved in a context (example: teacher and students; boss and office staff; theater artists and audience and so on)

(2)Capture tools include cameras, biometric devices

(3)Using Extenics theory and principles for

①Extenics Theory for defining the Content Objects (people, situation/ context) with detailed descriptions using matter-affair principles and possibly create an xml based object descriptor

②Extenics Theory for Interpretation of behavior

③ Extenics Theory for Transformation of behavior

(4) Using multiple methods for enhancing measurement accuracy for measurable cues face, gesture, voice

(5)Using Representative diagrams to show the processes, input, measurement

Based on the above, the authors have proposed a three stage Multimodal Frameworks for Human Behavior Interpretation and Transformation System which has been shown in Figure 1.

Fig.1　Evolution of Framework

The proposed model is shown in Figure 1. There are three stages:

(1) System Conditioning

(2) System Operations

(3) System Transforming

System conditioning: This stage essentially is for creation of subject data.

Improving the captured data (face enhancement, 3D modeling to get good input images for analysis, tabulate captured data for subjects involved).

① Mapping of subject data Background Experience, cultural background, financial background, friend circle, Social media cues taken from face book, twitter and other content

② Further earlier work profile record, academic performance analysis, personality analysis, counseling reports, health reports all contribute to describe the subjects

③ System Conditioning is adoption of required devices to capture source information based on application. In case the application is behavior interpretation of students the conditioning means database of students, teachers, setting devices in classroom, laboratories, creation of individual databases defining aspects like personality, history of academic achievements etc. In case application is related to office environment the experience record, skill sets data, setting of data capture in office environment. The conditioning depends on application planned. Similarly the comparison matrix needs to be defined. The comparison matrix defines the expected behavior and helps in measurement of intensity of the behavior measured

The system conditioning thus gives a refined input to the next stage to eliminate system errors due to unconditioned data.

The Context or Situation is the determining factor of behavior measurement. The context can be teacher student interaction in classroom, boss, peers in office, team of friends in a relaxed environment etc. The context determines the comparison matrix for behavior interpretation.

System Operations includes input, processes and output.

(1)Inputs: The inputs for human behavior interpretation framework could be Images, Video, Audio Data for the context and environment, prior historical data of the subject. The data capture can be for individual subjects or even for a group as interactions happen in groups and the source, channel and receiver.

(2)Process and Tools: The human behavior interpretation is multimodal in character and needs to be processed accordingly. Since several tools, methodologies are available. Appropriate ones can be used; new processes can be evolved using various theories. Authors are using Extenics Theory for Evolution of framework

(3)Output: The output is the interpretation based on inputs and the process, tools used.

The interpretations give compliance or divergence from to expected behavior. Based on the degree of divergence behavior interpretation is proposed.

System Transformations

Extenics Theory has demonstrated that matter can be changed as desired. The subjects are given the behavior change / transformation opportunity through different interventions. These are interventions done for systems environments where a desired behavior is required and the measurements are again done to ensure desired behavior from the subjects. The process would be iterative in nature and behavior desired would be function of time, transformations and challenge opportunities for the subjects. The measurements are again done by System Operations stage and degree of variation in term of divergence is indicative of behavior change due to transformation strategy given.

The use of Extenics adds new dimension of transformation thereby making the model more holistic.

3Conclusion

3.1Relevance of Extenics Theory

There is enough evidence that Extenics is appropriate for variety of problems from the literature review and the framework proposed as it gives the basis for the following:

(1)Defining multimodal elements like face, gesture, voice, physical aspects like personality under different contexts, cultural issues, prior exposure, experience, and learning. The usage of matter affair elements can help in defining and measurement of these modalities

(2)Availability of variety of methods in extensics to define systematic processes to undertake data-mining of datasets to interpret results. In this case the output desired is human behavior interpretation

(3)Using methods of goal setting explained in extensics to find strategies for transformation of behavior

3.2Appropriateness of Framework Proposed

The framework proposed has three stages to ensure the following

(1)Feeding inputs which are preprocessed to eliminate and reduce system errors inn operational phase and variations in gender, age, ethnicity, facial hair, cosmetic products, obstructing objects like hair, glasses, caps, turbans, pose, lighting changes can be overcome by input conditioning as uniform input (face with correct illumination, pose; video which has captured the correct trigger point, captured the peak of behavior display and also termination of the behavior; voice with noise removed, correct volume, pace, biometric inputs conditioned to acceptability level of the input for next stage)

(2)Giving due consideration to " context / situation" in the initial stage itself

(3)Giving due consideration to other multimodal factors which are not captured by input devices but available as "latent inputs" which are characteristic of the subject. These may be called the Latent characterizes developed in the subject and should be ignored as pointed in literature review.

(4)The framework evolution involves analysis and synthesis to evolve and create new framework. This involves studying and using system engineering, problem solving approaches, standards.

(5)It encompasses multimodal aspects like face recognition measurement and interpretation, gesture recognition, measurement and interpretation, context analysis, psychology, personality, cultural issues, prior exposure, experience, and learning.

(6)The biometric measurements can also be added.

(7)The framework is device independent as inputs from different devices can be usedThe authors propose to undertake advocacy of the above model, do mathematical modeling as future scope of the proposed framework.

References：

[1] ZAHARIADIS T, DARAS P, BOUWEN J, et al. Towards a Content-Centric Internet[C]∥ Towards the Future Internet Emerging Trends from European Research, [S.L.]:IOS Press, 2010:227-236.

[2] DARAS P, AXENOPOULOS A, DARLAGIANNIS V, et al. Introducing a Unified Framework for Content Object Description[J]. Int J Multimedia Intelligence and Security,2010,(3/4):351-375. http://www.cs.upc.edu/～tsteiner/papers/2010/rucod-specification-ijmis2010.pdf.

[3] CROCE V, Rich Unified Content Description[EB/OL].[2015-01-20].http://vcl.iti.gr/is/UC/RUCoD.pdf.

[4] KOPP S, KRENN B, Marsella S, et al. Towards a Common Framework for Multimodal Generation: The Behavior Markup Language[J]. Lecture Notes in Computer Science, 2006, 4133:205-217.

[5] LONG J. Relationships between common graphical representations in system engineering[EB/OL].[2015-01-10].www.vitechcorp.com/resources/technical_papers/200701031634430.CommonGraphicalRepresentations_2002.pdf

[6] CAI W, YANG C Y. A new transdisciplinary science-Extenics[C/OL].SISOM. Acoustics and Robotics, Bucharest:[s.n.],2013:21-22.[2015-01-04] www.imsar.ro/SISOM_Papers_2013/DO2.pdf

[7] ESTEFAN J A.Survey of Model based systems engineering (MBSE) Methodologies[R]. Incose MBSE Focus Group,2007.[2015-02-10]www.omgsysml.org/MBSE_Methodology_Survey_RevA.pdf

[8] ELM W C, GUALTIERI J W, MCKENNA B P, et al. Integrating cognitive systems engineering throughout the systems engineering process[J]. Journal of Cognitive Engineering and Decision Making, 2008,2(3):249-273. DOI 10.1518/155534308X377108

[9] MARTIN J N. Systems Engineering Guidebook: A Process for Developing Systems and Products[M]. Boca Raton, FL: CRC Press, Inc., 1996.

[10] ROEDLER G. What is ISO/IEC 15288 and Why Should I Care?[S].ISO/IEC JTC1/SC7/WG7, Geneva: International Organization for Standardization,2002.

[11] Processes for Engineering a System:ANSI/EIA 632[S].US, 1999.

[12] IEEE Standard for Application and Management of the Systems Engineering Process: IEEE Std 1220-1998[S]. Institute for Electrical and Electronic Engineers, 1998.

[13] Systems Engineering-System Life Cycle Processes:ISO/IEC 15288:2002[S]. International Organization for Standardization/International Electrotechnical Commission, 2003.

[14] Systems Engineering-System Life Cycle Processes: IEEE Std 15288TM-2004[S].Institute for Electrical and Electronic Engineers, 2005.

[15] SAINI R, RANA N. Facial expression recognition techniques, database & classifiers[J].International Journal of Advances in Computer Science and Communication Engineering,2014,2(2). http://www.sciencepublication.org/ijacsce/documents/v2issue2/3.pdf

[16] FASEL B, LUETTIN J. Automatic Facial Expression Analysis: A Survey[J].Pattern recognition, 2003,36(1): 259-275, http://infoscience.epfl.ch/record/82565/files/rr99-19.pdf

[17] KAUR M, VASHISHT R. Comparative study of facial expression recognition techniques[J/OL]. International Journal of Computer Applications, 2011,13(1):43-50. http://www.ijcaonline.org/volume13/number1/pxc3872368.pdf

[18]. WANG L, HU W, TAN T. Recent developments in human motion analysis[J]. Pattern Recognition,2003, 36(3): 585-601.

[19] MOESLUND T, HILTON A, KR¨UGER V. A survey of advances in vision-based human motion capture and analysis[J]. Computer Vision and Image Understanding,2006,104(2-3): 90-126.

[20] POPPE R. A survey on vision-based human action recognition[J]. Image and Vision Computing, 2010, 28(6):976-990.

[21] ONG S C W, RANGANATH S. Automatic sign language analysis: A survey and the future beyond lexical meaning[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2005,27(6): 873-891.

[22] SALAH A A, SEBE N, GEVERS T. Communication and automatic interpretation of affect from facial expressions[J]. Affective Computing and Interaction Psychological, Cognitive and Neuroscientific Perspectives, DOI: 10.4018/978-1-61692-892-6.ch008

[23] ZENG Z, PANTIC M., ROISMAN G, et al. A survey of affect recognition methods: Audio, visual, and spontaneous expressions[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2009, 31(1):39-58.

[24] VINCIARELLI A, PANTIC M, BOURLARD H. Social signal processing: Survey of an emerging domain[J]. Image and Vision Computing,2009, 27(12): 1743-1759.

[25] AARTS E H L, ENCARNAO J L. True Visions: The Emergence of Ambient Intelligence[M]. Berlin：Springer,2006.

[26] GUESGEN H W, MARSLAND S. Spatio-temporal reasoning and context awareness[M]//Handbook of Ambient Intelligence and Smart Environments.US:Springer,2010: 609-634.

[27] PENTLAND A. Looking at people: Sensing for ubiquitous and wearable computing[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2000, 22(1):107-119.

[28] CORTES J B, GATTI F M. Physique and self-description of temperament[J/OL]. Journal of Consulting Psychology1965,29 (5):432-439. http://www.dcs.gla.ac.uk/～vincia/papers/sspsurvey.pdf

[29] AMBADY N, BERNIERI F, RICHESON J. Towards a histology of social behavior: judgmental accuracy from thin slices of behavioral stream[J]. Advances in Experimental Social Psychology, 2000,32(00):201-272.

[30] EKMAN P, FRIESEN W V. The repertoire of nonverbal behavioral categories.[J]. Semiotica,1969,1: 49-98.

[31] KNAPP M L, HALL J A. Nonverbal Communication in Human Interaction[M]. New York: Harcourt Brace College Publishers, 1972.

[32] RICHMOND V P, MCCROSKEY J C. Nonverbal Behaviors in interpersonal relations[M]. Bacon: Allyn and Bacon, 1995.

[33] PETTY R E, CACIOPPO J T. The elaboration likelihood model of persuasion[J]. Advances in Experimental Social Psychology, 1986,19(4): 124-205.

[34] FOGG B J. The Behavior Grid: 35 Ways Behavior Can Change[EB/OL].[2015-01-18].http://bjfogg.com/fbg_files/page7_1.pdf.

[35] FOGG B J. The Behavior Grid[EB/OL].[2015-12-11].http://www.BehaviorGrid.org.

[36] CHATTERJEE S， DEV P. Proceedings of the 4th International Conference on Persuasive Technology[C].AMC,2009:42-46.

[37] BUSSO C, DENG Z G, YILDIRIM S,et al. Analysis of emotion recognition using facial expressions, speech and multimodal information[C/OL]//International Conference on Multimodal Interfaces,:ACM,2004,205-211. http://graphics.cs.uh.edu/wp-content/papers/2004/ICMI2004-emotionrecog_upload.pdf

[38] Wu P X, Liu J Q.Solution to Creating Multi-function Product Based on Extension Design Methods[J]. Journal of Guangdong University of Technology, 2015,31(3):10-17.

[39] Yu Z W, Li X S.Modeling of basic-element and its application[J]. Journal of Guangdong University of Technology, 2015,31(3): 5-9.

[40] Ye Y W, Ren S D, Ye L Q, et al. Fault diagnosis for automobile coating equipments based on extension neural network[J]. Journal of System Simulation,2015(3): 542-548.

[41] Luo L W, Yang C Y. Study on ceramic logistics packaging design based on gene extension modular design[J]. Journal of Guangdong University of Technology, 2015,31(2): 11-16.

[42] LI Z Y, GAO F, Extension model modeling for oracle bone inscriptions based on HowNet[J]. Computer and Modernization,2015(5): 30-34.

[43] CHEN G F, YU Z X, ZHOU P. Application of improved matter-element extension model to goaf safety evaluation[J]. Jiangxi Nonferrous Metals,2015(4): 91-96.

[44] LI W, XIAO H B, NING X D, et al. Application of extension theory to large span tunnel collapse risk evaluation[J]. Journal of Natural Disasters, 2015(3): 97-103.

[45] XIAO H M. Research on the creative imitation innovation based on basic element[J]. Operations Research and Management Science, 2015(5): 264-269.

[46] LU B R, WANG H L, ZHOU H Z, et al. Research on parking guidance system evaluation based on Fuzzy matter element[J]. Journal of Shandong Jiaotong University, 2014(2): 21-26.

[47] SUN X J. Extension model of command decision[J]. Ship Electronic Engineering,2015(2): 37-40.

Evolving Multimodal Frameworks for Human Behavior Interpretation and Transformation System

Ravi Limaye1, Akhilesh Upadhyay2, Nigam S R1

(1.Department of Electronics and Communication, AISECT University, Bhopal 462041, India;2.Sagar Institute of Research and Technology, Bhopal 462041, India)

Abstract:Human Behavior Interpretation System has been topic of interest from various disciplines. The nature of behavior is multimodal and authors have attempted to integrate various findings from literature review to find various approaches to evolve Human Behavior Interpretation System in developing a framework for human behavior interpretation. The important aspects of frame work include representation, data flow, context, defining datasets and their representation, data analysis, interpretation, application development, system engineering and Extenics approach. The paper summarizes the above factors which are helpful to evolve frameworks for human behavior interpretation system. In conclusion authors have evolved a framework which has three stages. Stage first is conditioning of the system depending on application, second stage is the operations of the system. Finally third stage is of transformation for expected behavior.

Key words:content objects; framework; human behavior interpretation framework; system engineering; FACS; model-based systems engineering (MBSE); ISO/IEC 15288; ANSI/EIA 632; Extenics theory; system conditioning

收稿日期:2015-12-22

作者简介:拉维·莱美(1965-),男，博士，主要研究方向为计算机与信息技术、机器视觉及在线学习等. 通信作者: 阿卡里氏·阿帕德(1973-)，男，教授，博士生导师，主要研究方向为电子工程. E-mail：akhileshupadhyay@gmail.com

doi:10.3969/j.issn.1007-7162.2016.02.002

中图分类号:N031

文献标志码:A

文章编号:1007-7162(2016)02-0005-14