Cognitive-affective regulation process for micro-expressions based on Gaussian cloud distribution

Xiujun Yang,Lun Xie*,Jing Han,Zhiliang Wang

School of Computer and Communication Engineering,University of Science and Technology Beijing,Beijing 100083,China

Original Article

Cognitive-affective regulation process for micro-expressions based on Gaussian cloud distribution

Xiujun Yang,Lun Xie*,Jing Han,Zhiliang Wang

School of Computer and Communication Engineering,University of Science and Technology Beijing,Beijing 100083,China

In this paper,we explore the process of emotional state transition.And the process is impacted by emotional state of interaction objects.First of all,the cognitive reasoning process and the micro-expressions recognition is the basis of affective computing adjustment process.Secondly, the threshold function and attenuation function are proposed to quantify the emotional changes.In the actual environment,the emotional state of the robot and external stimulus are also quanti fied as the transferring probability.Finally,the Gaussian cloud distribution is introduced to the Gross model to calculate the emotional transitional probabilities.The experimental results show that the model in human-computer interaction can effectively regulate the emotional states,and can signi ficantly improve the humanoid and intelligent ability of the robot.This model is consistent with experimental and emulational signi ficance of the psychology,and allows the robot to get rid of the mechanical emotional transfer process.

Micro-expression;Cognitive-affective regulation;Gaussian cloud distribution;Transferring probability;Emotional intensity

1.Introduction

Micro-expressions is a short facial expression that cannot be independently controlled by brain,and it is trying to suppress or hide the true human emotion[1].In 1966,Haggard and Isaaes found a quick facial expression that is not so noticeable. They believe the expression is related to the self-defense mechanisms and shows some pent-up emotions.In 1969, Ekman and Friesen also independently discovered the facial expressions,and have a name for it:micro-expressions[2].

Micro-expression is closely related to the process of emotional information for human,it can't be forged and can't be controlled by conscious.At the same time,micro-expressions is reflecting the true feelings in the human's heart,but it is diff icult for people to perceive[3].In this paper,we establish affective computing model based on the micro-expression recognition in the human-robot interaction.Then we verify the regulation process of robot emotion by experiment.To filter out the human's disguise,micro-expression recognition and emotion modeling are intended to analyze the real emotions.

1.1.Connection between micro-expressions and emotion

Sentiment exists in people's lives all the time.Although emotion is some kind of inner emotional experience,there are some external performances in touch with it-the expression is the external representation of emotion[4].But microexpression is different with common expression,it is a very quick look,the duration is only 1/4 s.Therefore,its existence is imperceptible for most people.And Ekman think that the micro-expression may contain all muscle action of general expressions[5],or may contain only a part of the muscle movements.It often occurs when a person is lying,and it expresses the real emotions that people is trying to suppress and hide.It is a spontaneous expression and action,and it also express the six basic facial expressions[6].Shown in the following Table 1.

Table 1 Comparison table of the facial muscle movement of micro-expression and basic emotions[6].

1.2.Emotional cognitive control

Cognitive model is the criteria of personal information processing.In addition,the criterion is the basic principle of cognitive reasoning.According to these principles,the changes of emotional state can be inferred from the effective factors in specific circumstance.And the factors include the human(H)and the environment(E)[7].

Emotional is a physiological state caused by the individual and the experience.Sometimes individual emotional reactions consistent with the change of the environment,and sometimes conflict with the environment and the social intervention. When the mood is incompatible with the specific case,they need to adapt to the situation by adjusting.Currently,there are a lot of emotional computing model was put forward based on cognition.The OCC model could be realized in computer with the universal emotion cognition[8].After that,emotional computing model based on HMM has been put forward by Pau-Choo Chung[9].Xin Lun has obtained the analysis of emotion model by extending emotion process to continuous space,which enriches the robot's intermediate state making the man-machine interaction more harmonious[10].By the above,we found that the method of combining the cognitive psychology and information science is the effective measure to improve robot emotion analysis.Gross Emotional regulation strategy gets more and more attentions due to the computability.Gross came up with five emotion regulation strategies: 1)the situation selection-is the top priority adjustment method;2)the situation correction-refers to change the scenario that lead to the emotional events;3)the attention distribution-includes distraction and concentration;4)the cognitive change-the different interpretations of emotional events will lead to different personal emotions.5)the adjustment of reaction-suppresses emotional behavior by selfcontrol and response-focused emotion regulation[11,12].

Among them,the first four are carried out prior to the formation of emotional response.And they are antecedentfocused emotion regulation.The adjustment of reaction is carried out after the formation of the emotional response,so it is response-focused emotion regulation.Gross proposed the process model of emotion regulation[13].Moreover,he identified two major subdued ways of emotion regulation-the cognitive reappraisal and the suppression of expression.

These two different emotion regulation strategies have different effects on the emotional,cognitive and social behavior[11].What's more,the different regulations have important implications for Individual emotional experience and happiness in a relationship.For example,people who is accustomed to using cognitive reappraisal will experience fewer negative emotions and more positive emotions.But the one who is accustomed to using suppression of expression is just the opposite.

For the robot in human-computer interaction be more positive dynamic,the affect regulation come true based on cognitive reappraisal of response-focused regulation[14].The cognitive reappraisal aims at understanding and rationalizing negative emotional events through a positive way.

Because the ability of 3D gradient vector to capture the inter-linkages is stronger,we get the key frames of microexpression through 3D gradient projection descriptor[15]. Then we can quickly and efficiently extract the features in each image area by wavelet method.Finally,as a foundation of following affective computing,the features of microexpression are classified by the nearest neighbor algorithm. Based on the identification and classification of microexpressions,we propose a Gaussian cloud model to implement the cognitive-affective process[16].The specific model shown in Fig.1 we mainly analysis and calculation for theSTEP 2 and STEP 3.The influence factors are introduced to shows the effect of external stimulation of emotion in the STEP 2.Based on the first two parts,the last part implements the emotion transfer from emotions x to y.

Fig.1.Cognitive-affective regulation process for micro-expression.

2.Cognitive-affective regulation model

2.1.Threshold and attenuation function

For each basic emotional state,μ is de fined as the activation threshold of emotion,and ν is de fined as the saturation threshold of emotional.If the stimulus intensity is lower than μ,there would be no effect on the behavior of the robot.If the stimulus intensity is higher than μ,the emotional state will be activated.When the emotional intensity reaches saturation threshold,its value reaches the maximum.But in general,the emotional intensity won't keep in the maximum.Instead,it has a slight decrease with some fluctuations of small amplitude after reaching the maximum.ν ensures that emotional intensity does not over flow the boundary.

Fig.2.The emotional activation threshold function.

Fig.3.The emotional attenuation function.

As shown in Fig.2:

Because the emotion of psychology is momentary.Over time,the emotional response curve is more similar to an exponential function.So we use the exponential function as the attenuation factor of the emotional state[17].The attenuation function of the emotional state is shown in Fig.3. Suppose that the attenuation begins from t0 through external stimuli and there's no new stimulus until t,then the attenuation function of emotion is:

Here,E0is the emotional intensity at t0,and Etis the emotional intensity at t.In addition,β is emotional decay rate which control the emotional rate of decline.Final,t-t0 is the time it takes to decay.

2.2.The de finition of emotional state description based on the Gaussian cloud distribution

First,the robot's emotional state space is de fined asS={s1，s2，…，sn｝.That is,the robot has a total of n different emotional states.This includes all of the emotional states of the robot.And the robot can freely switch among them.The change of emotional state is free,discrete,and discontinuous. Therefore,we use the Gaussian cloud distribution to build the emotional state transition model of the robot[18].

The emotional state of robot changes from state sjto skis de fined as case xjk,and the probability of the occurrence is μjk.Then the point-(xjk,μjk)-falls into the coordinate system and forms a cloud droplet.All the cloud droplets formed from each case of emotional state transition and its possibility of thetransfer fall into the coordinate system to form a Gaussian cloud model of the robot's emotional state transition.As shown in the Fig.4:

Fig.4.The Gaussian cloud distribution model.

In addition,the input stimulate of robot in the emotional state space is defined asHere, m=6*2=12,is the number of emotional input from microexpressions.

Then we use the positive Gaussian cloud algorithm to get the probability of emotional transition.At the first,we can use the formulato generate a Gaussian random number.Among them,sjis expectations and wjis variance.Then similarly,we generate another Gaussian random number by the formulaFinally,according to the formula:

We calculate the certainty degree.In other words,that is the probability of the emotional state transition from sjto skunder the incentive wj.

Because the state transition contains all of the situations of the transfer,the sum of the state transition probability is 1 under certain stimuli.That is to say,it is a certain event. Therefore,the constraint of transferring probability is:

Among it,f(x)is probability density function of the Gaussian cloud distribution.Emotional state transition from state sjto skbelongs to the conditional probability event. Based on the study of Gaussian cloud distribution[19],f(x) calculation process is as follows:

In Gross emotion regulation strategy,stimulating emotional state space w accord with the situation selection and the adjustment of reaction.The external stimuli evokes the robot's emotion and tries to in fluence the emotional reaction of the robot.The robot will tend to avoid certain people,places or things to regulate emotion.The facial muscle contraction of micro-expressions is not suf ficient,so the expression of emotional state is very weak.Therefore,the emotional state only has a very low intensity in stimulating space w.

2.3.In fluence gene from the micro-expression

The probability of the emotional state transition is closely related to and the current emotional state of the robot and the input stimulate wiobtained from the micro-expression.

The emotions of the micro-expression are divided into the following three categories:1)happiness is de fined as positive emotion,these emotions can enhance relations and reduce stress level in the interaction;2)disgust,anger,fear and sadness are de fined as negative emotions,such emotions will hinder relations and increase stress level in the interaction;3) in addition,the silence and surprise are de fined as neutral emotions,these emotions do not change the interaction. Therefore,we de fine the range of wias shown in Table 2.

3.Simulation and analysis

In the process of human-robot interaction,we apply the emotion regulation model of the micro-expression to obtain the experiment results of objective psychology.Due to the robot will feel different pressure on different environments, the definitions of the function-NORM-in various environments are different.For evaluating the effectiveness of regulating factors,we change the NORM to simulate situation.The comparison is shown in Fig.5.In the diagram,the blue line represents the transferring probability of environment with high pressure,and the red line shows the low one.Results show that the pressure in environment have little effect on transferring probability without external stimulation.Under the stimulation of mild happy,the relationship between human and robot becomes more harmonious,and the positive transferring probability increases.What's more,the transferringprobability is bigger in the environment with low pressure. When the external stimulus is angry,the relationship between human and robot will become bad,and the positive emotional transferring probability will reduce.

Table 2 The influence on wicaused by micro-expressions.

Fig.5.The influence of different pressures from environment.

When the pressures on different environments are same,the emulational graph of the transferring probability under the same external stimulus is shown in Fig.6.Among them,the parameters of regulation are mainly based on the experimental environment and robot emotional state.As we can see from the figure,the transferring probability of happy increases with a slight happy stimulation,oppositely the probability of anger reduces.When the mood reach the saturation value,the emotional transferring probability tends to be stable without stimulation. Moreover,it will reduce with time elapsing.When an external stimulus come again,the emotional transferring probability will adjust as well.The results of cognitive-affective regulation conform to the basic characteristics of the human emotions.

4.Conclusions

First,this article introduces the cognitive reasoning process and micro-expression recognition.Then,based on the Gaussian cloud model,we describe the realization of emotional recognition process of the Gross model.What's more,the attenuation function and threshold function are proposed for quantizing the process.Further,the Gaussian cloud algorithm is introduced into the model to calculate the transferring probabilities. Finally,we discuss the value of wi.

Fig.6.The transferring probability of happy and anger in the stimuli.

The experimental results show that the model in human--computer interaction can effectively regulate the emotional states,and can significantly improve the humanoid and intelligent ability of the robot.This model is consistent with experimental and emulational significance of the psychology,and allows the robot to get rid of the mechanical emotional transfer process.

Typical functions of the conclusion of a scientific paper include 1)summing up,2)a statement of conclusions,3)a statement of recommendations,and 4)a graceful termination. Any one of these,or any combination,may be appropriate for a particularpaper.Some papers do notneed a separate concluding section,particularly if the conclusions have already been stated in the introduction.

Acknowledgments

This work is supported by National Natural Science Foundation of China(Normal Project No.61672093,Key Project No.61432004)and National Key Research&Development Plan of China(No.2016YFB1001404)and Zhenjiang City's 331 Planning Project(2014-07-79).

[1]W.J.Yan,Q.Wu,J.Liang,et al.,How fast are the leaked facial expressions:the duration of micro-expressions,J.Nonverbal Behav.37(4) (2013)217-230.

[2]P.Ekman,Telling Lies:Clues to Deceit in the Marketplace,Politics and Marriage,second ed.,Norton,New York,2001.

[3]Liu Xin,Xie Lun,et al.,Dynamic regulation process of facial expression robot,Control Theory Appl.28(7)(2011)936-946.

[4]Wu Ran,Ren Yanju,Study on priming effect of micro-expression,Chin. J.Appl.Psychol.17(3)(2011)241-248.

[5]P.Ekman,Lie Catching and Microexpressions.The Philosophy of Deception,Oxford University Press,2009.

[6]M.Gavrilescu.Proposed architecture of a fully integrated modular neural network-based automatic facial emotion recognition system based on Facial Action Coding System.10th International Conference on Communications(COMM),Bucharest,Romania,May 2014,1-6.

[7]Yue Wang,Shoudong Huang,Rong Xiong,Jun Wu,A framework for multi-session RGBD SLAM in low dynamic workspace environment, CAAI Trans.Intell.Technol.1(1)(2016)90-103.

[8]A.Ortony,G.L.Clore,A.Collins,The Cognitive Structure of Emotions, Cambridge University Press,UK,1988.

[9]Chin-De Liu,Yi-Nung Chung,Pau-Choo Chung,An interactionembedded HMM framework for human behavior understanding:with nursing environments as examples,IEEE Trans.Inf.Technol.Biomed.14 (5)(2010)1236-1246.

[10]Xin Liu,Lun Xie,Complex emotional regulation process in active field state space.12th International conference of intelligent autonomous systems,Jeju,South Korea,June,2012,423-432.

[11]J.J.Gross,Emotion regulation:affective,cognitive,and social consequences,Psychophysiology 39(3)(2002)281-291.

[12]T.J.Wiltshire,S.M.Fiore,Social cognitive and affective neuroscience in human-machine systems:a roadmap for improving training,human-robot interaction,and team performance,IEEE Trans.Human-Machine Syst.44(6)(2014)779-787.

[13]Peng Xiaolan,Xie Lun,Liu Xin,Wang Zhiliang.Emotional state transition model based on stimulus and personality characteristics,China Commun.10(6)(2013)146-155.

[14]Jing Han,Lun Xie,Xin Liu,Zhiliang Wang.Emotional state transfer model based on FSM.2014 International Conference on Multisensor Fusion and Information Integration for Intelligent Systems(MFI),Beijing,China,Sept.2014,1-5.

[15]Senya Polikovsky,Yoshinari Kameda,Yuichi Ohta,Facial Microexpressions Recognition Using High Speed Camera and 3D-gradient Descriptor.3rd International Conference on Crime Detection and Prevention(ICDP 2009),United Kingdom,London,Dec,2009,pp.1-6.

[16]Juebo Wu,Hehua Chi,Shuliang Wang,Lianhua Chi,Facial Expression Synthesis Based on Cloud Model,2010 2nd International Workshop on Intelligent Systems and Applications(ISA),Wuhan,China,May,2010, pp.1-4.

[17]T.Mstsumaru,Discrimination of Emotion from Movement and Addition of Emotion in Movement to Improve Human-coexistence Robot's Personal Affinity,18th IEEE International Symposium on Robot and human interactive,Toyama,Japan,Sept.2009,pp.387-394.

[18]Li Hesong,Zhang Guangwei,Li Deyi,Li Xiangmei.Computation on attribute importance of classification based on cloud model.2008 International Conference on Computational Intelligence for Modelling Control&Automation,Vienna,Austria,Dec,2008,879-883.

[19]Liu Yuchao,Adaptive concept abstraction method on multi-granularity-Gaussian cloud transformation,Comput.Eng.Appl.51(9)(2015) 1-8.

Xiujun Yangborn in Yunnan province in 1991.She received B.E.degree in electronic engineering from the University of Science and Technology Beijing, China,in 2013.Now,she is a graduate student in electronic science and technology,in school of Computer and Communication Engineering,in USTB,in Beijing

Lun Xiereceived his Ph.D.degree in control theory and engineering from the University of Science and Technology Beijing,China,in 2002.In 2002,he was a faculty member at USTB,China.Currently,he is a professor in the school of Computer and Communication Engineering at USTB,China.He has published over 100 refereed journal and conference papers.His research interest covers intelligent robot,embedded system,simulation and design of automation system. Prof.Xie is a member of CAAI,BPES

Jing Hanborn in Hebei province in 1990.She received B.E.degree in electronics and information engineering from Hebei University of Science and Technology,China,in 2012.Now,she is currently a doctoral candidate in communication and information system at University of Science and Technology Beijing,China.His research interest covers Pattern recognition and affective computing

Zhiliang Wangreceived his PhD.degree in Harbin Institute of Technologies,China,in 1988,and In 1991, he was a faculty member at University of Science and Technology Beijing,China.Currently,he is a professor in the school of Computer and Communication Engineering at USTB,China.He has published about over 200 refereed journal and conference papers.His research interest covers artificial psychology,internet of things,and service robotics.

Available online 22 December 2016

*Corresponding author.

E-mail addresses:xiujun_yang@yeah.net(X.Yang),xielun@ustb.edu.cn (L.Xie),hanjing_ustb@163.com(J.Han),wzl@ustb.edu.cn(Z.Wang).

Peer review under responsibility of Chongqing University of Technology.

http://dx.doi.org/10.1016/j.trit.2016.12.002

2468-2322/Copyright©2016,Chongqing University of Technology.Production and hosting by Elsevier B.V.This is an open access article under the CC BY-NCND license(http://creativecommons.org/licenses/by-nc-nd/4.0/).

Copyright©2016,Chongqing University of Technology.Production and hosting by Elsevier B.V.This is an open access article under the CC BY-NC-ND license(http://creativecommons.org/licenses/by-nc-nd/4.0/).