Toward a Neurophysiological Measure of Image Quality Perception Based on Algebraic Topology Analysis

Chang Liu,Xiaoyu Ma,Yijie Zhou,Jiaojiao Wang,Dingguo Yu

1 Institute of Intelligent Media Technology,Communication University of Zhejiang,Zhejiang 310018,China

2 Key Lab of Film and TV Media Technology of Zhejiang Province,Communication University of Zhejiang,Zhejiang 310018,China

3 College of Media Engineering,Communication University of Zhejiang,Zhejiang 310018,China

Abstract: The bandwidth of internet connections is still a bottleneck when transmitting large amounts of images, making the image quality assessment essential.Neurophysiological assessment of image quality has highlight advantages for it does not interfere with natural viewing behavior.However,in JPEG compression, the previous study is hard to tell the difference between the electroencephalogram (EEG) evoked by different quality images.In this paper, we propose an EEG analysis approach based on algebraic topology analysis, and the result shows that the difference between Euler characteristics of EEG evoked by different distortion images is striking both in the alpha and beta band.Moreover, we further discuss the relationship between the images and the EEG signals,and the results implied that the algebraic topological properties of images are consistent with that of brain perception, which is possible to give birth to braininspired image compression based on algebraic topological features.In general, an algebraic topologybased approach was proposed in this paper to analyze the perceptual characteristics of image quality, which will be beneficial to provide a reliable score for data compression in the network and improve the network transmission capacity.

Keywords: image quality assessment; electroencephalogram;algebraic topology analysis;Euler characteristic

I.INTRODUCTION

The development of mobile communication systems dramatically enriches people’s experience.However,massive image data also brings significant challenges for communication bandwidth.Even though the average bandwidth available to users and upstream limits for service providers are rising, the bandwidth of the Internet connection remains a bottleneck when transmitting large amounts of data, primarily when the amount of images are transmitted over the Internet[1,2].As a consequence, it is necessary to reduce the amount of data transmission to a certain extent,that is,image compression,which leads to the deterioration of image quality.Accordingly, the assessment of image quality has become particularly important.Subjective quality assessment is a relatively mature research field,and Mean Opinion Scores(MOS)are considered as ground truth for evaluating, validating, or benchmarking multimedia systems or services[3, 4].However, it is difficult to determine the reliability of each judgment in subjective opinion testing.To make matters worse is that the straightforward task of giving a judgment response such as score task has a disrupting effect because it interferes with natural viewing behavior.Consequently, scientists began to explore the neural mechanism of image quality perception,where neurophysiological approaches are taken as complementary methods to traditional psychophysical ones since quality assessment processes take place inside the media consumer’s brain[5–7].

In the wake of the electroencephalogram (EEG)technique, neurophysiological assessment of image quality becomes more economical and portable[5, 8,7, 9–14].Scholleret al.found that image quality changes evoked an event-related potential called P300,which was positively correlated with the magnitude of the change[15].Moreover, since P300 is not directly associated with sensory processing, the Steady-State Visual Evoked Potentials (SSVEPs) based paradigm was investigated as a complementary approach by Muelleret al.[7, 8].The result showed that the adjustment amplitude of SSVEP had been significantly negatively correlated with MOS.However,since both P300 and SSVEPs are Evoked Potentials (EPs), a subject has to participate in particular experimental paradigms to elicit them,which contradicts the experimental procedure in the conventional image quality assessment field[4].Therefore,αwave, as a spontaneous potential, has received extensive attention.Hayashiet al.[16]observed that the value of the power of EEG inαband was higher in the image with higher quality.Aleksandra finds that watching low-quality video content decreases the power of the frequency band in the parietal and occipital regions by power spectral density (PSD) analysis[17, 18].As a result,this paper first analyzes the quality perception effect in the alpha band of the subjects in the conventional image quality assessment experiment.

Accordingly, we investigated the effect of JPEG compression on the quality perception of subjects.However, PSD analysis in the alpha band cannot distinguish the quality perception difference efficiently.Fortunately,algebraic topology is promising to further explore the perception differences of images of different quality,enabling the analysis of the whole brain’s functional patterns without losing the local knowledge of the brain regions.Topological Data Analysis(TDA)provides new topological and geometric tools to analyze EEG signals[19–21].The neural activity distribution emerging in different brain areas will entail different brain patterns,and we expect to find its expression in subtle yet highly significant differences in topological characteristics.Santoset al.explored topological phase transitions in functional brain networks[22,23].They considered that a topological invariant, called the Euler characteristic,suffices to characterize the sequence of topological phase transitions in the complex network.Applying it to the resting-state functional magnetic resonance imaging analysis in the Human Connectome Project shows that topological phase transitions occur when the Euler entropy has a singularity, which remarkably coincides with the emergence of multidimensional topological holes in the brain network[24].These studies suggest that Euler entropy may be a powerful tool to reveal the essential response of the brain[25,26].As a consequence,Euler characteristics of the algebraic topological network are selected in this paper to analyze the brain responses to images with different distortion levels.

In our study,twenty participants were considered in the conventional image quality assessment procedure,and EEG signals were collected synchronously during the test.The neurophysiological feature of image quality perception was investigated by exploiting Euler characteristics of the EEG signal and further compared with the PSD method.Finally, the relationship between the algebraic topological features of the neural response and the image was further studied in detail.

II.METHOD AND MATERIAL

The framework of our proposed neurophysiological measure of image perception is shown in Figure 1.A subject(or observer)is presented with an image with a specific distortion level.Meanwhile,EEG data is collected by a special cap with electrodes synchronously(Step I).Phase-locking value(PLV)[27]among sensors is computed to construct a distance matrix(Step II)in order to obtain theVietoris-Ripssimplex(Step III).Finally, Euler characteristics are applied to analyze the neurophysiological features in the brain stimulated by images across the qualities(Step IV).

Figure 1. Neurophysiological method for image quality perception analysis.Step I: Collecting the EEG data.Step II:Calculating the distance matrix among electrodes.Step III:Constructing the Vietoris-Rips simplex.Step IV:Extracting the algebraic topology characteristics of brain network.

2.1 Stimuli

Forty source images were collected from a subset of the image database in[2]and all images had the same mean luminance and were resized to a reasonable size for display on a screen resolution of 1440×1080.All images were distorted by JPEG compression, and the Q-value of the High Quality (HQ) image is 70 while that of the Low Quality(LQ)image is 8.

2.2 Procedure

In the image quality assessment test,the subjects were asked to judge the clarity of the presented image.After a general introduction to the experiment and the preparation of the EEG cap, subjects started the test with EEG recording, which is shown in Figure 2.Eighty high-quality or low-quality images were randomly presented.Each trial began with a fixation slice that lasted 1 second, and then a distorted image appeared randomly for 5 seconds.A rest slice appeared followed to remind subjects that they could take a break for 1 second.Subsequently,the subsequent trial started to run.

Figure 2. Image quality assessment procedure.Each trial began with a fixed section lasting 1 second, followed by a randomly presented HQ or LQ image lasting 5 seconds,and subjects were required to evaluate image quality.A rest screen was then shown to remind the subjects that they could rest for one second.Then the subsequent trial began.

2.3 Subjects and Equipment

Data were collected from twenty healthy volunteers(9 males, 11 females; age: 23±4) with normal (or corrected to normal) vision.All the subjects were asked to look at the images with a view angle of 20 degrees and evaluate how clear the images were.Moreover, they were all naive to this image quality assessment experiment.Test equipment was Neuracle 64 System (Neuracle product; Sensor array: 64-channel adult-sized head cap under 10-20 system; Reference electrode: in the middle of Cz and Pz; EEG acquisition software: EEG Recorder; Wireless amplifier:NSW364).The sample rate was 1000 Hz,and the filtering window was 0.3 to 100 Hz.

2.4 Topological Data Analysis(EEG)

Topological data analysis for EEG data is shown in Figure 1 and the following is detailed:

EEG acquisition and preprocessing.EEG data are collected by EEG cap and filtered byαband (8∼13 Hz) and the filtered EEG signal of each trial among the whole brain region iswhereNis the length of data andMis the number of electrodes.Two seconds EEG data after the image appeared were used for analysis, which was downsampled to 250Hz for faster computing.Therefore,M=64 andN=500 in this paper.

Distance matrix computation.The distance between electrodes is calculated by the phase-locking value analysis of the filtered signals from electrodes in the EEG cap:

Firstly,applying the Hilbert transform toFto obtainH(F)and then calculating the instantaneous phase of each electrodeϕ:

Calculating the phase lock value between electrodespandq:

And the distanceDmatrix is obtain by(3):

Simplicial complexes construction.Simplicial complexes are constructed byVietoris-Ripsfiltration according to the distance matrixD, which is shown in Figure 3.A topological structure to an otherwise disconnected set of points is established to create a sequence of simplexes.The Betti number of a generic topological spaceSis composed byβ0,β1andβ2in this paper.β0,β1,β2is the number of connected components, holes, voids inS, respectively.In fact,according to the previous study, it is hard to obtain significant features directly from single Betti curve.Fortunately, further extracting the feature of the Betti curve by Euler entropy will hopefully reveal the perception differences of the subjects stimulated by images in different qualities.

Figure 3. Vietoris-Rips flitration.

Euler characteristics analysis.Each network has an associated topological structure, its simplicial complex, constituted by its nodes (k= 1), edges (k= 2),triangles (k= 3), tetrahedrons (k= 4), and higher (k-1)-dimensional parts, the so-calledk-cliques.The alternate sum of the numbers ofk-cliques determines the Euler characteristicχ.The Euler entropySχof the associated brain network is obtained by the alternate sum of the numbersClkofk-cliques:

The phase transition point of Euler entropy occurs at the intersection of Betti curves,and each side of the transition point of Euler entropy represents different topological structures,which means a critical topological change occurs in the brain networks.

2.5 Topological Data Analysis(Image)

In order to investigate the image quality perception from the perspective of algebraic topology,we further analyze the image itself by TDA, which is shown in Figure 4.In this paper,we proposed a method to calculate the algebraic topological properties of images and use this method to calculate the algebraic topological properties of images of different quality.Firstly,we divided the image into 64 pieces, that is 8 pieces× 8 pieces, and then calculated the distance of piecexand pieceyby (5).And the algebraic topological network of the image is calculated based on the obtained distance matrix according to the method mentioned above.

Figure 4. Image analysis based on algebraic topology.Step I: Dividing the image into 8 pieces × 8 pieces.Step II:Calculating the distance matrix among pieces.Step III:Filtrating and constructing the Vietoris-Rips simplex of image.

Wherer(i,j),g(i,j) andb(i,j) is the value of the red,green and blue channel of the pixel(i,j)in piece,respectively.

III.RESULT AND DISCUSSION

Topological structures of EEG data evoked by distortion images are constructed byVietoris-Ripsfiltration as shown in Figure 5.A topological structure to an otherwise disconnected set of points is established where a sequence of simplexes is created.A functional brain network is built for each value of the correlation thresholdε ∈[0,1] in the filtration process by assigning an edge linking two brain regions if their normalized correlation level is larger than 1-ε.Asεis enhanced, new edges are gradually attached, thus changing the topology of the brain network, which becomes increasingly denser and harder to analyze.Consequently,we use the Euler characteristic to track these changes in the evolution of both surfaces and networks.In addition, compared to the EEG mapping result (Model: BEM, template: MNI, source located:MNE),strong activities on right temporo-parietal cortex were observed both in algebraic topology structure as well as EEG mapping.

Figure 5. Vietoris-Rips simplex and EEG mapping.Red indicates high levels of neuronal activation in EEG mapping.The filtration process in a functional brain network where ε is from 0.1 to 0.45 is illustrated in the figure’s top half.

Furthermore, the functional brain networks evoked by different quality images are compared, as shown in Figure 6.Generally speaking, more brain regions are involved when the participant perceives an LQ image compared to an HQ one.Moreover,the topological structure of EEG elicited by an LQ image is more complicated since more higher-dimension simplexes are observed.Consequently, we hypothesize that the blurred images make the task more difficult, leading to more mental activities.On that basis,the Euler entropies of brain networks for different values are calculated.It is a remarkable fact that Euler entropy has a negative peak with the change ofε.Since the value ofεcorresponding to the negative peak of Euler entropy is different between the HQ and LQ situation,it is promising to extract the feature of the brain network associated with image quality perception.The Euler characteristics of brain network constructed by EEG signal in alpha band evoked by images in HQ and LQ deterioration levels are shown in Figure 6.According to Figure 6,the phase transition point of Euler entropy in the brain network evoked by the LQ image is earlier than that of the HQ image in the alpha band.Moreover, it is evident that the phase transition difference of the brain network caused by JPEG compression is pronounced(p<0.05).

Figure 6. Topological data analysis using Euler characteristics in functional brain networks.The simplex, Euler entropy curve,and phase transitions characteristic of brain network constructed by EEG signal in alpha band evoked by images in HQ and LQ deterioration levels.

In addition, we compared the approach proposed in this paper with the PSD analysis used in previous studies[17, 18], and the result is shown in Figure 7.The PSD for all segments was calculated, and alpha power for each segment was computed as the mean within the alpha band.Figure 7 (b) showed that low quality image has evoked high EEG power inαband,which is in consist with the previous study[28].However, no significant difference in the mean alpha frequency was observed(p>0.5).By contrast, the EEGTDA method proposed in this paper extracted distinct quality perception features (p<0.05), which provides strong support for the effectiveness of our approach.Moreover, through algebraic topological analysis, we found that the beta band of EEG also has a significant quality perception effect.The phase transition point of Euler entropy in the brain network evoked by the LQ image is significantly later than that of the HQ image in the beta band(p<0.05).

Figure 7. Mean alpha values per quality condition vs.EEG-TDA method.(a)Stimuli;(b)PSD analysis;(c)EEGTDA in alpha band;(d)EEG-TDA in beta band.

Figure 8 indicates that the algebraic topological properties of images are similar to those of EEG signals.When the image quality is low, the network structure of the image is relatively complex, which is consistent with the algebraic topological characteristics of the EEG signals it evoked.This probably is due to the fact that as the quality of the image decreases,the differences between the pixels become blurred,resulting in more simplicities.This result seems to imply that there is some essential correlation between the algebraic topological features of images and that of the brain, which is worthy of our attention in the future research.For instance,since the algebraic topological features of the image can reflect its quality and are related to the perception of the participants, it is hoped that a new brain-inspired image compression method can be developed to compress the data through the algebraic topological features of the image without affecting the quality perception.

Figure 8.Algebraic topological characteristic of images vs.brain response.

The research of communication systems based on user experience is undoubtedly the current development trend.Scientists have been looking for ways to describe users’ real feelings.The above results show that the method proposed in this paper reveals the essence of users’perception of image quality from the perspective of cognition to a certain extent,which provides a neurological basis for the research of image quality assessment and will promote its development.A better image quality assessment method will provide a more reliable guarantee for all links of communication and improve the service quality of communication.On the premise of ensuring the quality of service,reduce the size of the image files as much as possible so as to transmit as much data as possible in the environment of limited bandwidth,which is very important to the development of the communication field.In addition, understanding image transmission from the perspective of brain science will innovate the research ideas in traditional communication.Brain cognition will reveal the essential characteristics of image quality.By ensuring the transmission of image qualityrelated information, high-quality and low bandwidth transmission of images will be realized, which is expected to give birth to a communication network based on intention perception,which may bring new vitality to the field of communication.

IV.CONCLUSION

In this paper, we proposes an algebraic topologybased EEG analysis approach and applies it to the image quality assessment task.Our approach acquired quality-related neural information via integrating the EEG collection with conventional image assessment procedures and obtained physiologically meaningful brain responses to different distortion levels images by topological data analysis.The validation experiment results show that more brain regions in the right temporo-parietal cortex are involved when the subject perceives an LQ image than an HQ one.And comparing to the PSD-based method, the proposed approach effectively extracted the EEG features with significant quality perception differentiation.The phase transition point of Euler entropy in the LQ situation in the alpha band is earlier than that of the HQ situation,while the opposite is true in the beta band.Moreover, we further discuss the relationship between the algebraic topological features of the images and the EEG signals.The results implied that the algebraic topological properties of the image are consistent with the brain’s perceptual topological properties,which is possible to give birth to brain-inspired image compression based on algebraic topological features.In general, an algebraic topology-based approach is proposed to analyze the perceptual characteristics of image quality and the effectiveness is verified by the experiment,which will be beneficial to provide a reliable score for data compression in the network and improve the network transmission capacity.

ACKNOWLEDGEMENT

This work was supported by the Key Research and Development Program of Zhejiang Province (Grant No.2019C03138 and No.2019C01002).The experiment leading to these results has been performed in Key Lab of Film and TV Media Technology of Zhejiang Province and Shaanxi Joint International Research Center on Integrated Technique of Brain-Computer for Unmanned System.