A Visual Lossless Image ⁃Recompression Framework

Ping Lu，Xia Jia，Hengliang Zhu，Ming Liu，Shouhong Ding，and Lizhuang Ma

（1.ZTE Corporation，Shenzhen 518057，China；2.Shanghai Jiao Tong University，Shanghai 200240，China）

A Visual Lossless Image ⁃Recompression Framework

Ping Lu1，Xia Jia1，Hengliang Zhu2，Ming Liu1，Shouhong Ding2，and Lizhuang Ma2

（1.ZTE Corporation，Shenzhen 518057，China；2.Shanghai Jiao Tong University，Shanghai 200240，China）

In this paper，we propose a novel image recompression frame⁃work and image quality assessment（IQA）method to efficient⁃ly recompress Internet images.With this framework image size is significantly reduced without affecting spatial resolu⁃tion or perceptible quality of the image.With the help of IQA，the relationship between image quality and image evalu⁃ation scores can be quickly established，and the optimal quali⁃ty factor can be obtained quickly and accurately within a pre⁃determined perceptual quality range.This process ensures the image’s perceptual quality，which is applied to each input image.The test results show that，using the proposed method，the file size of images can be reduced by about 45%-60% without affecting their visual quality.Moreover，our new im⁃age⁃recompression framework can be used in to many differ⁃ent application scenarios.

image recompression；image quality assessment；user experi⁃ence；visual lossless

1 Introduction

B ecause of rapid growth in the number of images in the network and user demands for better image quality and faster loading，image⁃compression tech⁃nology has become a research focus.Many commer⁃ cial applications have been designed to improve user experi⁃ence and save cost by reducing the size of color images.Many companies have developed image⁃compression algorithms and have achieved a higher compression ratio without obvious loss of visual quality.Google has developed its own web image for⁃mat［1］，and Mozilla has developed mozjpeg［2］image⁃com⁃pression format.

The Joint Photographic Experts Group（JPEG）standard［3］and JPEG 2000 standard［4］reduce the size of images without obviously affecting image quality.Most images currently on the Internet are JPEG images，and the JPEG baseline algorithm has been used widely in many digital⁃imaging applications. JPEG lossless compression saves device storage memory and transmission bandwidth［5］.Our work mainly focuses on recom⁃pressing JPEG images in order to further reduce their size［6］.

Image quality is very important.In general，most people are less interested in how lossy compression is implemented；their main concern is that the visual quality of the compressed im⁃age is reasonable.Most people are willing to trade off fine im⁃age quality for the ability to save more images in a limited space.We use the Image Quality Assessment（IQA）method to preserve the perceptual quality of images.This method can be used to accurately predict the quality of a compressed image prior to compression.

2 Related Works

Many objective IQA algorithms have been proposed to evalu⁃ate image quality［7］.The goal of research on objective IQA is to develop quantitative measures for automatically evaluating image quality.Depending on whether the image is an original or whether a reference image is used，objective image quality metrics can be classified as full⁃reference，reduced⁃refer⁃ence，or no⁃reference.We use full⁃reference IQA in our pro⁃posed recompression system.Over the past decade，research⁃ers have proposed various utility IQAs，including mean squared error（MSE），peak signal⁃to⁃noise ratio（PSNR），struc⁃tural similarity（SSIM）index，information content weighted SSIM（IW⁃SSIM）index，and feature similarity（FSIM）index.

Early image⁃quality metrics that were widely used are MSE and PSNR.These metrics are determined by averaging the squared intensity differences of distorted and reference image pixels and were popular because they are simple to calculate and have clear physical meanings.

In 2004，Wang et al.［8］proposed SSIM，a state⁃of⁃the⁃art IQA model.SSIM is based on the hypothesis that the human vi⁃sual system（HVS）is highly adapted to extract structural infor⁃mation from a visual scene.This metric involving structural similarity can provide a good approximation of the perceived image quality.Wang et al.［9］then proposed a multiscale ex⁃tension of SSIM（MS⁃SSIM）that performed better than SSIM. Wang and Li［10］further improved on MS⁃SSIM by introduc⁃ing a new information content⁃weighting quality⁃score pooling strategy.The resulting IW ⁃SSIM performed better than MS⁃SSIM.

In 2011，Shoham et al.proposed a perceptual image quality measure called Block⁃Based Coding Quality（BBCQ）［11］. This metric evaluates the pixel⁃wise error using PSNR，added artifactual edges along coding block boundaries，and texturedistortion.In BBCQ，a weighted geometric average is used to combine these three measures.

Lin Zhang et al.proposed the FSIM index［12］.This new metric uses phase congruency and gradient magnitude to con⁃struct the local similarity map.The authors suggest that phase congruency and gradient magnitude play an important role in characterizing local image quality.

We have designed and implemented an image⁃recompres⁃sion framework in which image recompression does not percep⁃tibly reduce image quality.This framework is robust and efficient in many different applications.Experimental results show that the framework adaptively recompresses massive col⁃or JPEG images and that the loss associated with this recom⁃presion is imperceptible to the human eye.

3 Application Descriptions

Our framework for efficient image recompression（Fig.1）is intended to save as much bandwidth and storage as possible in applications involving massive Internet images.

4 Architecture and Design

4.1 Overview

Our proposed image⁃recompression system is based on IQA.It accepts an input image，typically in JPEG format，and outputs a recompressed JPEG image.This recompressed image is perceptually identical to the input image，but the file size is smaller.In our framework，image recompression can also be customized for different applications.

The image⁃recompression framework has six components：input image，initial recompression，quality measure，system control，image recompression，and output image（Fig.2）.

4.2 Components

The first component is the input image，which initializes the system.The framework computes the original quality factor of this image.The second component is initial recompression，during which the fixed quality factor is used to recompress the input image.

The third component is the quality measure，which is used to determine the quality of the recompressed image relative to the input image.This measure is based on gradient and texture similarity quality（GPT⁃IQA）and is more practical and accu⁃rate than other perceptual image quality measures.GPT⁃IQA has a range of 0 to 1，with 1 indicating an identical image and 0 indicating the worst image.GPT⁃IQA is based on gradient similarity，PSNR similarity，and texture distortion similarity. These three measures are combined by using an arithmetic mean，given by： where SGPTis the GPT⁃IQA score and graSim，p sn rSim and tdSim are the three measurement factors for GPT⁃IQA. We first divide the input image into many tiles，the sizes of which depend on the input image resolution.Then，the three factors are determined for each image tile.This metric evalu⁃ates the difference in quality between the original image and reference image.

The fourth component is the system controller，which con⁃trols image recompression.The distribution of the quality fac⁃tor and metric fit a sigmoid function.In Fig.3，we use the two quality⁃score pairs to construct the sigmoid function.The first pair is the original quality factor and a score of 1；the second pair is a fixed quality factor and calculated score.Then we can obtain the optimal compression quality factor from the prede⁃termined perceptual quality score.We can also obtain an opti⁃mal compression level for different applications by modifying this score.

The fifth and sixth components are image recompression and output image.The optimal target compression quality fac⁃tor determined by the system controller is used to recompress the input image，and the final image is then output.

4.3 Image Quality Assessment

Previous work indicates that gradient magnitudes and tex⁃ture significantly affect image quality.Therefore，we propose using an IQA method based on gradient similarity and texture similarity to measure image quality.We also add PSNR to make the results more closely resemble a subjective evaluation.

An image gradient is a directional change in the intensity or color in an image.Image gradients may be used to extract infor⁃mation from images，and calculation of the image gradient well⁃covered topic in image processing.Gradient operators can be expressed by convolution masks.In this paper，we use Prewittoperator as the gradient operator.The gradient magnitude is given by：

We compare the similarity between imageXandY.The similarity measure forG（）XandG（Y）is：

whereεis a positive constant that depends on the dynamic range of image’s gradient magnitude values.

Then PSNR similarity and texture similarity are given by：

Finally，we pool the three measures to obtain a similarity score for imagesXandY（1）.

4.4 Optimal Quality Factor

Generally，for satisfactory recompression，the optimal target compression level can be figured out by continuous iterations. However，immoderate iterations often severely limit recompres⁃sion performance.To reduce the number of iterations and in⁃crease recompression efficiency，we focus on an optimal com⁃pression level.

Using our image quality measure with its default parame⁃ters，we first analyze the relationship between the image quali⁃ty factor and GPT⁃IQA similarity score.We observe more than 10，200 Internet images and construct a perceptual similarity prior，i.e.，the quality⁃score distribution of one image can be fitted to a sigmoid function，defined as：

where x is the similarity score，and f（x）is the compressed quality factor.Given（x1，f（x1，）and（x2，f（x2，），the unknown coefficients are：

For the above distribution function，we have to first compute two of quality⁃score pairs.Whens=1，Q（）s approximates the source compression level of the input image.We can effi⁃ciently avoid one of those two pairs by estimating its compres⁃sion quality from the source image.Given the quantization ma⁃trix of the source image Msand the baseline matrix Mb，we can estimate the compression quality using their linear trans⁃formations，given by：

where q is the image quality and Mq，is its quantization matrix based on Mb，.According to（17），we can initialize one of the quality⁃score pairs，s1=1.0，Q（s1）.Moreover，given Q=60，we can determine thats2，Q（s2）=50.

After obtaining the two quality⁃score pairs，we can produce the corresponding optimal quality factor.Given the score threshold St，the optimal quality factor is：

4.5 Algorithm

Algorithm 1 shows the steps of image⁃recompression.

Algorithm 1.Image Recompression

Input：An image and required parameters

1.Get file size and other information of input image.If image size is very small or the image is not a normal image，then directly copy it.

2.Get quantization tables and source quality factor of the input image.If source quality factor is less than 60，then copy it.

3.Judge whether the quantization tables are standard quantization tables，if not，save the image according to its quantization tables.

4.Compress the input image byQlow，and compare the resulting image with input image through GPT_IQA method to get quality scoreSlow.

5.Predict parameters a and b of Sigmoid function from quality⁃score pairs（Qinit，Sinit）and（Qlow，Slow）.

6.Get optimal quality factorQoptaccording to the sigmoid function and threshold quality scoreSthresholdSthreshold.IfQoptis betweenQlowand Qinit，turn to step（8），else turn to step（7）.

7.GetQoptas the average value ofQlowandQinit.

8.Recompress the input image byQopt.

Output：recompressed image

5 Experiments and Evaluation

We investigated the potential of GPT⁃IQA in our image⁃re⁃compression framework.Our experiments were run on a PC with Intel®Xeon®CPU w3530 at 2.80 GHz and using Windows 7.

5.1 Proposed IQA and Other IQAs

There are many publicly available image datasets in the IQA community，including TID2013［14］，TID2008［15］，CSIQ［16］，and LIVE［17］.In our experiment，we chose the CSIQ as our test dataset.Two commonly used performance metrics are used to evaluate the IQA metrics：Spearman rank⁃order corre⁃lation coefficient（SROCC）and Kendall rank⁃order correlation coefficient（KROCC），which can measure the prediction mono⁃tonicity of an IQA metric.To prove that our method is more ac⁃curate than existing IQA for assessing JEPG image quality，we determine the quality of every compressed JPEG image and then obtain the SROCC and KROCC（Table 1）.

▼Table 1.Comparison of different IQAs

In Table 1，our method and FSIM are better than the other two IQAs.However，our method is faster than FSIM in obtain⁃ing the optimal quality factor，and our proposed IQA only needs to calculate image’s quality score twice，whereas FSIM needs to calculate it more times.

5.2 Compression Time and Ratio for Different Types of Images

Different types of images have different structures；there⁃fore，compressing all types of images to the same ratio is not ideal.Table 2 shows the different compression ratios for differ⁃ent types of images.The average picture compression time is less than 10 ms for images with a resolution of less than 400× 400 pixels.The average compression time is 23.734 ms for im⁃ages of objects with a resolution of less than 640×480 pixels. The average compression time is 157.091 ms for images of paintings with a resolution of less than 2300×2300 pixels.

We can compress images of buildings more than images of objects because most images of buildings have fewer colors and their structures are more regular.

In Fig.4，there are no perceptible differences between the pictures before and after compression.Thus the compression is lossless.The compressed image file is much smaller than the original image file，but the quality of the recompressed image remains high.Therefore，the lossless image⁃compression algo⁃rithm performs very well.

6 Conclusion

In this paper，we proposed a new IQA method for recom⁃pressing JPEG images.The loss associated with this recom⁃pression is imperceptible to the human eye.We obtain the opti⁃mal quality factor quickly and accurately within a predeter⁃mined perceptual quality range.The experimental results showthat our framework reduces the size of massive images by about 45%-60%，with a loss of quality that is imperceptible to the human eye.File size is significantly reduced without affect⁃ing the perceptual image quality.This improves user experi⁃ence，saves storage，and saves transmission bandwidth.

▼Table 2.Different compression ratios for different types of images

［1］Wikipedia.（2015，May 4）.WebP［Online］.Available：http：//en.wikipedia.org/wi⁃ki/WebP

［2］Josh Aas.（2014，Jul.20）.Mozilla advances JPEG encoding with mozjpeg 2.0［On⁃line］.Available：https：//blog.mozilla.org/research/2014/07/15/mozilla⁃advances⁃jpeg⁃encoding⁃with⁃mozjpeg⁃2⁃0/

［3］G.K.Wallace，“The JPEG still picture compression standard，”Communications of The ACM，vol.34，no.4，pp.30-44，1991.doi：10.1145/103085.103089.

［4］M.Rabbani and R.Joshi，“An overview of the JPEG 2000 still image compres⁃sion standard，”Signal Processing：Image Communication，vol.17，no.1，pp.3-48，Jan.2002.doi：10.1016/S0923⁃5965（01）00024⁃8.

［5］W.B.Pennebaker and J.L.Mitchell，JPEG：Still Image Data Compression Stan⁃dard.New York，USA：Van Nostrand Reinhold，1993.

［6］S.Ding，F.Huang，Z.Xie，et al.，“A novel customized recompression framework for massive internet images，”in Computational Visual Media Conference，Bei⁃jing，China，Nov.2012，pp.9-16.doi：10.1007/978⁃3⁃642⁃34263⁃9_2.

［7］Z.Wang and A.C.Bovik，Modern Image Quality Assessment.USA：Morgan& Claypool Publishers，Mar.2006.

［8］Z.Wang，E.P.Simoncelli，and A.C.Bovik，“Multiscale structural similarity for image quality assessment，”in Conference Record of the Thirty⁃Seventh Asilomar Conference on Signals，Systems and Computers，Pacific Grove，USA，Nov.2004，vol.2，pp.1398-1402.doi：10.1109/ACSSC.2003.1292216.

［9］Z.Wang and Q.Li，“Information content weighting for perceptual image quality assessment，”IEEE Transactions on Image Processing，vol.20，no.5，pp.1185-1198，May 2011.doi：10.1109/TIP.2010.2092435.

［10］T.Shoham，D.Gill，and S.Carmel，“A novel perceptual image quality measure for block based image compression，”in Proc.SPIE 7867，San Francisco，USA，Jan.2011，pp.786709-786715.doi：10.1117/12.872231.

［11］L.Zhang，D.Zhang，X.Mou，and D.Zhang，“FSIM：a feature similarity index for image quality assessment，”IEEE Transactions on Image Processing，vol. 20，no.8，pp.2378-2386，Aug.2011.doi：10.1109/TIP.2011.2109730.

［12］Z.Wang，“Rate scalable foveated image and video communications，”Ph.D.thesis，Dept.of ECE，The University of Texas at Austin，Austin，USA，Dec. 2001.

［13］Z.Wang and A.C.Bovik，“A universal image quali⁃ty index，”IEEE Signal Processing Letters，vol.9，no.3，pp.81-84，Mar.2002.doi：10.1109/ 97.995823.

［14］N.Ponomarenko，O.Leremeiev，V.Lukin，et al.，“Color image database TID2013：peculiarities and preliminary results，”in 4th European Workshop on Visual Information Processing，Paris，France，Jun. 2013，pp.106-111.

［15］N.Ponomarenko，V.Lukin，A.Zelensky，et al.，“TID2008—a database for evaluation of full⁃refer⁃ence visual quality assessment metrics，”Advances of Modern Radioelectronics，vol.10，pp.30-45，2009.

［16］E.C.Larson and D.M.Chandler，“Most apparent distortion：full⁃reference image quality assessment and the role of strategy，”Journal of Electronic Im⁃aging，vol.19，no.1，pp.001006：1-001006：21，Jan.2010.doi：10.1117/1.3267105.

［17］H.R.Sheikh，M.F.Sabir，and A.C.Bovik，“A sta⁃tistical evaluation of recent full reference image quality assessment algorithms，”IEEE Transactions on Image Processing，vol.15，no.11，pp.3440-3451，Nov.2006.doi：10.1109/TIP.2006.881959.

Manuscript received：2015⁃03⁃16

Biographiesphies

Ping Lung Lu（lu.ping@zte.com.cn）received the ME degree in automatic control theory and applications from South East University.He is the chief executive of the Cloud Computing&IT R&D Institute of ZTE Corporation.His research interests include augmented reality and multimedia services technologies.

Xia Jiaa Jia（jia.xia@zte.com.cn）received the MS degree from Dalian University of Tech⁃nology in 2001.She is a leader of Multimedia Technology Research Team at the Cloud Computing&IT R&D Institute of ZTE Corporation.Her research interests in⁃clude computer vision and its application field.

Hengliang Zhug Zhu（hengliang_zhu@163.com）received the MS degree from Fujian Nor⁃mal University，China in 2010.He is now a PhD candidate in the Department of Computer Science and Engineering，Shanghai Jiao Tong University，China.His cur⁃rent research interests include image and video editing，computer vision，computer graphics，and digital media technology.

Ming Liug Liu（liu.ming83@zte.com.cn）received the BE and MS degrees from Harbin Engineering University，China in 2008 and 2011.He is now a senior engineer at the Cloud Computing&IT R&D Institute of ZTE Corporation.His research interests in⁃clude augmented reality，visual search，and deep learning.

Shouhong Ding Ding（dingsh1987@yahoo.com.cn）received the BS and MS degrees from Dalian University of Technology，China in 2008 and 2011.He is currently a PhD candidate in the Department of Computer Science and Engineering，Shanghai Jiao Tong University，China.His current research interests include image and video edit⁃ing，computer vision，computer graphics，and digital media technology.

Lizhuang Mang Ma（ma⁃lz@cs.sjtu.edu.cn）received the PhD degree from Zhejiang Uni⁃versity，China in 1991.He is now a full professor and the head of Digital Media Technology and Data Reconstruction Lab at Shanghai Jiao Tong University，China. He is also the chairman of the Center of Information Science and Technology for Traditional Chinese Medicine at Shanghai Traditional Chinese Medicine University. His research interests include computer⁃aided geometric design，computer graph⁃ics，scientific data visualization，computer animation，digital media technology，and theory and applications for computer graphics，CAD/CAM.

This research work was supported in part by China"973"Program under Grant No.2014CB340303.