A Selective Moving Window Partial Least Squares Method and Its Application in Process Modeling☆

Ouguan Xu*,Yongfeng FuHongye Su,Lijuan Li3Zhijiang College,Zhejiang University of Technology,Hangzhou 3004,China

2State Key Laboratory of Industrial Control Technology,Institute of Cyber-Systems and Control,Zhejiang University,Hangzhou 310027,China

3College of Automation and Electrical Engineering,Nanjing University of Technology,Nan jing 210009,China

A Selective Moving Window Partial Least Squares Method and Its Application in Process Modeling☆

Ouguan Xu1,*,Yongfeng Fu1,Hongye Su2,Lijuan Li31Zhijiang College,Zhejiang University of Technology,Hangzhou 310024,China

2State Key Laboratory of Industrial Control Technology,Institute of Cyber-Systems and Control,Zhejiang University,Hangzhou 310027,China

3College of Automation and Electrical Engineering,Nanjing University of Technology,Nan jing 210009,China

A R T I C L E I N F O

Article history:

Received 15May 2013

Received in revised form 14 September 2013

Accepted 16 October 2013

Available on line 20 June 2014

SMW-PLS

Hydro-isomerization process

Selective updating strategy

Soft sensor

A selective moving window partial least squares(SMW-PLS)soft sensor was proposed in this paper and applied to a hydro-isomerization process for on-line estimation of para-xylene(PX)con ten t.Aiming at the high frequency of model updating in previous recursive PLS methods,a selective updating strategy was developed.The model adaptation is activated once the prediction error is larger than a preset threshold,or the model is keptunchanged. Asa result,the frequency of model updating is reduced greatly,while the change of prediction accuracy is minor. The performance of the proposed model is better as compared with that of other PLS-based model.The compromise between prediction accuracy and real-time performance can be obtained by regulating the threshold.The guidelines to determine the model parameters are illustrated.In summary,the proposed SMW-PLS method can deal with the slow time-varying processes effectively.

©2014 Chemical Industry and Engineering Society of China,and Chemical Industry Press.All rights reserved.

1.Introduction

Partial least squares(PLS)regression has many excellent attributes such as simple model structure,stable and robust performance,and fewer training samples needed[1,2],so it is widely used in process modeling[3,4],process control[5,6],process monitoring and fault diagnosis[2,7,8].However,the PLS model may be locally valid and its performance will be degraded due to high level noise and disturbance in samples or time-varying industrial process such as catalytic decaying, equipment aging,or process drifting[9-13].Many adaptive PLS models [9-16]have been proposed to deal with the dynamic behavior of processes.The basic recursive PLS was first proposed by Helland et al. [9]and then modified by Qin[10].A representation was given for the old data that maintain the in formation without increasing the dimensionality.The b lock-wise recursive PLS algorithm developed by Qin [11]was extended from the basic form with a moving window and a forgetting factor.The algorithm could adapt the model based on new data and the old PLS model,avoiding re-modeling the old data.A fast moving window algorithm[12]was derived to update the kernel PLS model.The proposed approach adapted the parameters of inferential model with the dissimilarity between the new and oldest data.The time varying characteristics of processes could also be dealt with effectively by moving window approach[11].However,the effect of discarded sample on the model could not be evaluated properly.In this case,a new recursive PLS model was developed by Liu[14]through updating the mean and variance of the new sample and old ones,so part of historical in formation of the abandoned sample remained.The effective model was expanded to an on line dual updating method by Mu et al.[15],integrating the model updating and the output offset updating.Since the dual updating strategy takes the advantages of the two methods,it is more effective in adapting process changes.A similar dual updating scheme was proposed by Ahmed et al.[16]for the prediction of melt index of a continuous polymerization process.The recursive PLS models are updated repeatedly either in block-wise or sample-wise once any new sample(s) is available,inducing a heavy load on the model manager or computational machine.In order to improve the real-time performance of the model, the frequency of model updating should be properly controlled.Among the proposed adaptive PLS models,the frequency of model updating is reduced with the dual updating method proposed by Mu et al.[15],since the model updating is activated at regular intervals.Different from the strategy proposed by Muetal.[15],the decision of which updating method to be perform ed is based on the prediction error[16].A novel adaptive modeling method was proposed by Lee et al.[17].Depending on the model performance assessment,partial or complete adaptation is utilized to remodel the PLS method.The adaptive modeling method shows better updating performance and lower updating frequency compared to the block-wise recursive PLS modeling technique.

For the purpose of on-line application,more attentions have been paid to the real-time performance of the model.In this paper,a selective modeling strategy is proposed for the moving window PLS to decrease the model adaptation frequency by a p reset threshold.The modelwill be updated if its prediction accuracy exceeds a confidence limit, otherwise,the model remains unchanged.The selective moving window PLS(SMW-PLS)method is applied to an industrial example to predict para-xylene(PX)content in the hydro-isomerization process of C8-aromatics.

2.SMW-PLS Method

2.1.PLS method

Given a pair of input and output datasets X and Y that have been standardized,where X∈ℜn×mand Y∈ℜn×l.The linear relationship of the two matrices can be expressed as

whereβis the coefficient matrix and e is the noise vector.

The PLS regression builds a linear model by decomposing matrices X and Y in to bilinear term s,

where E1and F1are the residual matrices,p1and q1are the loading vectors,respectively,and t1and u1are the latent score vectors of the first PLS factors determined by

where w1and c1are the normalized eigenvectors of corresponding dominant eigenvalue of matrices XTYYTX and YTXXTY,respectively.

The relationship between t1and u1is

where b1is the regression coefficient and r1is the residualvector.

After the calculation of the first factor,if t1and u1do not contain enough in formation,the second factor is calculated by decomposing the residuals E1and F1with the same procedure for the first factor. The procedure is repeated until the model accuracy is satisfied.Finally, the value^βcan be estimated by

2.2.Moving window PLS method

Industrial processes often experience time-varying changes,such as catalyst decaying,sensor and process drifting,as well as degradation of efficiency.Several adaptive algorithm s[9,11,15,18]have been proposed to update the model based on new process data that reflect process changes.Xu and Chen[18]have shown that the simple PLS model presents the best real-time performance among the PLS-based models,but its accuracy and tracing performance are the worst.In order to improve the prediction accuracy and tracing performance of the model,the moving window approach is proposed for the simple PLS method.The model will be updated once a new sample is available while the oldest sample is discarded for the training data.As a result,the number of training samples is kept constant.

2.3.Strategy of selective updating for the moving window PLS method

The moving window PLS model needs updating repeatedly if any new sample is available.This high updating frequency is time consuming.Hence,an approach to reduce the updating frequency is greatly required.

A strategy of selective updating for them ode l is proposed.The idea is as follows.If the m ode l predicted value(^yk)is consistent with the actual measurement value(yk)of the process,and the prediction error is less than a preset threshold,that means the current m ode l reflects the behavior of process exactly and does no t need updating.The new sample(a pair of[χk,yk])is added to the training sample set and the oldest one is omitted.Subsequently,the sampling dataset is renewed and the number of training samples remains unchanged.When the erro r between^ykand ykexceeds the p reset threshold,the available pair of[χk,yk]will be incorporated into the training sample set,while the oldest one is abandoned.Then the model updating will be activated with the latest sampling dataset. A relative prediction error bound which is a small positive number, is introduced asa threshold for measurement of complexity conditions. A new sample will be introduced to sampling set and the model is updated once Eq.(7)is satisfied.

where RE is the absolutely relative error,andδis a predefined positive threshold,which can be set flexibly according to the demands and will be discussed in the next section.

2.4.Procedure of the on-line algorithm

With the above moving window PLS model and selective updating strategy,the procedure of on-line modeling of a continuous process can be designed as follow s.

Step 1.Select the length of training samples,N,and an appropriate

threshold,δ,given a pair of training datasets XNand YN,where XN∈ℜN×mand YN∈ℜN×l.

Step 2.Calculate the regression coefficient^βby Eq.(6).

Step 3.Predict the model output with^βand the newly available measurement X by

Step 4.If a new sample(s)is available,add the new sample(s)in to the training dataset and discard the oldest one(s)to com pose a new sampling dataset,go to Step 5;or return to Step 2.

Step 5.If Eq.(7)is satisfied,go to Step 2;or return to Step 3.

3.Modeling of a Hydro-isomerization Process

3.1.Hydro-isomerization process of C8-aromatics

Fig.1.Schematic diagram of the hydro-isomerization of C8-aromatics process.A301—reactor effluent air condenser;A302—deheptanizer air condenser;A303—recycle overhead air condenser;C301—recycle compressor;E301—reactor feed/effluent exchanger;F301—reactor feed heater;P301—isomerization feed pump;P302—deheptanizer feed pump; R301—isomerizaton reactor;T301—deheptanizer column;T302—recycle column;V301—separator d rum;V302—deheptanizer reflux drum;V303—recycle column reflux drum; V304—recycle compressor K.O.drum.

In the present work,an industrial hydro-isomerization process of C8-aromatics is studied,which is one of the important parts of PX join t process in a refinery.Its schematic diagram is shown in Fig.1. The raffinate(C8-aromatics with lean PX)from Eluxyl adsorption unit, together with the C8-naphtha and C8-paraffin(C8(N+P))fractions from recycle column T302 and hydrogen from recycle compressor C301,exchange heat in heat exchanger E301 with the effluent from reactor R301.Then the mixture is heated to its required temperature in heater F301 and introduced in to reactor R301,where a nonequilibrium mixture of C8-aromatics is converted in to a com position close to that in thermodynamic equilibrium.The reaction products are delivered to the separator d rum V301 after condensation in air condenser A301.The gas products from separate d rum V301 are utilized as recycle hydrogen while the liquid products are pumped into deheptanizer column T301.The components from the top of column T301,which are hydrocarbons lighter than heptane and C8(N+P), enter reflux drum V302 after condensation by air condenser A302, while a small amount of them flow s back to the top of column T301 and the rest is fed in to recycle column T302 for separation of light isomerate,C8(N+P).The light isomerate is pumped back to the isomerization reactor and the heavy isomerate from column T301 is fed to xylene fractionation unit for separation of heavy aromatics.The components from the top of the recycle column are condensed by condenser A303 and flow in to the reflux drum of recycle column T302.A small amount of them is delivered back to the top of T302,and the rest is pumped to the reforming stabilizer.

3.2.Selection of input and output variables

According to the kinetic analysis and industrial experience,many factors affect the isomerization process of C8-aromatics,such as reactor temperature(T),reactor pressure(Pt),weight hourly space velocity (WHSV),ratio of H2to hydrocarbons(CH2/HC),hyd rogen con ten t of recycle hydrogen(CH2),flow rate of feed(Q),and con ten t of PX,MX, OX,EB,C8(N+P),and other componen ts(denoted as A)in the feed [18].The partial pressu re of hydrogen p lays an important ro le in the isomerization reaction,expressed by

where PH2is the hydrogen partial pressure,MPa;Ptis the reactor pressure,MPa;CH2is the hydrogen content of recycle hydrogen,%; and CH2/HCis the ratio of H2to hydrocarbons,m o l·m ol−1.

The total feed of the reactor is com posed of two parts,the fresh feed from Eluxyl adsorption unit and the recycle feed from recycle column T302.Accordingly,the components of the total feed are the weighted results of the two parts

where yi,yniand yciare the con ten t of total feed,fresh feed and recycle feed,respectively,%;Qt,Qnand Qcare the flow rate of total feed,fresh feed,and recycle feed,respectively,kg·h−1.

Nine variables are selected as inputs of the model.The purpose of isomerization process is to obtain more PX,so the concentration of PX in the feed of deheptanizer column T301 is selected as the output. Accord ingly,the inputs and output of the model are determined and listed in Table 1.

3.3.Process simulation and discussion

3.3.1.Performance criteria for the model

In order to demonstrate the efficiency of SMW-PLS method, three performance indices are introduced for evaluation,i.e.,prediction accuracy,capability to track the process trend and running time(t).Twostatistical criteria,average relative error of prediction(AREP)and relative root mean square error(RMSE),are used to assess the prediction performance of the inferential model,which are

Table 1Input and output variables of the model

where Nsis the number of validation samples,ykand^ykare the measured and prediction values,respectively.RMSE is also a performance index to evaluate the capability to track the trend of evolving process. And the running time is a measure of the real-time performance of the model.

3.3.2.Model performance assessment

732 data samples were collected continuously from an industrial hyd ro-isomerization process every 8 h per day in 2005.The first 60 samples are used as training set and the rest as validation set.For comparison,PLS,moving window recursive PLS(MW-RPLS)and moving window PLS(MW-PLS)are simulated in add it ion to the proposed SMW-PLS.In these models,the number of latent variables is set as 6.

For PLS model,the first 60 samples are used to train the model.The parameters of the model are kept constant for the validation.Typical validation results are shown in Fig.2,where the predicted values are compared with the actual ones.The dashed and solid lines denote the measurements and model predictions,respectively.The relative prediction errors varying with time(represented by the number of collected samples)are also presented in Fig.2.For MW-RPLS model,the size of moving window(N)is set as 60.The oldest sample will be discarded once a new one is available.The model is updated through mean and variance of the data.The validation results are shown in Fig.3.For MW-PLS method,the size of moving window(N)is60.The first60 samples are used to train the model.When a new sample is introduced,the oldest sample will be omitted,and the model updating is activated by PLS.The simulation results are shown in Fig.4.As for the parameters in SMW-PLS model,their values are the same as those in MW-PLS model in add it ion to the threshold(δ)of 0.01.The model will be updated once the condition REk＞δis satisfied.The validation results are demonstrated in Fig.5.

Among the four PLS-based models,the prediction accuracy and tracking trend of the PLS are the worst.Other three models show better accuracy,with almost all the absolutely relative prediction errors smaller than2%.These results are attributed to the updating scheme.The intrinsic behavior of the process is reflected exactly by the dynamic models.

Fig.2.Comparison between actual values and those predicted by the PLS model.

Fig.3.Comparison between actual values and those predicted by the MW-RPLS model.

Fig.4.Comparison between actual values and those predicted by the MW-PLSmodel.

Fig.5.Comparison between actual values and predictions by the SMW-PLS model.

Table 2Comparison of the performance criteria between the models

The performance of these models can also be compared directly by ARPE,RMSE and t,as shown in Table 2.The precision of the PLS is worse than that of the other three models.Among the dynamic models, such asMW-RPLS,MW-PLSand SMW-PLS,the difference between their prediction errors is very slight except the case of δ=0.05 for SMW-PLS, while running time differs greatly.Especially,the running time for SMW-PLS is two orders of magnitude smaller than that of MW-PLS method.The proposed model reduces the running time greatly with a minor influence on the prediction accuracy.The advantage of the selective updating strategy is then presented.

3.3.3.Determination of model parameters

Three parameters,the size(length)of moving window,N,the number of latent variables,a,and the threshold,δ,need to be determined. The guidelines for appropriate values of these parameters are discussed below.

For the SMW-PLS soft sensor,the effects of moving window size(N) and the number of latent variables(a)on the prediction performance are shown in Fig.6.More training samples may lead to more robust performance for PLS-based models as more historical information of the process is integrated.However,the weight of new sample may be reduced with more training samples.Hence the behavior of model, such as prediction accuracy,tracking ability and real-time performance, may be degraded.The window size is determined to minimize the estimation error for the validation dataset.In Fig.6,better performance can be found with the window size ranging from 40 to 80.And the number of latent variables is optimized with the same manner.As observed from Fig.6,the estimation error differs slightly with then umber ranging from 2 to 8.

The effects of the two parameters on running time are investigated. The results are shown in Fig.7.More modeling samples and latent variables need more running time.In order to improve the real-time performance of the model,suitable window size and number of latent variables are suggested to be 60 and 6,respectively.

Fig.6.The AREP with different window sizes and numbers of latent variables.

Fig.7.The running time with different window sizes and numbers of latent variables.

The predefined threshold,δ,is a com promising index between prediction accuracy and running time.With smaller value ofδ,more new samples are introduced and model updating is activated more frequently.In other words,smaller δ gives more accurate prediction but longer running time.The effect of threshold on the model performance is discussed and comparisons of the results are shown in Table 2(with N=60 and a=6).We can conclude that the prediction accuracy of model changes a little as δ increases from 0.01 to 0.03,while the running time is reduced from 0.04 s to 0.0051 s.The running time for=0.03 is almost one order of magnitude smaller than that forδ=0.01.However, forδ＞0.03,the accuracy changes a lot,while the running time changes slightly.The SMW-PLS model presents its good real-time performance through regulating the threshold.In fact,determination of the threshold depends generally on the trade-off between prediction accuracy and running time.In practice,δis often chosen less than 0.05.

4.Conclusions

In this paper,a novel adaptive PLS-based model is proposed to deal with time-varying industrial process.Good performance of the SMWPLS method is shown by on-line prediction of PX content for a hydroisomerization process.Compared with other adaptive approaches,the running time of SMW-PLS model is decreased greatly by selective model adaptation.The strategy of selective updating will be activated when the prediction error is larger than the preset threshold.The proposed selective modeling scheme offers better adaptation performance and lower updating frequency than the previous methods.

[1]S.Wold,Multi-way principal components and PLS-analysis,J.Chem ometr.l(1) (1987)41-56.

[2]J.F.MacGregor,C.Jaeckle,C.Kiparissides,M.Koutoudi,Process monitoring and diagnosis by multiblock PLS methods,AIChEJ.40(5)(1994)826-838.

[3]P.Facco,F.Dop licher,F.Bezzo,M.Barolo,Moving average PLS soft sensor for on line product quality estimation in an industrial batch polymerization process,J.Process Control19(3)(2009)520-529.

[4]T.Komulainen,M.Sou rander,S.L.Jam sa-Jounela,An on line application of dynamic PLS to a dearom atization process,Com put.Chem.Eng.28(12)(2004)2611-2619.

[5]H.J.Galicia,Q.P.He,J.Wang,Comparison of the performance of a reduced-order dynamic PLS soft sensor with different updating schemes for digester control, Control.Eng.Pract.20(8)(2012)747-760.

[6]S.K.S.Fan,Y.J.Chang,Multiple-input multiple-output double exponentially weighted moving average controller using partial least squares,J.Process Control20(6)(2010) 734-742.

[7]J.L.Godoy,J.R.Vega,J.L.Marchetti,A fault detection and diagnosis technique for multivariate processes using a PLS-de composition of the measurement space, Chemom.Intell.Lab.Syst.128(2013)25-36.

[8]S.J.Qin,Survey on data-d riven industrial process monitoring and diagnosis,Annu. Rev.Control.36(2)(2012)220-234.

[9]K.Helland,H.E.Berntsen,O.S.Borgen,H.Martens,Recursive algorithm for partial least squares regression,Chemom.Intell.Lab.Syst.14(1-3)(1992)129-137.

[10]S.J.Qin,A recursive PLS algorithm for system identification,AIChEAnnual Meeting, St.Louis,USA,1993.

[11]S.J.Qin,Recursive PLS algorithms for adaptive data modeling,Com put.Chem.Eng.22 (4-5)(1998)503-514.

[12]J.Liu,D.S.Chen,J.F.Shen,Development of self-validating soft sensors using fast moving window partial least squares,Ind.Eng.Chem.Res.49(22)(2010)11530-11546.

[13]S.Kim,R.Okajim a,M.Kano,S.Hasebe,Development of soft-sensor using locally weighted PLS with adaptive similarity measure,Che mom.Intell.Lab.Syst.124(2013) 43-49.

[14]R.Liu,Some Studies on Soft Sensor Technology and Their Applications to Industry Process,(Ph.D.Thesis)Zhejiang University,Hangzhou,China,2004.(in Chinese).

[15]S.Mu,Y.Zeng,R.Liu,P.W u,H.Su,J.Chu,On line dual updating with recursive PLS model and its application in predicting crystal size of purified terephthalic acid (PTA)process,J.Process Control16(6)(2006)557-566.

[16]F.Ahm ed,S.Nazir,Y.Yeo,A recursive PLS-based soft sensor for prediction of the melt index during grade change operations in HDPE plants,Korean J.Chem.Eng. 26(1)(2009)14-20.

[17]Y.H.Lee,M.Kim,Y.H.Chu,C.Han,Adaptive multivariate regression modeling based on model performance assessment,Chem om.Intell.Lab.Syst.78(2005)63-73.

[18]O.Xu,X.Chen,Recu rsive PLS soft-sensor with moving window of fixed length and it commercial application,J.Chem.Eng.Chin.Univ.23(6)(2009)1044-1050 (in Chinese).

☆Supported by the National Natural Science Foundation of China(61203133, 61203072),and the Open Project Program of the State Key Laboratory of Industrial Control Technology(ICT1214).

*Corresponding author.

E-mailaddress:ogxu@zjut.edu.cn(O.Xu).