LS,LAD组合损失的高维统计性质分析

张凌洁,苏美红,张海

(西北大学数学系,陕西西安 710127)

LS,LAD组合损失的高维统计性质分析

张凌洁,苏美红,张海

(西北大学数学系,陕西西安 710127)

线性模型;高维;稳健估计;罚稳健估计;LS+LAD的凸组合

DO I：10.3969/j.issn.1008-5513.2013.05.014

1 引言

研究与应用的阶段,将稳健统计数据扩展到其他评估和测试的问题,同时建立稳健估计的渐近理论,并讲述有关稳健性的相关知识;文献[6]对文献[4]提出的案例(a)lim sup p＜∞做了分析;文献[7]给出M估计中每个估计量的渐近有效性;文献[8]介绍“一步法”的Huber(M)估计线性模型;文献[9-11]给出ˆβ的一致正态渐近分布;文献[12]提出多参数线性模型M估计的渐近性和一致性;文献[13-15]在一般损失函数下给出高维稳健估计和高维罚稳健估计求解‖ˆβ‖(‖ˆβ-β0‖)的方程组,并对如何适当选择损失函数的问题做以分析.

经典地,通常研究p固定或p/n→0(观测数n→∞比预测数p→∞的速度快)的情况,对于噪声服从正态分布,最小二乘LS是优的,而关于损失函数ρ是双指数分布,最小绝对偏差LAD是优的.

估计未知参数β时,当ε的分布是已知的(如正态的,均匀的,Weibull的等),通常采用最大似然估计法来估计未知参数.若ε的分布是未知的,通常采用LS﹑M inMax(MM)和LAD等作估计.如果误差是正态分布,LS和最大似然估计是相同的,但是在响应变量和解释变量中, LS却受离群值的影响.对响应变量LAD是稳健的,但LAD对于解缺乏唯一性.近几年提出组合的方法是为了处理不确定性模型选择的问题,该方法不仅节省了计算时间,提高了估计精度,而且在不确定性模型选择时,也给出了较好的估计量.比如组合的方法可以改善回归的性能问题[16];用于稳固和收缩系数估计的组合方法能提高预测[17];用回归函数的参数和非参数的组合回归估计时,组合估计量优于核估计量[18].

为了减弱LS受离群值的影响和LAD对解缺乏唯一性,用LS+LAD的凸组合形式[19-20],即其中0≤δ≤1.显然,当δ=0时,模型为LAD估计,当δ=1时,模型为LS估计.适当选择δ是为了得到未知参数的最小渐近方差.组合模型允许组合一些已有模型来估算误差,对已有模型的估计进行改善,使其具有更多的性质:使不确定性模型的选择有了依据,节省了计算时间﹑提高了预测精度和估计的收敛率.特别,组合模型解决了解缺乏唯一性的问题.

然而,损失函数是LS+LAD凸组合形式的高维性质还不清楚.本文主要是在高维背景(观测数n和预测数p均趋于无穷大,即下,对LS+LAD的高维稳健性质(p＜n)和高维罚稳健性质(p≫n)作以分析,性质分析中主要运用了prox函数和Stein′s identity[14],得到了稳健估计和罚稳健估计的显示表达,结果显示这种凸组合损失函数模型集成了LS和LAD损失的优点,同时消弱了它们的不足,具有优良的高维统计性质.

2 LS+LAD高维稳健性质分析(p＜n)

3 LS+LAD高维罚稳健性质分析(p≫n)

4 结论

在高维稳健回归中,LS估计和LAD估计已有相对完善的理论结果,但是它们还存在一定的问题.LS在响应变量和解释变量中受离群值的影响;LAD在解释变量中受离群值的影响,同时还对解缺乏唯一性.

本文主要针对损失函数为LS+LAD的凸组合形式,研究了高维背景(观测数n和预测数p均趋于无穷大,即

ˆ运用了prox函数和Stein′s identity,得到了凸组合损失下高维稳健估计‖β‖和高维罚稳健回归估计‖βˆ-β0‖的显示表达,结果表明这种凸组合损失函数模型集成了LS和LAD损失的优点,同时消弱了它们的不足,具有优良的高维统计性质.

[1]陈希孺,赵林城.线性模型中的M方法[M].上海:上海科学出版社,1996.

[2]Huber P J.Robust estimation of a location parameter[J].Ann.Statist.,1964,35,73-101.

[3]Huber P J.The 1972Wald lecture.Robust statistics:A review[J].Ann.Statist.,1972,43:1041-1067.

[4]Huber P J.Robust regression:asym p totics,con jectures and M onte Carlo[J].Ann.Statist.,1973,1:799-821.

[5]Huber P J,Ronchetti E M.Robust Statistics[M].2nd ed.Hoboken,NJ:John W iley and Sons Inc,2009.

[6]Relles D.Robust Regression by M odifed Least Squares[C].New Haven:Ph.D.Thesis,Yale University,1968.

[7]Yohai V J.Robust Estimation in the Linear M odel[C].New Haven:Ph.D.Thesis,Yale University,1974.

[8]Bickel P J.One-step Huber estimates in the linearmodel[J].J.Amer.Statist.Assoc.,1975,70:428-434.

[9]Portnoy S.Asym p totic behavior of M-estim ators of p regression param eters w hen p2/n is large[J].Ann. Statist.,1984,12:1298-1309.

[10]Portnoy S.Asym p totic behavior of M estimators of p regression parameters when p2/n is large[J].Ann. Statist.,1985,13:1403-1417.

[11]Le Cam L.On the assum ptions used to prove asym ptotic norm ality of m aximum likelihood estim ates[J]. Ann.Statistics,1970,41:802-828.

[12]M ammen E.Asym ptotics w ith increasing dimension for robust regression w ith app lications to the bootstrap[J].Ann.Statist.,1989,17:382-400.

[13]Nou redd ine E l Karoui,Derek Bean,Peter B ickel,et al.On robust regression w ith high-d im ensional p red ictors[J].Proc.Natl.Acad.Sci.USA,2013,110(36):14557-14562.

[14]Derek Bean,Peter Bickel,Noureddine El Karoui,et al.Penalized Robust Regression in High-Dimension[C]. Berkeley:Technical Reports of Departm en t of Statistics University of California,2011.

[15]Derek Bean,Peter Bickel,Noureddine El Karoui,et al.Op timal Ob jective Function in High-Dimensional regression[C].Berkeley:Technical Reports of Department of Statistics University of California,2011.

[16]Sculley D.Combined regression and ranking[J].Washington,DC,USA,K.D.D.,2010,10:25-28.

[17]W ei X iaoqiao.regression-based forecast com bination m ethods[J].Rom anian Jou rnal for Econom ic Forecasting,2009(4):5-18.

[18]Fan Yanqin.Asym p totic Normality of a combined regression estimator[J].Journal ofM u ltivariate Analysis, 1999,71:191-240.

[19]Enrique Castillo,Carm en Castillo,A li S Had i,et al.Combined regression m od les[J].Com pu t.Stat., 2009,24:37-66.

[20]Rosasco E,De V ito A,Caponnetto M,et al.Are loss functions all the same?[J].Neural.Com put., 2003,16(5):1063-1076.

[21]Akaike H.In formation Theory and an Extension of the Mmaximum Likelihood Princip le[D].Budapest: Akadem iai K iado,1973.

[22]Schwarz G.Estimating the dimension of amodel[J].Ann Stat.,1978,6:461-464.

[23]Xu Zongben,Zhang Hai,W ang Yao,et al.L1/2Regu larization[J].Sci.China In f.Sci.,2010,53:1159-1169.

[24]Zhang Hai,Liang Yong,Gou HaiLiang,et al.The essential ability of sparse reconstruction of dif erent com p ressive sensing strategies[J].Sci.China In f.Sci.,2012,55:2582-2589.

[25]Xu Zongben,Chang X iangyu,Xu Fengm in,et al.L1/2Regu larization:a th reshold ing rep resentation theory and a fast solver[J].Neu ral Networks and Learning System s.2012,23(7):1013-1027.

[26]Tibshirani R.Regression shrinkage and selection via the Lasso[J].Journal of the Royal Statistical Society Series,1996,58(1):267-288.

[27]M allow s C L.Som e comm ents on Cp[J].Technom etrics,1973,15:661-675.

[28]Peter Buhlmann Sara van de Geer.Statistics for High-Dimensional Data[M].New York:Springer,2010.

[29]Fan JQ,LiR Z.Variab le selection via nonconcave penalized likelihood and its oracle p roperties[J].Journal of the American Statistical Association,2001,31:1348-1360.

The statistical analysis of the com bined loss of
LS,LAD in h igh-d im ension

Zhang Lingjie,Su Meihong,Zhang Hai

(Department of Mathematics,Northwest University,X i′an 710127,China)

This article studies a convex combination of the Least Squares(LS)and Least Absolute Deviation(LAD).By studying the robust statistical properties of high-dimensional and penalized robust statistical p roperties of high d im ension when the number of observations n and the num ber of p rediction p tends to inf nitythe exp ressions of robust estim ation and penalized robust estim ation are obtained.The

result reveals that the loss function model of convex combination combines the advantages of the LSand LAD, at the same time,it relatively weakens their shortcom ings,thus it has excellent high dimensional statistical p roperties.

linearmodel,high dimension,robust estimation,penalized robust estimation, convex combination of LS+LAD

O23

A

1008-5513(2013)05-0536-08

2013-05-16.

国家自然科学基金(11171272).

张凌洁(1986-),硕士生,研究方向:机器学习.

2010 MSC:94A 15