MaximumLikelihoodEstimationZhouYahongSchoolofEconomicsSHUFE
EstimationThreeAsymptoticallyequivalenttestproceduresThelinearmodelundernormalityPartIMaximumLikelihoodEstimationZhouYahongSHUFEMaximumLikelihoodEstimation
EstimationThreeAsymptoticallyequivalenttestproceduresThelinearmodelundernormality1Estimation2ThreeAsymptoticallyequivalenttestprocedures3ThelinearmodelundernormalityZhouYahongSHUFEMaximumLikelihoodEstimation
TomaximizeL=Πif(xi,θ)—likelihood—givenrealizedX,findvalueofθ,tomaximize∑nlnL=lnf(xi,θ)i=1Firstordercondition∂∑nlnL=∂∑nlnf(xi,θ)=g0∂θi=1∂i=θi=
Equivalently,tomaximize∑nlnL=lnf(xi,θ)i=1Firstordercondition∂∑nlnL=∂∑nlnf(xi,θ)=g0∂θi=1∂i=θi==Πif(xi,θ)—likelihood—givenrealizedX,findvalueofθ
Firstordercondition∂∑nlnL=∂∑nlnf(xi,θ)=g0∂θi=1∂i=θi==Πif(xi,θ)—likelihood—givenrealizedX,findvalueofθ,tomaximize∑nlnL=lnf(xi,θ)i=1ZhouYahongSHUFEMaximumLikelihoodEstimation
=Πif(xi,θ)—likelihood—givenrealizedX,findvalueofθ,tomaximize∑nlnL=lnf(xi,θ)i=1Firstordercondition∂∑nlnL=∂∑nlnf(xi,θ)=g0∂θi=1∂i=θi=1ZhouYahongSHUFEMaximumLikelihoodEstimation
—
—∑n0=∂lnf(xi,θˆ)∂θ∑i=1[]n=∂∑nlnf(xi,θ0)+∂2lnf(xi,θ0)(θˆ−θ0)i=1∂θ∂θ∂θ′i=1UsefulnessoftheFOC/Linearization—
Facilitatecomputation—∑n0=∂lnf(xi,θˆ)∂θ∑i=1[]n=∂∑nlnf(xi,θ0)+∂2lnf(xi,θ0)(θˆ−θ0)i=1∂θ∂θ∂θ′i=1UsefulnessoftheFOC/Linearization—
EstimationThreeAsymptoticallyequivalenttestproceduresThelinearmodelundernormalityEstimationByapproximation∑n0=∂lnf(xi,θˆ)∂θ∑i=1[]n=∂∑nlnf(xi,θ0)+∂2lnf(xi,θ0)(θˆ−θ0)i=1∂θ∂θ∂θ′i=1UsefulnessoftheFOC/Linearization——
EstimationThreeAsymptoticallyequivalenttestproceduresThelinearmodelundernormalityEstimationConsistency,asymptoticnormality,andefficiency,√[]−1n(θˆ−→dθ0)N−1(0,nI(θ0))=N(0,E−∂2lnf(xi,θ0))∂θ∂θ′where[][]I(θ)=E−∂2lnL=∂2lnf(xi,θ)nE−∂θ∂θ′∂θ∂θ′[]=∂lnf(xi,θ)∂lnf(xi,θ)nE∂θ∂θ′I(θ)istheinformationmatrixZhouYahongSHUFEMaximumLikelihoodEstimation
approachone∑[][]2lnf(xi,θˆ)(θˆ)Iˆ(θˆ)=1n∗−∂=−∂2Ln∂θ∂θ′∂θˆ∂θˆ′iapproachtwo[][n∑]n∗1∑Iˆ(θˆ)=n∗gˆigˆ′=gˆigˆ′niii=1i=1wheregˆi=g(xi,θˆ),whichisknownasBHHHestimatorandtheouterproductofgradients,
approachtwo[][n∑]n∗1∑Iˆ(θˆ)=n∗gˆigˆ′=gˆigˆ′niii=1i=1wheregˆi=g(xi,θˆ),whichisknownasBHHHestimatorandtheouterproductofgradients,∑[][]2lnf(xi,θˆ)(θˆ)Iˆ(θˆ)=1n∗−∂=−∂2Ln∂θ∂θ′∂θˆ∂θˆ′iZhouYahongSHUFEMaximumLikelihoodEstimation
EstimationThreeAsymptoticallyequivalenttestproceduresThelinearmodelundernormalityEstimationEstimatetheasymptoticVarianceoftheMLEapproachone∑[][]2lnf(xi,θˆ)(θˆ)Iˆ(θˆ)=1n∗−∂=−∂2Ln∂θ∂θ′∂θˆ∂θˆ′iapproachtwo[][n∑]n∗1∑Iˆ(θˆ)=n∗gˆigˆ′=gˆigˆ′niii=1i=1wheregˆi=g(xi,θˆ),whichisknownasBHHHestimatorandtheouterproductofgradients,
EstimationThreeAsymptoticallyequivalenttestproceduresThelinearmodelundernormality1Estimation2ThreeAsymptoticallyequivalenttestprocedures3ThelinearmodelundernormalityZhouYahongSHUFEMaximumLikelihoodEstimation
Likelihoodrationtest:LR=2(lnL(θˆ)−lnL(θ¯)),whichasymptoticallyχ2JunderthenullhypothesisTheWaldtest:W=(c(θˆ)−q)′(Var(c(θˆ)−q))−1(c(θˆ)−q)=∂c(θˆ)∂′c(θˆ)(c(θˆ)−q)′(∂θ′(θˆ)∂θ)−1(c(θˆ)−q)TheL(agrang)eMultiplie(rTest–)Rao’sScoretest:LM=∂L(θˆ)1∂′L(θˆ)∂θ′[I(θˆ)]−∂θ.EstimationThreeAsymptoticallyequivalenttestproceduresThelinearmodelundernormalityThreeAsymptoticallyequivalenttestproceduresThenullhypothesisH0:c(θ)=θˆdenotetheMLEandθ¯
TheWaldtest:W=(c(θˆ)−q)′(Var(c(θˆ)−q))−1(c(θˆ)−q)=∂c(θˆ)∂′c(θˆ)(c(θˆ)−q)′(∂θ′(θˆ)∂θ)−1(c(θˆ)−q)TheL(agrang)eMultiplie(rTest–)Rao’sScoretest:LM=∂L(θˆ)1∂′L(θˆ)∂θ′[I(θˆ)]−∂θ.EstimationThreeAsymptoticallyequivalenttestproceduresThelinearmodelundernormalityThreeAsymptoticallyequivalenttestproceduresThenullhypothesisH0:c(θ)=θˆdenotetheMLEandθ¯:LR=2(lnL(θˆ)−lnL(θ¯)),whichasymptoticallyχ2JunderthenullhypothesisZhouYahongSHUFEMaximumLikelihoodEstimation
TheL(agrang)eMultiplie(rTest–)Rao’sScoretest:LM=∂L(θˆ)1∂′L(θˆ)∂θ′[I(θˆ)]−∂θ.EstimationThreeAsymptoticallyequivalenttestproceduresThelinearmodelundernormalityThreeAsymptoticallyequivalenttestproceduresThenullhypothesisH0:c(θ)=θˆdenotetheMLEandθ¯:LR=2(lnL(θˆ)−lnL(θ¯)),whichasymptoticallyχ2JunderthenullhypothesisTheWaldtest:W=(c(θˆ)−q)′(Var(c(θˆ)−q))−1(c(θˆ)−q)=∂c(θˆ)∂′c(θˆ)(c(θˆ)−q)′(∂θ′(θˆ)∂θ)−1(c(θˆ)−q)ZhouYahongSHUFEMaximumLikelihoodEstimation
EstimationThreeAsymptoticallyequivalenttestproceduresThelinearmodelundernormalityThreeAsymptoticallyequivalenttestproceduresThenullhypothesisH0:c(θ)=θˆdenotetheMLEandθ¯:LR=2(lnL(θˆ)−lnL(θ¯)),whichasymptoticallyχ2JunderthenullhypothesisTheWaldtest:W=(c(θˆ)−q)′(Var(c(θˆ)−q))−1(c(θˆ)−q)=∂c(θˆ)∂′c(θˆ)(c(θˆ)−q)′(∂θ′(θˆ)∂θ)−1(c(θˆ)−q)TheL(agrang)eMultiplie(rTest–)Rao’sScoretest:LM=∂L(θˆ)1∂′L(θˆ)∂θ′[I(θˆ)]−∂θ.ZhouYahongSHUFEMaximumLikelihoodEstimation
EstimationThreeAsymptoticallyequivalenttestproceduresThelinearmodelundernormality1Estimation2ThreeAsymptoticallyequivalenttestprocedures3ThelinearmodelundernormalityZhouYahongSHUFEMaximumLikelihoodEstimation
Thedensityforεi—–f(εi)=√1exp{−ε2/2σ2}.2piσ2ilogLikelihoodlnL(θ)=−n∑nln(2pi)−nlnσ2−[1/2σ2](yi−x′β2)22ii=1=−nn2ln(2pi)−lnσ−[1/2σ2′](y−Xβ)(y−Xβ)22EstimationThreeAsymptoticallyequivalenttestproceduresThelinearmodelundernormalityThelinearmodelundernormalityThemodelyi=x′β+εiiZhouYahongSHUFEMaximumLikelihoodEstimation
logLikelihood∑nlnL(θ)=−nln(2pi)−nlnσ2−[1/2σ2](yi−x′β2)22ii=1=−nn2ln(2pi)−lnσ−[1/2σ2′](y−Xβ)(y−Xβ)22EstimationThreeAsymptoticallyequivalenttestproceduresThelinearmodelundernormalityThelinearmodelundernormalityThemodelyi=x′β+εiiThedensityforεi—–f(εi)=√1exp{−ε2/2σ2}.2piσ2iZhouYahongSHUFEMaximumLikelihoodEstimation
EstimationThreeAsymptoticallyequivalenttestproceduresThelinearmodelundernormalityThelinearmodelundernormalityThemodelyi=x′β+εiiThedensityforεi—–f(εi)=√1exp{−ε2/2σ2}.2piσ2ilogLikelihoodlnL(θ)=−n∑nln(2pi)−nlnσ2−[1/2σ2](yi−x′β2)22ii=1=−nn2ln(2pi)−lnσ−[1/2σ2′](y−Xβ)(y−Xβ)22ZhouYahongSHUFEMaximumLikelihoodEstimation
ThusβˆML=(X′X−1)X′y=bandσˆM2L=e′enEstimationThreeAsymptoticallyequivalenttestproceduresThelinearmodelundernormalityThelinearmodelundernormalityThefirstorderconditions∂lnL=1X′(y−Xβ)=0∂βσ2and∂lnL=−n+1′(y−Xβ)(y−Xβ)=0∂σ22σ22σ4ZhouYahongSHUFEMaximumLikelihoodEstimation
EstimationThreeAsymptoticallyequivalenttestproceduresThelinearmodelundernormalityThelinearmodelundernormalityThefirstorderconditions∂lnL=1X′(y−Xβ)=0∂βσ2and∂lnL=−n+1′(y−Xβ)(y−Xβ)=0∂σ22σ22σ4ThusβˆML=(X′X−1)X′y=bandσˆM2L=e′enZhouYahongSHUFEMaximumLikelihoodEstimation
EstimationThreeAsymptoticallyequivalenttestproceduresThelinearmodelundernormalityThelinearmodelundernormality[]∣∂2lnL/∂β∂β′∂2lnL/∂β∂σ2∣∣∂2′lnL/∂σ2∂β∂2lnL/∂(σ2)2∣[]θˆ=−(1/σ2)X′X−(1/σ4)X′ε−(1/σ4)ε′Xn/(2σ4)−ε′ε/σ6hence[]−1X)−10[I(β,σ2)]=σ2(X′02σ4/nZhouYahongSHUFEMaximumLikelihoodEstimation
