DataMiningandKnowledgeDiscovery,7,321–339,2003©∗SAHARONROSSETsaharonr@@@@.,Ra’anana,43000IsraelEditors:Hand,Keim,NgReceivedOctober1,,inparticularcustomervalueandlengthofservice(ortenure)modeling,andpresentanovelsegment-basedapproach,,(BI):lifetimevalue,lengthofservice,churnmodeling,-pectfromacustomer(Novo,2001).Theexactmathematicaldefinitionanditscalculationmethoddependonmanyfactors,suchaswhethercustomersare“subscribers”(asinmosttelecommunicationsproducts)or“visitors”(asindirectmarketingore-business).InthispaperwediscussthecalculationandbusinessusesofCustomerLifetimeValue(LTV)inthecommunicationindustry,,whichareahighpriorityofAmdocs’customersinthecommunicationindustry:churnandretentionanalysis,fraudanalysis(MuradandPinkas,1999;Rossetetal.,1999),campaignmanagement(Rossetetal.,2001),,,theLTVofacustomerorasegmentisimportantcomplementaryinformationtotheirchurnprobability,asitgivesasenseofhowmuchisreallybeinglostduetochurnandhowmucheffortshouldbe∗AlsoofStanfordUniversity,DepartmentofStatistics.
,,anLTVmodelhasthreecomponents:customer’svalueovertime,customer’,thereisanadditionalissue,whichistheneedtocalculateacustomer’,wewouldneedtocalculateseveralLTV’sforeachcustomerorsegment,correspondingtoeachpossibleretentioncampaignwemaywanttorun().BeingabletoestimatethesedifferentLTV’:,withsomeex-amples,,therearethreefactorswehavetodetermineinordertocalculateLTV:’svalueovertime:v(t)fort≥0,wheretistimeandt=,thecustomer’sfuturevaluehastobeestimatedfromcurrentdata,(LOS)model,describingthecustomer’“survival”functionS(t)fort≥0,(t)asthecustomer’s“instantaneous”probabilityofchurnattimet:f(t)=−dS/dtThequantitymostcommonlymodeled,howeveristhehazardfunctionh(t)=f(t)/S(t).HelsenandSchmittlein(1993)discusswhyh(t)isamoreappropriatequantitytoestimatethanf(t).(t),whichdescribeshowmucheach$:Exponentialdecay:D(t)=exp(−at)forsomea≥0(a=0meansnodiscounting)Thresholdfunction:D(t)=I(t≤T)forsomeT>0(whereIistheindicatorfunction).Giventhesethreecomponents,wecanwritetheexplicitformulaforacustomer’sLTVasfollows:∫∞LTV=S(t)v(t)D(t)dt()0
CUSTOMERLIFETIMEVALUEMODELS323Inotherwords,-forward,theessenceofthechallengelies,ofcourse,inestimatingthev(t)andS(t),forLOSwecanuseahighlysimplisticmodelassumingconstanttchurnrate—soifweobserve5%churnrateinthecurrentmonth,wecansetS(t)=—acustomer’sindividualcharacteristics,contractsandcommitments,,,eitherbecauseitismodeling“local”effectsrelevantforthepresentonlyandnotforthefuture,“goldenpath”-basedapproach,,withcovariatesvectorsx,...,xrepresentingtheir“current”stateandchurnindicators1Nc,...,’tenurewiththecompanyisanimportantchurnpredictorsince1NLOSfrequentlyshowsastrongdependencyoncustomer“age”,,...,,usagehistory,paymenthistory,-dependentaccumulatedattributes(,trends).Ourdiscussionisgoingtoviewtimeasdiscrete(measuredinmonths),andthusthet’swillbeintegersandf(t)willbeaprobabilityfunction,,,1999,(t)hasaparametricform(Exponential,Weibulletc.)withtheparametersdependingonthecovariates,.(1999)mention,suchapproachesaregenerallynotappropriateforLTVmodeling,sincethesurvivalfunctiontendstobe“spiky”andnon-smooth,withspikesatthecontractenddates.
-parametricapproaches,suchastheCoxproportionalhazards(PH)model(Cox,1972),(t)oftheform:′h(t)=f(t)/S(t)=λ(t)expβx()iiiioralternatively:′log(h(t))=log(λ(t))+βx()iiSothereisafixedparametriclineareffect(intheexponent)forallcovariates,excepttime,whichisaccountedforinthetime-varying“baseline”riskλ(t).Adifferentsemi-parametricapproachistakenbyManietal.(1999),whobuildaNeuralNetworksemi-parametricmodel,whereeachpossibletenurethasitsownoutputnode(thetenureisdiscretizedtothemonthlylevel).,makesLOSmodelingaspecialcaseofsurvivalanalysiswhereeachsubjectisobservedonlyonceintime,andcustomerswhodisconnectedbeforethismonthare“leftcensored”.Consequentlywecanapproachiteitherasasurvivalanalysisproblemorastandardsupervisedlearningproblemwherethetime(’stenurewiththecompany)“baselinehazard”effect,timecanbetreatedasbeingcategoricalratherthannumerical,,,considerthatacustomer’schurnriskisinfacthish(t)value(sinceifthecustomeralreadyleftwewouldnotobservehim).Thusamodeloftheform:′log(P(c=1))=α(t)+βx()iiiisobviouslyequivalentto().TheKaplan-Meierestimator(KaplanandMeier,1958)offersafullynon-parametricestimateforS(t)byaveragingoverthedata:∑I(t≥t)iiS∑(t)=()I(t≥t)+CitiWhere∑•I(t≥t)isthenumberofcustomerswhosetenureisatleasttmonthsii•“leftcensored”,whicharetypicallyusedforLTVcalculations.
CUSTOMERLIFETIMEVALUEMODELS325TheKaplan-Meierapproachisobviouslyinappropriateforourpurpose,-basedapproach,presentednext,,“segment”,representingasetofcustomerswhoaretobetreatedasoneunitforthepurposeofplanning,“homogeneous”inthesensethatthecustomersinitare“similar”,atleastforthepropertyexamined(),examining,:–themarketinganalystisinterestedinexaminingsegments,notindividualcustomers–thesesegmentshavebeenpre-definedusingAmdocsCMSorsomeothertool–theyare“homogeneous”intermsofchurn(andhenceLOS)behavior–theyarereasonablylargeBasedupontheseassumptions,(sinceallcustomerswithinthesegmentaresimilar)-Meierapproachisreasonablehere,(potentiallydistant)past,andsomaynotrepresentthecurrenttendenciesinthissegment,whichmaywellberelatedtorecenttrendsinthemarket,-parametricestimateofthehazardsrate∑I(t=t)I(c=1)iiih∑(t)=()I(t=t)ii∑WhereI(t=t)I(c=1)(usuallytakeninmonths),(t)throughthesimplecalculation:∏∏∏S(u+1)S(u)−f(u)S(t)===(1−h(u))()S(u)S(u)u<tu<tu<tWhereS(0)=1,ofcourse.
’LTVplatform,,wegenerallyhavetoconsidertwostatisticalconcepts:–Bias/Consistency:ifwehadinfinitedata,wouldourestimateconvergetothecorrectvalue?Howfarwoulditendupbeing?–Variance:howmuchuncertaintydowehaveintheestimateswearecalculatingfortheunknownvalue?Theseconceptshaveconcretemathematicaldefinitionsforthecaseofsquarederrorlossregressiononly(althoughmanysuggestionsexistforgeneralizedformulationsforothercases—see,forexample—Friedman,1997).Howevertheprinciplestheydescribeapplytoanyproblem:–Themoreflexibleand/oradequatethemodelis,thesmallerthebias.–Themoredataonehas,andthemoreefficientlyoneusesit,-homogeneityassumptionmentionedpreviously,,evenwithoutthisassump-tion,ifweassumethatthemarketingexpertplanningthecampaignisonlyinterestedinthesegmentasawhole,,“large”(asareindeedmostreal-lifesegmentsencounteredinthecommunicationindustry),andthatthereisareasonableamountofchurnineachsegment,wecansafelyassumethatthesegmentbasednon-parametricestimateswillalsohavelowvariance,’scurrentvalueisusuallyastraightforwardcalculationbasedonthecustomer’scurrentorrecentinformation:usage,priceplan,payments,collectionefforts,callcentercontacts,,:seasonality,businesscycles,economicsituation,competitors,personalprofilesandmore,,whileeitherleavingthewholevalueissuetotheexperts(Manietal.,1999),orconsideringcustomers’currentvalueastheirfuturevalue(Novo,2001).Workingatthesegmentlevelalsomakesthevaluecalculationtaskeasier,sinceitimplieswedonotneedtohaveanexactestimateofindividualcustomers’futurevalue,butcan
,’BIplatformsystemsistheChurnManagementSystem(CMS).Thekeyoutputsofthesystemarechurnandloyalsegments,aswellasscoresforeachindividualinthetargetpopulation,whichrepresenttheindividual’’profileandbehaviorchanges:customerdata,usagesummaries,billingdata,accountsreceivableinformation,,,-maticandinteractivetoolswhichtheCMSutilizestodiscoverandanalyzepatterns,andtoperformpredictivemodeling,haveproventobehighlysuccessfulwhencomparedtothestateoftheartdata-miningtechniques(RossetandInger,2000;Ingeretal.,2000;Neumannetal.,2000).’sanalysistool’,-tenthechurnrateinthepopulationisverysmallbutinthesamplethetwoclasses(churnandloyal)-ulationisaccountedforintheLOSmodel,.,,,theproportionofchurn-tersforeachtenuretasdefinedinthefollowingformula(thisis(),adjustedforsampling)∑I(t=t)I(c=1)iiip=∑∑()tfactor·I(t=t)I(c=0)+I(t=t)I(c=1)iiiiii
=(churntoloyalsampleratio)/(churntoloyalpopulationratio).Now,givenacustomerwhoiscurrentlyattenuret,wecanuse()togettheS(t)—theprobability0ofacustomertoreachtenuret.()S(t)=(1−p)×(1−p)×···×1−p()t−1t−2t0AndthenwecangettheexpectedLOSasfollows:h∑ELOS=S(t)()t=0wherehisthehorizon,,inadditiontothisthresholdapproachisplannedforthefuture,,anexistingattributewithinthedata-mart,orafunctionofseveralexistingattributes(-totalcosts).’sage(tenure)field,enterthehorizonandenterthefullpopulationchurnrate(thesamplechurnrateisalreadyderived).Itisalsonecessarytoselect/definethecustomervalue,whichmaybeoneofthreeoptions:anequalvalueforallcustomers,afieldthatwaspreviouslyselectedasthevalue(inthiscasethe
.“averagebill”waspreviouslyselected),:∑LTV=ratio×ELOSvc()jjjj∈segmentv,carethevalueandchurnindicatorforthej-thcustomer,respectively,,Loyalsegments(“Class:Stay”)havehigherLTVsthanChurnsegments,,whichaimmainlyatincreasingLOS(asecondarygoalisincreasingvalue).:modelingtheeffectsofacompany’sactionsonitscustomers’:Company“A”hasidentifiedasegmentof“Citydwellingprofessionals”,(,reducedpricehandsetupgradeetc.),onthe
’svalueandLOS,,differentacceptanceratebycustomers,,(orestimate)beforewecancalculatetheireffectonLTV:•(,letter,commentonwrittenbill).Denotethesuggestion(orcontactchannel)costbyC.•Thecostincurredifthecustomeracceptstheincentive().•Theprobabilitythatacustomerintheapproachedsegmentwillagreetoaccepttheincentive(whichcanbearound100%iftheincentiveiscompletelyfree,butthatisrarelythecase).Thisquantityhastobeestimatedfrompastexperience,orsimplyguessed(inwhichcasedifferentvaluesforitcanbetried,toseehoweachwouldaffecttheoutcome).DenotetheacceptanceprobabilitybyP•,iftheincentiveisfreevoice-mail,thecustomer’(i)byv(t).•Theeffectoncustomer’((i)leavethecompanyinthenextXmonths).DenotethenewsurvivalfunctionbyS(t).Givenallofthese,calculatingtheexpectedchangeinLTVofacustomerfromsuggestinganincentiveisastraightforwardROIcalculation:(∫)∞[](i)(i)(i)LTV−LTV=P·S(t)v(t)−S(t)v(t)D(t)dt−G−C0AsforthebasicLTVcalculationdescribedinSection2,andevenmoreso,themainchallengeisinobtainingreasonableandusableestimatesfortheabovequantities,inparticularthe(i)(i)functionsv,:onethatbuildsonoursegment-levelLTVcalculationapproachpresentedabove,andanotherthatmakesfurthersimplifyingassumptions,-levelcalculationAswasmentionedbefore,workingatthesegmentlevelallowsustoaverageourinformationoverthewholesegmentandavoidparametricassumptions,whileassumingthatthesegmentis“homogeneous”.
’sLTV,weneedtodescribehowwechangetheLOSmodelpersegment,:(usuallywithapenaltyforcommitmentviolationthatmakesitunprofitabletoleaveduringthisperiod),,,toestimatepost-incentiveLOSforaspecificsegmentandaspecificincentive,weneedtoknow:(i)–Commitmentperiodincludedinincentive,denotebycmt(i)–Reductioninchurnprobabilityfromincentive,denotebyrcWhichgivesusforaspecificcustomer:t∏()()()(i)(i)(i)(i)S(t)=It<cmt+It≥cmt1−c(a+u)rc(i)u=cmtwhereaisthecustomer’scurrent“age”,andc(a+u)isthechurnprobabilityestimateforagea+()and()nowgivesusapost-incentiveexpectedLOSestimateof:Nht∑∑∏()1(i)(i)(i)ELOS=cmt+1−c(a+u)rcN(i)(i)j=1t=cmtu=,andweareassumingagainthatthe“age”(includingage),,customerswill“onaverage”returntothechurnbehaviorthatwouldcharacterizethemattheir“age”havetheynotchurnedforotherreasons(ratherthanthecommitmentfromtheincentive).Theincentive’’,whenofferingafreevoicemailincentivethereducedvaluewouldbethevoicemailcostandtheincreasedvaluedwouldbederivedfromtheincreaseinbilledincomingcallsandtheincreaseinoutgoingcallsduetothecustomer’,weget
332ROSSETETAL.(i)(i)(i)thatforeverycustomer:v=v(1+change),wherechangeisthepercentagechangeinvalueduetotheincentive,:()N∑[]1(i)(i)(i)avLTV−avLTV=P·ELOSv(j)−ELOSv(j)−G−CNj=1(i):(t),S(t)=(1−p),wherepisthecustomer’,v(t)=v,andisnotaffectedbytheincentive’:h−1h−1∑∑t∼LTV=v(1−p)=v(1−pt)=vh(1−p(h−1)/2)oldt=0t=:LTV=P(hv−G)+(1−P)LTV−:h(h−1)∼LTV−LTV=Pvp−PG−Cnewold2which,givenP,GandCandignoringtheinaccuracyinourcalculationgivesustheelegantresultthat:LTV−LTV>0⇔vp>2(PG+C)/(Ph(h−1))newold
,wegettheintuitiveconclusion,thatifwehaveareasonablemodelforvandp,,’thaveacaller-idfeature,whosehandsetwasnotupgradedinthepastyearandwhohaverecentlychangedtheirpaymentmethod(forexamplefromdirectdebittocheck).Amarketinganalystcameupwithtwopossibleincentivesforthissegment:anupgradeatadiscountedprice(letsassumeforsimplicitythatallwillbeofferedthesamenewhandset),thatthefieldselectedasvaluewasthemonthlyaveragebill,theselectedhorizonis12monthsandthepopulationchurnrateis5%(inthesampleit’sabout50%).TheLTVofthissegmentis$4,967,,,(inthisexampleweused$100and$10respectively).Ontheotherhand,theacceptanceratewillbehighersinceit’samoreattractiveoffer(caller-id—10%ofthechurnersand20%oftheloyals,upgrade—20%ofthechurnersand30%oftheloyals).Actually,,,amoresophisticatedhandsetwillprobablyincreasetheusageandthustheaddedvalue,whileaddingacaller-idwillhaveverylittleornoimpactontheusage(therelativevalueincreasefortheupgradeis10%inthisexampleandnoneforthecaller-id).,loyalcustomerswillalsobecommittedto12moremonths,soeventhoughtheyweren’tabouttochurninthenext
:definition(left)andallocation(right).$2,413,338andduetoofferingafreecaller-idis$1,982,,,,offeringanincentivetothissegmentisdonealsotoincreasetheusage/$29,091,’,theestimatedchangeinLTVduetotheupgradeincentivewasnegative:-$485,,thecaller-idincentiveyieldedanestimatedLTVincreaseof$1,422,,.
CUSTOMERLIFETIMEVALUEMODELS335incentivesondifferentsegmentscannotbeeasilyguessedevenwhenalltheincentive’’smechanismforestimatingthatimpact,itispossibletofittheappropriateincentive(outofthegivenoptions)-basedLTVmodelsforotherproblemsWehavedescribedthesegment-basedapproachtoestimatingthecurrentLTVofacustomer(Section2)andtoestimatingtheeffectonLTVofretentionbyincentiveinthecommunica-tionindustry(Section4).,-sell(addingnewproductstothecustomer)andup-sell(addingservicestoexistingproducts)-marketingactivityLTVcalculationisexactlythesame,-campaignLTVestimation(Section4)isvalidheretoo:–thecostsofrunningthecampaign–thecostsassociatedwiththecustomeracceptingtheoffer(inthecaseofcross/up-sellthecostofthepromotionaloffermade,ifany)–theprobabilityofacceptancebythecustomer–theeffectofacceptanceoncustomerLOS–-sellcampaignsaremuchlesslikelytoincludecommitments(andhenceaffecttheLOS)andmuchmorelikelytoaffectcustomerusagepatterns(andhencecustomervalue).Inaddition,theofferedproductorserviceisusuallyavailabletothegeneralpublic,,,thesegment-basedapproachwediscussed(anditsimplementation)canbeapplied,almostas-is,totheproblemofestimatingtheeffectofcross-sellcampaignsoncustomerLTV,-telecommarketingproblems,letusconsiderdirectmarketingcampaigns,-Cup1998,
(2000),wehavedescribedseveralsolutionstothisproblem,amongthemasegment-basedsolution,selecting11“mostprofitable”,thismodelwaslessaccurateonthisdatasetthanthemodelswhichtargetindividualcustomersaccordingtopropensityscores(seeSection7below),howeverithastheadvantageofbeinghighlyinterpretable,“marketingparadigm”,essentially,asgoodasthequalityofthesegmentation—ifthesegmentsarelarge,homogeneousandstatisticallycorrelatedwiththepropertybeingmodeled,“lifetimevalue”totheoutcomeofthecurrentcampaignonly,ignoringissuessuchasloyaltyandreferrals(asdirectmarketersusuallydo),thenourproblemisessentiallyreducedtopredicting,atthesegmentlevel,thetotalexpecteddonationinthiscampaign,,“delinquencycycle”.Letusconcentrateonalimitedexample,ofdecidingwhethertoreducethecreditlimitofcustomerswhoarelikelytobecomedelinquent(accumulatedebt)andpossiblyendupbeingwrittenoff(aslostdebtordebttransferredtoexternalcollectionagencies).Oncewebringpotentialdebtconsiderationsintotheequation,wemayhavetoadjustour“currentLTV”(),saynp(t),todenotethenon-paymentprobabilityforage∫∞t,togiveusLTV=S(t)v(t)np(t)D(t)’s(orasegment’s)LTV,’sLOS(aneffectwewouldhavetomodel);itwillaffectcustomervalue;itcanalsoaffectthenon-paymentprobabilitynp,,,ifweareabletoestimate(followingasimilarmethodologytotheonewedescribedforretention)theeffectofouractivityonsegment-levelaverageLOS,averagecustomervalue,andaveragenon-paymentprobability,wecancalculateandutilizeasegment-levelestimateofthenetLTVeffectofreducingasegment’
CUSTOMERLIFETIMEVALUEMODELS337concludedthatanoptimaldecisionrulefordeterminingwhichcustomerstosuggestthein-centivetowasbasedontheirvalue-weightedpropensityforchurn,andhadtheform:suggestincentive⇔vp>const,(value-weightedpropensityscores)forincludingcustomersincampaigns—forretentionaswellasothertasks—,ifwewanttojustifyitasanoptimalpolicyweneedtomakesomesimplifying,sometimesproblematic,-/up-sellmodels,-sellismoreconcernedwithcustomervalueandlesswithlengthofservice,wecanformulateanalternativevalue-weightedpropensityapproach,wheretherelevantpropensityscorewouldbethecustomer’:–Thatthediscountingmodelisathresholdfunction–Thatnochurnoccurswithinthisperiod(c)–Thatthechangeincustomervaluevisfixed(and“known”)(c)Thenifwedenotethepropensityforacceptancebypweeasilyseethatanoptimaldecisionruleforprofitabilityofpresentingacross-selloffertoacustomerhastheform:(c)(c)(c)vp−pG>C(whereasbeforeCisthechannelcostandGistheacceptancecost).,ifweonlyconsiderthecurrentcampaignasthehorizon,thenthepropensitytorespond,multipliedbyexpectedpurchaseordonation,(RossetandInger,00),,letusnowassumethatthenonpaymentprobabilitynp(v)isafunctionofthecustomer’’scredittov<·(1−np(v))whileifwelimitthecustomer’screditweexpectv·(1−np(v)).Soundertheseassumptionsanoptimal00decisionwouldbetolimitthecustomer’:
’,whichwillenableustoreachourultimategoal—togetusefulandactionableinformationabouttheeffectsofdifferentbusinessandmarketingactivitiesoncustomers’,,B34,187–,,variance,0/,1(1):55–,,:,11:395–,A.,Vatnik,N.,Rosset,S.,andNeumann,-cup2000Question1Winner’,2(2):,,.“,53:457–,.,Drew,J.,Betz,A.,andDatta,-99,94–,,-99,251–,E.,Vatnik,N.,Rosset,S.,Duenias,M.,Sassoon,I.,andInger,-cup2000Question5Winner’,2(2):,,,,-cup99:Knowledgediscoveryinacharitableorganization’,1(2):85–,S.,Murad,U.,Neumann,E.,Idan,I.,andPinkas,—-99,409–,S.,Neumann,E.,Eick,U.,Vatnik,N.,andIdan,-2001,456–,,-PLUS,,ChurnandRetentionAnalysis,BadDebtAnalysis,,’sHighInstituteforScienceandEngineering,
’-dimensionalpredictionmodels,.
