Probabili(esandProbabilis(cModels
EduardoEyrasComputa(onalRNABiology
PompeuFabraUniversity-ICREABarcelona,Spain
Master in Bioinformatics UPF 2017-2018
Probabili(es
Probabili(esandProbabilis(cModels
VariablesConsideravariableXtorepresentanevent:“theoccurrenceofsomething”VariableXcantakeanumberofpossiblevalues,e.g.x1,x2,….(Xisusuallycalledarandomvariable,eitherdiscreteorcon(nue)ProbabilityTheprobabilityofapar(culareventxiisthepropor(onof(mesthatittakesplaceifwemeasureXasufficientnumberof(mes.WecanwritethisasP(X=xi)orP(xi)
Probabili(esandProbabilis(cModels
Probabilitydistribu0onThefunc(ongivenbyP(X=s)foreachsistheprobabilitydistribu0onofX,alsodenotedasP(X)Probabili(estakevaluesbetween0(somethingneveroccurs)and1(somethingalwaysoccurs),andthesumofprobabili(esofallpossibleeventsisalwaysnormalizedto1.
€
0 ≤ P(s) ≤1, and P(s) =1s∈S∑
€
s∈ S
SamplespaceItisthesetofpossibleoutcomesforsomeeventorrandomvariable.S
Probabili(esandProbabilis(cModels
€
0 ≤ P(s) ≤1, and P(s) =1s∈{sun,...}∑
Theprobabilitydistribu(onforX:
Example Xrepresentstheweatherfortomorrow.TheprobabilityofrainingtomorrowiswriVenas:
Whichcanbealsosimplifiedas€
P(X = rain)
€
P(rain)
event probability
sun 0.1
clouds 0.3
rain 0.3
snow 0.2
sleet 0.1
P(not − sunny) = P(s) = 0.9s∈{clouds,rain,snow,sleet}
∑
Probabilityofasubset:
Joint,condi(onalandmarginalprobabili(es
ThejointprobabilityfortwoeventsA=aandB=biswriVen
€
P(a,b) ≡ P(A = a∩ B= b)
Thisdistribu(onisdefinedoverallpossiblevaluesthatAandBcantakeP(A = s, B = r)(foralls,rvaluesfromthesamplespace)
Jointprobability
Joint,condi(onalandmarginalprobabili(es
3importantproper(esofthejointprobabilityJointprobabili(esaddupto1ReversibilityNotalways P(A = a∩B = b) = P(A = a)P(B = b)
P(r,s∑ A = r∩B = s) =1
Thisdistribu(onisdefinedoverallpossiblevaluesthatAandBcantakeP(A = s, B = r)(foralls,rvaluesfromthesamplespace)
P(A = a∩B = b) = P(B = b∩A = a)
P(b) = P(b,ai )ai
∑
Whenthejointprobabili(esareknownwecancalculatethemarginaldistribu(on:
Itdescribesthedistribu(onofB“ignoring”othervariables.Itisobtainedfromthejointdistribu(onbysummingoveroneofthevariables:
Joint,condi(onalandmarginalprobabili(es
Generallyif
€
ai∩ a j =∅
aii = A
Marginaldistribu0on
P(a | b) = P(a,b)P(b)
=P(a,b)P(b,a ')
a '∑
€
P(a | b)a∑ =1
Joint,condi(onalandmarginalprobabili(es
P(a,b) = P(a | b)P(b)
Wecanalsodefinethejointprobabilityintermsofthecondi0onalprobability:
Condi0onalprobabilityThecondi(onalprobabilityofaneventA=awithrespectaneventB=bisdefinedas:
Generally,wedonotusejointprobabili(esdirectly,butcondi(onalprobabili(es
Notethatsince P(a,b) = P(b,a) P(a | b)P(b) = P(b | a)P(a)Itfollowsthat
€
P(A |B)Thecondi0onalprobabilityrepresentsthedistribu(onofAgiven
thatweknowthevalueofB(thisisdifferentfromajointprobability)
Joint,condi(onalandmarginalprobabili(es
P(a | b) = P(a,b)P(b)
=P(a,b)P(b,a ')
a '∑
P(b) = P(b,a)a∑ = P(b | a)P(a)
a∑
Wecanalsocalculatethemarginaldistribu0onalsofromthecondi(onalprobabili(es:
Joint,condi(onalandmarginalprobabili(es
P(b) = P(b,a ')a '∑
P(a | b)a∑ = P(a,b)P(b)a
∑ =P(a,b)
a∑P(b,a ')
a '∑
=1
Notethatthecondi0onalprobabilityisnormalizedto1
P(a | b) = P(a,b)P(b)
Themeaningofprobability
Probabili(escandescribe“frequenciesofoutcomesinrandomexperiments”
Probabili(escandescribe“frequenciesofoutcomesinrandomexperiments”
Themeaningofprobability
Probabili(escanbeusedmoregenerallytodescribedegreesofbeliefinproposi(ons(notnecessarilyinvolvingrandomvariables).Forexample:“Theprobabilitythatthebutleristhemurderer,giventheevidence”Thuswecanuseprobabili(estodescribeassump(onsorhypotheses,andtodescribeinferencesgiventhoseassump(ons.Therulesofprobabili(esensurethatiftwopeoplemakethesameassump(onsandreceivethesamedata,thentheywilldrawiden(calconclusions.
ConsiderHtobethesetofassump(onsorhypotheses,whicharepartofourprobabilis(cmodelWecanconsiderthepreviousdefini(ons:
Themeaningofprobability
€
P(a,h) ≡ P(A = a∩H = h) = P(a,h is true)
€
P(a) = P(a,h)h∈H∑Marginalprobability:
€
P(h) = P(h,a)a∑
Condi(onalprobability: P(a | h) = P(a,h)P(h)
=P(a,h)P(h,a ')
a '∑
Jointprobability:
H = h{ }
Exercise:verifytheseproper(es
€
P(a,b | h) = P(a | b,h)P(b | h) = P(b | a,h)P(a | h)
Productrule(followsfromthedefini(onofcondi(onalprobabili(es)
Sumrule(rewri(ngthemarginalprobabilitydefini(on)
€
P(a | h) = P(a,b | h)b∑ = P(a | b,h)
b∑ P(b | h)
Bayes’Theorem
Fromthepreviousofdefini(onsofcondi(onalprobabilitywecanwrite:
PosteriorprobabilityandBayes’theorem
P(a | h) = P(h | a)P(a)P(h)
=P(h | a)P(a)P(h | ak )P(a k )
ak
∑
P(a,h) = P(a | h)P(h) = P(h | a)P(a)
Thisallowstowritetheprobabilityofthedataacondi0onedtothehypothesish:
€
P(h | a) = P(a | h)P(h)P(a)
=P(a | h)P(h)P(a | h' )P(h' )
h'∑
Ortheotherwayaround,theprobabilityofthehypothesishcondi0onedtothedataa
Likelihood:
Posterior:
Bayes’theorem:
€
P(h | a) = P(a | h)P(h)P(a)
=P(a | h)P(h)P(a | h' )P(h' )
h'∑
ThisformoftheBayes’theoremdescribestheprobabilityofanhypothesis(H=h),giventhemeasurement(A=a).This probability is wriVen in terms of condi(onal probabili(es of themeasurement given the hypothesis (called likelihoods). From them, usingBayes theorem,we can es(mate howprobable is the hypothesis, given theobserveddata.
PosteriorprobabilityandBayes’theorem
Exercise:verifythisproperty
€
P(b | a,h) = P(a | b,h)P(b | h)P(a | h)
=P(a | b,h)P(b,h)P(a | b' ,h)P(b' ,h)
b'∑
Example:TransmembraneProteins
Wewanttogenerateamodelthatisabletoseparatesequencesthatformtransmembranehelicesfromsequencesthatformtheloops.Importanthypotheses(priorknowledge):heliceshavehighcontentofhydrophobicaminoacids(e.g.Isoleucine(I),Leucine(L),Valine(V),…).Thereisadifferencefromrandomness!!àWecanapplyprobabili(es
Ques0on:Givenoneormoreaminoacid,e.g.L,canwedis(nguishbetweenLinhelicesorLinloops?
Considerthefollowingprobabilis(cmodel,whereforeachaminoacid,theoccurrenceprobabilityisgivenbyqa,suchthatWiththismodel,theprobabilityofasequenceofresiduesx1…xncanbewriVenlikethis(assumingindependence):
€
P(x1x2...xn ) = qx1qx2qxn = qxii=1
n
∏
0 ≤ qa ≤1, and qa =1a∑
Example:TransmembraneProteins
Example:TransmembraneProteins
Consideranumberofknownstructures:sequencewithannotatedhelicesWecancalculatethefrequenciesofeachaminoacidainhelices=qa,andinloops=pa
Weestablishtwohypotheses(ormodels):(1)theloopmodelMloopgivenbytheprobabili(esobservedinloops:p(2)thehelixmodelMhelix givenbytheprobabili(esobservedinhelices:q
Example:TransmembraneProteins
Weestablishtwohypotheses(ormodels):(1)theloopmodelMloopgivenbytheprobabili(esobservedinloops:p(2)thehelixmodelMhelix givenbytheprobabili(esobservedinhelices:q
Consideranumberofknownstructures:sequencewithannotatedhelicesWecancalculatethefrequenciesofeachaminoacidainhelices=qa,andinloops=pa
GivenasequenceofNaminoacidss=x1…xNfromanunknownprotein,wecancalculate:
€
P(s |Mloop ) = px1 px2 pxN = pxii=1
N
∏ = L(s |Mloop ) Likelihoodofbeinginaloop
€
P(s |Mhelix ) = qx1qx2qxN = qxii=1
N
∏ = L(s |Mhelix ) Likelihoodofbeinginahelix
Theseprobabili(esarethelikelihoodofthedatagivenaspecificmodel
PosteriorProbability
WeactuallywanttoknowtheprobabilitythatthedataisdescribedbymodelMk,andnottheprobabilitythatthedatawouldariseifthemodelweretrue.Thatis,wewantthe:Posteriorprobability=TocalculateitweneedBayes´theorem:
€
P(Mk |D)
€
P(A,B) = P(A |B)P(B) = P(B | A)P(A)⇒ P(B | A) = P(A |B)P(B)P(A)
ThebasicprincipleofBayesianmethodsisthatwemakeourinferencesusingtheposteriorprobabili(es.
€
P(Mk |D) =L(D |Mk )P(Mk )L(D |Mi)P(Mi)
i∑
Iftheposteriorprobabilityofourmodelismuchhigherthantheothermodels,wecanbeconfidentthatthisisthebestmodeltodescribethedata.
Posteriorprobability Likelihood
PosteriorProbability
P(Mloop |D) =P(D |Mloop )P(Mloop )
P(D |Mloop )P(Mloop )+P(D |Mhelix )P(Mhelix )
P(Mhelix |D) =P(D |Mhelix )P(Mhelix )
P(D |Mhelix )P(Mhelix )+P(D |Mloop )P(Mloop )
P(Mk |D) =P(D |Mk )P(Mk )
P(D)
=P(D |Mk )P(Mk )P(D |Mi )P(Mi )
i∑
PriorProbability
Priorprobability:probabilityassociatedtoeachofthehypotheses(eachofthemodels),e.g.:
€
P(Mloop )
€
P(Mhelix )
Weassignpriorprobabili(estoeachmodel.Theymustadduptoone
€
P prior(Mk ) =1k∑
Thereisnopar(cularwaytoassigntheseprobabili(es.Theymustbees(matedusingsomepriorknowledgeaboutthedata(e.g.isitveryunlikelytofindhelicesinproteins?)Safechoice:uniform(uninforma(ve)probabili(es
NotethattheposterioriswriVenintermsoftheprobabilityofthehypothesisP(M).Thisiscalledtheprioranditrelatestotheques(on:Areallmodels(helix,non-helix)equallylikely?
€
P(Mk |D) =L(D |Mk )P(Mk )L(D |Mk )P(Mk )
k∑
Bayes’Theorem
ThebasicprincipleofBayesianmethodsisthatwemakeourinferencesusingtheposteriorprobabili(es.ThisprobabilityiswriVenintermsofcondi(onalprobabili(esofthemeasurementgiventheassump(on(calledlikelihoods).Fromthem,usingBayestheorem,wecanes(matehowprobableistheassump(on,giventheobserveddata.
€
P(Mk |D) =L(D |Mk )P(Mk )L(D |Mk )P(Mk )
k∑
Bayes’Theorem
€
posterior = likelihood × priorevidence
Yourini(albeliefYourimprovedbelief
degreetowhichyourbeliefexplainstheevidence
Alltheevidence
€
P(Mk |D) =L(D |Mk )P(Mk )L(D |Mk )P(Mk )
k∑
Bayes’Theorem
MedicaldoctorsstudyP(Symptom|Disorder)inpa(entsHowever,whentheyneedtomakeadiagnosis,theyes(mateP(Disorder|Symptom)E.g.,ifweknowP(Fever|Flu)andP(Flu),wecanthenuseBayes’Theoremtododiagnosis
Thedenominatoristhemarginal:Probabilityofhavingfeverwhetherornotyouhaveflu
Problem:Doesapa(enthavethediseaseornotApa(enttakesalabtestandtheresultcomesbackposi(ve.Thetestreturnsacorrectposi(veresultinonly98%ofthecasesinwhichthediseaseisactuallypresentandacorrectnega(veresultinonly97%ofthecasesinwhichthediseaseisnotpresent.Furthermore,0.008oftheen(repopula(onhavethisdisease.
€
P(c)
€
P(c )
€
P(+ | c)P(− | c)
€
P(+ | c )P(− | c )
€
P(c | +)?
Exercise
References
BiologicalSequenceAnalysis:Probabilis0cModelsofProteinsandNucleicAcidsRichardDurbin,SeanR.Eddy,AndersKrogh,andGraemeMitchison.CambridgeUniversityPress,1999ProblemsandSolu0onsinBiologicalSequenceAnalysisMarkBorodovsky,SvetlanaEkishevaCambridgeUniversityPress,2006Bioinforma0csandMolecularEvolu0onPaulG.HiggsandTeresaAVwood.BlackwellPublishing2005.