Upload
others
View
4
Download
0
Embed Size (px)
Citation preview
Statistical Models
University of Trento - FBK
16 February, 2015
1 / 12
Information about the course
Website: http://science.unitn.it/~pugliese/
Timetable: Monday 13.00 - 15.00Thursday 13.00 - 16.00
e-mail: [email protected], [email protected]
Office Hours:Pugliese Monday 9-10.30
Filosi Monday 17-18.30 (sending and e-mail before would be better)
Exam:Written exam: March, 27thScript Submission: March, 30thOral Presentation: April, 1st to 8th
Reference book:Julian J. Faraway, Practical Regression and Anova using Rhttp://cran.r-project.org/doc/contrib/Faraway-PRA.pdf
2 / 12
Course Subjects
Intro to statistical models (general ideas)
Intro to R a statistic environment
Intro to linear models
C.I., tests, linear models, residuals and PRESS
Regression in R
ANOVA and polynomial regression
Generalize Linear Models (GLM)
Model choice
3 / 12
What is Statistical Learning?Examples
Med Science: Predict whether a patient, hospitalized due to a heart attack, will have a secondheart attack. The prediction is to be based on demo- graphic, diet and clinicalmeasurements for that patient.
Economics: Predict the price of a stock in 6 months from now, on the basis of companyperformance measures and economic data.
Automation Identify the numbers in a handwritten ZIP code, from a digitized image.
Med Science: Estimate the amount of glucose in the blood of a diabetic person, from theinfrared absorption spectrum of that person’s blood.
Demography: Identify the risk factors for prostate cancer, based on clinical and demographicvariables.
Common features?Keywords: Data Mining, Statistics, Artificial Intelligence
4 / 12
Learning from data
A typical scenario:
Given an outcome measurement, quantitative (such as stock price) orcategorical (such as heart attack/no heart attack), that we wish to predictbased on a set of features (such as diet and clinical measurements).
We have a training set of data, in which we observe the outcome and featuremeasurements for a set of objects (such as people).
5 / 12
Learning from dataDifferent Approaches
SupervisedPresence of the outcome variable
Examples:
Regression
Machine Learning
Decision Tree
Kernel Estimators
UnSupervisedAbsence of the outcome variable
Examples:
Clustering
Hidden Markov Models
Pricipal Component Analysis,Dimesionality Reduction
Neural Network
Describe how the data are organized and clustered in case of unsupervised learning.Build a statistical model for predicting or estimating an output based on one or more inputs.
6 / 12
A brief of history of Statistical Learning
1805 - 1809
Lege
ndre
and Gau
ss: pu
blish
edthe
first p
aper
onlea
stsq
uare
met
hods.
1936
- 1940
Fisher : invented the linear
discriminant analysis method. A
newmethod called logistic
regressionhas been proposed.
1980 - 1986
Breim
an, F
riedm
an, O
lshen
and
Stone:
Classifi
catio
nan
d
Regre
ssio
nTr
ees
Hastie
, Tibs
hiran
i :Gen
erali
ze
Additive
Models
7 / 12
Examples of Statistical ModelsGraphically: Income vs Years of Education
● ●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
10 12 14 16 18 20 22
2030
4050
6070
80
Education
Inco
me
● ●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
10 12 14 16 18 20 22
2030
4050
6070
80
Education
Inco
me
8 / 12
General Definition of Statistical Models
Input or independent Variables X: In the example the Years of Education
Output or dependent Variables Y: In the example the Income
Definition:
Given X = (X1,X2, . . . ,Xp), we assume there is a relationship between Y and X such as we canwrite:
Y = f(X) + ε
where ε is a random error term andf represent the information that X provide about Y .
9 / 12
Why to estimate f?Prediction
Prediction Settings:
X is known
Goal Y cannot be obtained (easily)
Model:
Y = f (X ) for n → ∞, ε → 0
where f is an estimate of f and Y represents the resulting prediction of Y . In this case we are notinterested in the exact form of f , as soon as the prediction Y of Y are accurate.
10 / 12
Prediction Errors
Given the model Y = f (X ) which is an estimate generated fromY = f (X ) + ε:
Error and Variance
Define the expected value of the square difference between thepredicted and the true values as:
E(Y − Y )2 = [f (X )− f (X )]2︸ ︷︷ ︸Reducible error
+ Var (ε)︸ ︷︷ ︸Irreducible error
11 / 12
Why to estimate f?Inference
Inference Settings:
X is known
Goal Understand relationship between X and Y → exact form of f
Typical Questions:
Which predictors are associated with the response?
What is the relationship of each predictor with the response?
Can we use a linear equation to capture the relation between Yand X?
12 / 12