6. Implications for Analysis: Data Content. 1 Prerequisites Recommended modules to complete before viewing this module 1. Introduction to the NLTS2

6. Implications for Analysis: Data Content


2

Prerequisites

• Recommended modules to complete before viewing this module 1. Introduction to the NLTS2 Training Modules 2. NLTS2 Study Overview 3. NLTS2 Study Design and Sampling For parent/youth data sources:

• 4. NLTS2 Data Sources: Parent and Youth Surveys For school or student data sources:

• 5. NLTS2 Data Sources: School Surveys, Student Assessments, and Transcripts


3

Overview NLTS2 sample Multiple data sources NLTS2 data collection Availability of data Data discrepancies Missing data Response rates Restricted-use NLTS2 data Closing Important information


NLTS2 restricted-use data

• NLTS2 data are restricted.• Data used in these presentations are from a

randomly selected subset of the restricted-use NLTS2 data.

• Results in these presentations cannot be replicated with the NLTS2 data licensed by NCES.

4


5

NLTS2 sample

• Sampled in two stages First stage: Local education agencies (LEAs) and state-

operated special schools. Second stage:

• Students sampled to represent those in the 12 federally defined disability categories.

• Students were 13 to 16 in grade 7 or above, and in middle or high school.

• Complex sampling has implications for weighting.


6

Multiple data sources

• Parent/youth survey A parent interview or mail survey

questionnaire completed by youth’s parent or guardian

A youth interview or mail survey questionnaire completed by youth

• School characteristics survey School-level information about the

school that the student attended• Teacher survey

A general education academic subject teacher about student’s experiences in general education academic classroom

• Student’s school program survey School staff member who knew about

student’s program, including transition and vocational education

• Student assessments Direct assessment of student’s

scholastic abilities Student interview on attitudes about

school, self-determination Alternate Assessment completed by

student’s teacher if student was unable to participate in the Direct Assessment and interview

• Secondary school transcripts


7

Data sources• Although data were collected from many sources (parents,

school administrators, teachers, youth), the unit of analysis is the individual youth. In a given wave, a single teacher may have responded to multiple

surveys for students within the teacher’s school or classroom, but each survey is in reference to a single student.

• Generalizations cannot be made about families, communities, LEAs, schools, teachers, or classrooms. Although many youth may have attended any single school, data

from the School Characteristics Survey are stored in a youth-level record not a school-level record.


8

NLTS2 data collection• Parent/guardian interviews/surveys

Collected in 5 waves• Youth interviews/surveys

Collected in 4 waves• School data

Collected in 2 waves• Student assessments

Assessed once when youth was 16 years or older• Multiwave collection resulting in one file included with

Wave 2 data

• Secondary school transcripts Collected for the school years when youth in secondary school

included with Wave 5 data


9

NLTS2 data collection: TimelineWave 1 Wave 2 Wave 3 Wave 4 Wave 5

Year 1

2000-2001

Year2

2001-2002

Year 3

2002-2003

Year4

2003-2004

Year 5

2004-2005

Year6

2005-2006

Year 7

2006-2007

Year8

2007-2008

Year 9

2008-2009

Year10

2009-2010

Parent telephone interviews or surveys

Youth telephone interviews or surveys

Direct assessment andin-person interviews

Teacher Survey

Student’s School Program Survey

School Characteristics Survey

Transcripts


10

Availability of data

• Not all sources of data are available for a given youth. Within a wave, a youth may have one or more sources of

data, but not necessarily all sources.• For example, there may be a Wave 1 Parent Interview for a youth but

no Wave 1 school-level data or vice versa. Across waves, a youth may have data for a given source in

one wave and not another.• For example, there may be a Wave 1 Parent Interview and a Wave 5

Parent/Youth Interviews but no Wave 2 through 4 Parent/Youth data.


11

Data discrepancies

• Similar items in multisource, longitudinal data do not always agree. For example, all youth in the sample were in special

education in 2000. However• Some parents indicated that their son or daughter was never in

special education.• Parents reported disabilities other than those that the district and/or

school has reported. A parent may have reported the youth dropped out in one

wave as well has having reported he or she graduated from high school in another wave.


12

Data discrepancies• A secondary school transcript may report a different high school

leaving date than reported in a parent/youth interview.• A parent and a youth may provide conflicting responses on such

things as high school leaving status, postsecondary attendance, or employment status. Example: Parent/guardian and youth responses for why youth left

secondary school may differ. Example: Parent/guardian and youth responses for whether youth

attended a 2-year/junior college may differ.

Implications for Analysis: Data Content

Data discrepancies: Examples

13These results cannot be replicated with full dataset; all outputin modules generated with a random subset of the full data.


14

Missing values• There are many reasons for data to have missing values,

which are an unavoidable part of survey work.• Data can be completely missing for a given source.

The appropriate respondent to complete the survey could not be located.• Example: School had no record of student attending that school.• Example: Parent had moved and we no longer had contact information to

reach the family .• Example: Foster parent no longer had youth in his or her care.

Respondent refused to complete a questionnaire or participate in an interview.

Youth attended a known school but school did not compile a transcript for that youth.


Missing values• Respondent may have had data for a given data source, but

data can be missing within a file for that source. Respondent did not know or refused to answer a question. The question was not applicable. The question was skipped by design. Respondent missed a page in a questionnaire. Respondent terminated the interview before it was

completed. Respondent completed a different version of an instrument

that does not have that particular item.• For example, respondent completed the mailed Family Survey in lieu

of the Parent/Guardian interview.

15


16

Missing values

• Pay attention to the number of respondents when conducting any analysis. Across waves, the number of respondents for a given data

source will vary. Within a wave, the number of respondents for each data

source will be different from file to file. Within a data collection instrument, n’s will vary item by

item.


17

Missing values

• Longitudinal and/or multisource analyses are affected by missing values. The more sources of data that are being combined, the

larger the likelihood that data will be missing for at least one of those sources.

The number of respondents is typically smaller for cross-instrument or cross-wave analysis than for analysis of a single instrument.• For example, an analysis that requires a respondent to have all 5

waves of Parent/Youth Survey data will have a smaller number of respondents than any single wave of Parent/Youth Survey data.


18

Response rates• Data collected from multiple sources and multiple points

in time generated different response rates.• Generally there were higher response rates in earlier

waves than in later waves. Lower response rate between Wave 1 Parent and Wave 2

Parent/Youth largely due to attrition.• Combining data from multiple sources can result in a

smaller n if requiring that all respondents have all sources.


19

Response ratesParent/Youth Surveys

W1 Parent Survey 82%

W2 Parent/Youth Survey 61%




School Surveys

W1 (Teacher, Program, School) 36%,53%, 57%

W2 (Teacher and Program) 41%, 52%

Student Assessments

W1 Administration 63%

W2 Administration 72%

Transcripts

[multiple waves] 81%


20

Restricted-use NLTS2 data

• NLTS2 Web Seminar data NLTS2 data and statistical output are restricted and require

licensure for use. Data used in these presentations are a random subset of

the full NLTS2 database. Results cannot be replicated with fully licensed data as

demonstrations and examples are generated using a random subset of the data.

• If you plan to use NLTS2 data, you will need to obtain a license through NCES. http://nces.ed.gov/statprog/rudman/


21

Closing

• Topics discussed in this module NLTS2 sample Multiple data sources NLTS2 data collection Availability of data Data discrepancies Missing data Response rates Restricted-use NLTS2 data


22

Closing

• Next module for Parent/Youth data sources: 7. Implications for Analysis: Parent/Youth Survey Data

• Next module for school or student data sources: 8. Implications for Analysis: School Survey, Student

Assessment, and Transcript Data


23

Important information NLTS2 website contains reports, data tables, and other

project-related information http://nlts2.org/

Information about obtaining the NLTS2 database and documentation can be found on the NCES website http://nces.ed.gov/statprog/rudman/

General information about restricted data licenses can be found on the NCES website http://nces.ed.gov/statprog/instruct.asp

E-mail address: [email protected]

http://nlts2.org/

http://nces.ed.gov/statprog/rudman/

http://nces.ed.gov/statprog/instruct.asp

Documents

6. Implications for Analysis: Data Content. 1 Prerequisites Recommended modules to complete before viewing this module 1. Introduction to the NLTS2