Click here to load reader
Upload
lynhi
View
212
Download
0
Embed Size (px)
Citation preview
Balanced Literacy Proposal 1
Introduction & Background Information
A balanced literacy school focuses on activities such as shared and guided reading and writing through the utilization of literacy, listening and writing centers within a print-rich environment. This research project proposes to evaluate the implementation of the Balanced Literacy Initiative, which began three years ago, and was spearheaded by the Lakeshore Alliance for Student Success (LASS) in Northwest Indiana. Currently 25 elementary schools are receiving the training and resources needed to implement the program in grades K-2. The Balanced Literacy program is being implemented with the assistance of literacy coaches who provide professional development training for the participating teachers. A balanced literacy school consists of certain characteristics that serve specific purposes, which relate to the development of literacy (e.g. reading, spelling and writing). Appendix A provides a descriptive summary of each balanced literacy characteristic and its specific purpose. Participants
This study will include the following schools in Northwest Indiana who are participating in the Balanced Literacy Initiative:
• Olive Township Elementary • Prairie View Elementary • Rolling Prairie Elementary • Fieler Elementary • Kenwood Elementary • Maywood Elementary • Joseph Hess Elementary • Thomas Edison Elementary • Thomas Jefferson Elementary • Franklin Elementary • Irving Elementary • Iddings Elementary • Jonas E. Salk Elementary • Edgar Miller Elementary • John Wood Elementary • Longfellow Elementary • Grissom Elementary • Hosford Park Elementary • Nathan Hale Elementary • Hebron Elementary • Lake Prarie Elementary • Morton Elementary • Harding Elementary • Oak Hills Elementary • Three Creeks Elementary
Balanced Literacy Proposal 2
Research Design, Research Questions & Suggested Data Analysis
This evaluation study will incorporate multiple methods and designs in order to address the evaluation questions. The specific designs will include the mixed methods triangulation design, the quasi-experimental design (panel data with and without a comparison group) and a formal meta-analysis. The triangulation component of this evaluation study will be used to address the first two research questions and assist in the data validation process. The quasi-experimental component will be used to determine the degree to which (1) balanced literacy results in a statistically significant amount of growth on standardized achievement tests; (2) balanced literacy results in a narrowing of the achievement gaps and (3) balanced literacy results in an academic advantage as compared to comparison classrooms. Finally, the meta-analytic component will be used to identify specific school and/or classroom effects as well as implementation effects that moderate the effectiveness of the balanced literacy program on student achievement in reading, writing and spelling. The primary independent variable for this evaluation study will be participation in Balanced Literacy. Treatment fidelity will serve as an intervening factor. The primary dependent variable will include student standardized test performance. Table 1 below outlines each evaluation question, the proposed design and the suggested analysis for each question. Table 1. Evaluation Question and Evaluation Approach Linkage
Evaluation Question Design Suggested Analyses Evaluation question one: How well are the teachers implementing the balanced literacy approach?
Mixed methods triangulation
Descriptive and thematic analysis based on observations and surveys/interviews
Evaluation question two: To what extent is balanced literacy being implemented?
Mixed methods triangulation
Descriptive and thematic analysis based on observations and surveys/interviews
Evaluation question three: To what extent do the classrooms participating in balanced literacy show growth as measured by standardized achievement tests?
Quasi-experimental: panel design with and without a comparison group
Descriptive statistics Hierarchical linear modeling (value added)
Evaluation question four: What are the significant moderating factors with regard to the effectiveness of the implementation of balanced literacy?
Quantitative meta-analysis
Coding for moderators, computing effect sizes, testing for homogeneity of effect sizes and testing for significant moderators if effect sizes are heterogeneous
Balanced Literacy Proposal 3
Data Collection
Student level achievement data will be collected every fall and spring of each academic year for a total of four years of data (baseline year plus three implementation years). The teacher observation data will be collected at various points throughout the study. A copy of the teacher observation checklist form is attached (see Appendix B). In addition, a survey/interview will be developed by the researcher that will ask teachers about their implementation of balanced literacy in the classroom. The variables that will be measured are listed in Table 2. The color key associated with Table 2 is as follows: Grey represents the identification variables, yellow represents the dependent variables, green represents the control variables (mediators and moderators) and brown represents the independent variable. The required column in Table 2 indicates whether or not the data is required for the evaluation study. Table 2. Research Variables
Variable Measure Required
Student identifier Random number Yes
Classroom identifier Random number Yes
School identifier Random number Yes
Reading Standardized achievement test (e.g. ISTEP) Yes
Writing Standardized achievement test (e.g. ISTEP) Yes
Spelling Standardized achievement test (e.g. ISTEP) Yes
Comparison subject Standardized achievement test (e.g. ISTEP) No
Student gender Student records Yes
Student race Student records Yes
Student disability status Student records Yes
Student SES Qualifies for free and reduced lunch Yes
Student attendance School records Yes
School: percent low income School records Yes School: percent limited English School records Yes
School: parental involvement School records No
Extent of implementation Observation & surveys Yes
Ability level (gifted) School records Yes
Retention status (yes/no) School records Yes
Balanced Literacy Proposal 4
Kindergarten (1/2 day or full) School records Yes
Attended pre-school School records Yes
English as 2nd language School records Yes
Participated in PAT program School records Yes
Implementation fidelity Observation & surveys Yes
Type of assessment used School records Yes
Balanced Literacy Treatment group (experimental/control) Yes Procedure
School related data will be extracted directly from the Indiana Department of Education website (http://www.doe.in.gov/data/). Student and teacher level data will be obtained from the literacy coaches who will be collecting the data via teacher observation forms and student standardized achievement tests. However, not all of the schools are using the same student achievement assessment instruments. The teacher training will be implemented by Purdue University Calumet. A copy of the course syllabus is attached (see Appendix C). Data Quality Control Methods, Reliability and Validity
The data quality control methods that will be utilized in this evaluation study include the following:
1. Multiple raters will be used for establishing inter-rater reliability for all qualitative analyses resulting from the research survey/interview.
2. Formal reliability analyses will be conducted for all of the quantitative items on the research survey.
3. Multiple data sources will be used for triangulation purposes (e.g. student achievement data, teacher observations and teacher survey/interview).
4. Evaluator has designed the evaluation study such that potential threats to internal validity (history, selection bias, maturation and statistical regression to the mean) are minimized and threats to the statistical conclusion validity are minimized. Historical threats, maturational threats, selection bias and statistical regression to the mean are minimized due to the quasi-experimental component of the study. Threats to the statistical conclusion validity are minimized by carefully considering the implications of sample size when conducting quantitative analyses and by testing all of the statistical assumptions associated with each statistical analysis.
5. An on-line survey format will be used to minimize human error in data entry (e.g. SurveyMonkey or Zoomerang).
6. All quantitative data will be checked for accuracy. 7. Electronic back-up files of all data and relevant evaluation study
documents will be kept in a secure place.
Balanced Literacy Proposal 5
Ethical Considerations The data will consist of archival school level data that is provided on the Indiana Department of Education website, teacher level implementation data and student level demographic and achievement data. All of the student level data used for this study consists of data that is already being collected and therefore no special data collection efforts will be made relative to student achievement. Teachers will be asked to complete a research survey/interview that will be completed on a volunteer basis and informed consent for completing the survey/interview will be required. Confidentiality will be maintained at all times by assigning an identifier to each teacher and student so that names are not maintained within the primary investigator’s database. This study poses absolutely no physical harm to participants. Furthermore, preliminary studies have shown positive academic affects for students who are exposed to the treatment associated with this study (Biancarosa, G., Byrk, A., & Dexter, E., 2008)1. Finally, this study contains no deception.
Significance of Study
Although there have been some recent and preliminary studies conducted on the value-added effects of balanced literacy through the use of literacy coaches, Biancarosa et al (2008) have indicated that there is still a “pressing need” to determine why the effects of the literacy coaches vary between teachers and schools. This study seeks to not only replicate the findings of Biancarosa, et al, but to replicate them based on a sample of students and schools in Northwest Indiana and to determine the reasons for the variability in the effectiveness of the implementation through the use of a meta-analytic design (refer to Table 1).
Source of Funding
This study is funded through the Lakeshore Alliance for Student Success (LASS).
1 Paper presented at the National Reading Conference on December 3, 2008 in Orlando, FL. Slides: Assessing the Value-Added Effects of Literacy Collaborative Professional Development on Student Learning