Goal Based Review - SAT vs. PSAT Comparison

7/29/2019 Goal Based Review - SAT vs. PSAT Comparison

1/4

f. Selection of Goal-Based Model for New Evaluation

As I would truly get into trouble and my efforts would gocompletely unappreciated by evaluating the Freshman Academy, Ihave a new evaluation that will be welcomed with open arms. Ourhigh school, as many has struggled with an appropriate standardizedtest. In Maine, the state had an 11th grade assessment which it founddisfavor. Maine (and most of New England) adopted the SAT for its11th graders. This would create several problems: now being thestandard, more than 95% of 11th graders must take it and how to beststudy just for the SATs.

The 95% problem proved huge; as a large percentage ofstudents from this community have no post secondary plans. While agood year would see 75-80% of the students moving on, that was

about the percentage taking the SAT test. To get 95% involved gettingstudents with no future plans to take the test (the state picked up thecost). This often resulted in a room full of students the first school dayafter the test wishing they were not there for the makeup. Thesestudents would often only do just enough to have the test count, butreceived the lowest score possible. This brought the test averagedown considerably.

While very different tactics would have to be employed with thestudents above, it was decided that a more concentrated SATpreparation time was required prior to testing to bring up the average

for the other students. During student advisory for a month prior tothe May test date, juniors spent their time in English rooms studyingfor the test (Math scores had seen a slow, small increase, but Englishscores had plummeted due to the above).

(Now I can outline my evaluation, need to preface so theevaluation would make sense). I plan to evaluate data from previousyears to this years test data. Additionally, if I am able to get and usePSAT data for the same students the previous year, I can even showhoped improvements on a student-by-student basis. This could reallybe considered a Formative Evaluation for next year. Based upon very

preliminary data (and a very mechanical evaluation) from EducationalTesting Services (ETS, the SAT people), this will not be a DecisionMaking Evaluation as the average test scores are up in English.Therefore I will be conducting a Goal-Based Evaluation to try todetermine which portions of the preparation might be improved toincrease scores next year. If this proves successful, perhaps theallotted time could be split with Math and as now Science has a MaineOnly SAT achievement test. Results of this evaluation will be


2/4

beneficial to administrators that initiated this effort, guidancecouncilors and English teachers.


3/4

k. Data Sources for SAT Preparation Evaluation

As stated in f. above, I will be evaluating recent SAT data. Themost recent group of students to take the test completed a directedstudy program once a day for three weeks prior to the testing. Test

scores will be compared to past SAT scores (from other classes) and tothis particular groups PSAT scores from last year.

The primary source of the information will come from averagescores of students from past years of SAT testing. There are at leasttwo significant problems when comparing values from year to year; theexam changes and the curriculum for students change. A significantchange in the scoring occurred four years ago when a third score, forwriting was introduced. Even though the writing portion is always firstand does not take much time, it allows less time for the other sections,therefore the test scores can only be compared for the past four years.Also, ETS does change the exam slightly every year to make it morecurrent and in line with what colleges are looking for.

Secondly, as the years go by, students taking even the sameclasses, are exposed to different curriculum (even the classes thatstudents take before their junior year change). As my state haschanged the required standards two times in the last four years,students have had a different exposure to the same core areas.Despite all that, math is math and english is english. Many studieshave been done and if one does not try to do too much with the data,they are more or less similar for year to year.

Perhaps the best data (while not absolute) will be to look atresults for individual students between their sophomore and junioryears or between the PSAT and the SAT results. Similarly, the twotests are not meant to be identical, so direct correlations cant bedrawn. The PSAT (Preliminary SAT) is similar enough that students arenever surprised with the SAT testing, format or even types of questionson the SAT. The tests are used for different purposes which does notsupport comparison. However, studies also support that correlationscan be drawn.

The year-to-year SAT comparison is the easiest to get data for as

it is easily downloaded from several locations and is only available inan average score format. Some additional information is alsoavailable (class size, % of class taking test and a few others), it haslittle impact.

I will be looking at the quantitative data on a student-by-studentbasis between the PSAT and SAT scores. Currently the data is onlyavailable in a .pdf format but it is hoped prior to the July deadline, a


4/4

huge spreadsheet will be transferred that documents studentresponses on a question-by-question basis. This would be veryvaluable for the English teacher but may be inappropriate for thisevaluation (almost too much data) other than its useful format.Lastly, students were questioned after the testing and nearly all said

they thought the additional sessions were beneficial. Preliminaryresults concur with that but I will evaluate that.

Documents

Goal Based Review - SAT vs. PSAT Comparison