ARE APTITUDE TESTS AN ACCURATE PREDICTOR FOR EXAM SUCCESS?

Sources and terms used: Differential Aptitude Tests usually known as “DAT’s”. Educational Aptitude “EA”, Numerical Reasoning “NR”, Verbal reasoning “VR” Junior Certificate results referred to as “JC” State Exam Commission Statistics referred to as “SEC”

Similar to many schools, our school administered a Differential Aptitude test (DAT) in our Junior Cert year. Differential Aptitude Tests (DATs) were developed in the US, but have been standardised for use in Irish schools on over 10,000 post-primary students.

The DATs are used by school counsellors, with assessing the intellectual characteristics ,educational or vocational aptitudes .

The Educational Aptitude “EA” score is derived from a combination of the Verbal Reasoning “VR” and Numerical Reasoning “NR” portion of the aptitude tests. It is taken as a general indication of your ability to learn in a traditional academic environment such as a school, college or university.

They are widely recognised and have been acknowledged by certain sites such as careersportal.ie, Ireland’s national careers website.

We wanted to investigate whether DAT is a good predictor for exam success by comparing the performance of students in VR, NR and EA in the DAT tests with their Junior Certificate Results. We also compared these results to the State Exam Commission “SEC” statistics for 2013 using 59,000 students results.

Our sample population was 118 female students who sat the exams in 2013.

DAT tests use Percentiles and Staines to show the student’s performance relative to his/her peer.

Percentile: e .g. Out of 100 students in an exam Sean scores 90%. This might sound like a fantastic achievement, but if 20 of the other candidates scored 95% it puts Sean's score of 90% in a new perspective. A percentile score indicates the number of students who scored better than the Sean in the exam. Sean's mark of 90% falls in the 80th percentile, as 20% of the students achieved a greater mark than him.

Stanine is a number between 1 to 9 given to the students based on their percentile score. This then allows us to group students into categories such as average, above average etc. It also allows us to compare how students performed in different tests in a meaningful way, such as the DAT aptitude tests and the Junior Certificate.

We are aware that the results may be skewed as our sample population may have performed particularly well in English, so we put our theory to the test again, this time looking at maths.

Numerical Reasoning measures the abilities of the students to perform mathematical reasoning tasks.

After applying the same process and graphing our results we found that the students once again outperformed in the Junior Certificate compared with the prediction of the DAT test.

5. NUMERICAL REASONING

8. CONCLUSIONS

Our aim for this project was to see if the DAT aptitude tests, particularly the EA portion of the test is a good predictor for exam success.

Our research and statistical analysis on our data pool indicates that a large portion of our sample population performed significantly better in the Junior Certificate than there test results would have predicted.

There are many factors that may cause the student not to perform well on the day of the DAT tests e.g. illness but this cannot explain the significant number of students who performed much better in the Junior Certificate.

There are however a number of factors which the DAT tests do not take into account such as perseverance or determination, bereavement and any psychological well-being or any kind of trauma.

Overall we believe our study shows that the DAT aptitude test is not a good predictor for academic success.

We hope to do a further study in the future to see if there is a common factor among the study who significantly over performed in the Junior Certificate compared to the DAT test.

6. STATE EXAM COMMISSION

Up to this point, we have just been comparing the DAT stanines to the school stanines. Since a stanine is based on a student’s performance relative to his/her peers, our school stanines are of a higher standard than the national average. Below is a graph examining the school’s performance in Junior Cert English and Maths compared against the SEC national statistics. In the graph we can see that 70% of the students in our school took Higher Level Maths compared to only 50% nationally per the SEC.

We began by examining the Verbal Reasoning ‘VR’ portion of the DAT aptitude tests.

VR measures a students ability to see relationships among words and provided us with a prediction for what grades the students should receive in English in the Junior Certificate ‘JC’

We converted the JC results into a percentile, applied a stanine to them which provided a basis to compare the JC and VR.

Percentile Stanine Comment97-99 9

90-96 8

78-89 759-77 6

41-58 5

23-40 412-22 3

5-11 2

4 and below 1

Below average

Above average

Average

Comparing DAT results to the Junior Certificate Results:

From our pie chart we can tell that 20% of the students received a B grade in the Junior Certificate, this means that these students are in the 79th percentile i.e. 21% of the students sitting the exam performed better than them.

After converting all of the Junior Certificate results into a percentile, we then applied a stanine to them which provided a basis for which the Junior Certificate results could be compared to the DAT results on a like to like basis using the DAT stanine and the Junior Certificate stanine.

As we mentioned earlier, according to the DAT test a student’s educational aptitude gives the best indication of their ability to learn in a traditional academic environment such as a school, college or university.

We are aware that up to this point we have been comparing our school on a whole compared to the whole DAT results and not individually.

In our next graph we individually looked at all of the students calculating whether or not they matched their predicted DAT score with their Junior Certificate results.

We developed our study further by looking at those who over performed compared to their DAT results and the range in which they over performed by.

We then examined those who over performed by a margin greater than 30 percentile.

1. INTRODUCTION 2. WHY THIS QUESTION?

2. METHOD HOW WE CONDUCTED OUR ANALYSIS

3. VERBAL REASONING

4. VERBAL REASONING CONTINUED

7. EDUCATIONAL ABILITY