Do UNEB results predict competencies required to excel academically in law

school?

ByRobert Wamala (Ph.D)

School of Statistics and Planning, CoBAMs; Email - rwamala@isae.mak.ac.ug

The dean, school of law writes: “There has

been a mismatch between entry grades and the

performance during the LL.B program and the

legal profession” (Makerere University, 2011,

p.1)

PRESS RELEASE: Pre-Entry Exams for LL.B Entrants

Students are admitted with triple or even quadruple “As”, which unfortunately

is in many cases, not reflected in their performance in law school.

There is overwhelming evidence in support of successful performance of enrollees who

have excelled academically in the past. This evidence is supported by studies across:

1. Disciplines

• Accounting and business economics (Duff, 2004; Alan & Othman, 2005)

• Business management and national statistics (Halpern, 2007; Hoskins, Newstead

& Denni, 1997)

• Nursing (Navarro, Vitamog, Tierra, & Gonzalez, 2011) and Actuarial Science

(Wamala, 2013)

2. Education Levels

• Undergraduate (Alan & Othman, 2005 ; Halpern, 2007 ; Wamala et al, 2013)

• Graduate (e.g., Navarro et al., 2011; Gregory, 2004 )

Relevance of prior studies

Competencies for undertaking LawInstitution GuidelinesUniversity of Texas - Undergraduate catalogue [2012/2014] Enrollees of law should be able to demonstrate proficiency in:

communication - writing in English, critical proficiency in oral and graphic communication

Conceptual approaches and history of arts – ability to comprehend factual concepts and human creativity

Political and economic dimensions of a society Cultural diversity including nature and limits of knowledge

and academic fields.

Alabama State Bar [Association of lawyers in the USA] Importance of enrollees’ grounding in the disciplines:

Analytical writing, English and literature, Political science, Economics and accounting, History, philosophy, logic, scientific methods Public speaking

Competencies for undertaking Law [Cont.’]Institution Guidelines

John Hopkins University [The parent advising hand book] Competence in economics, history, political thoughts and

mathematics provides a basis for building enrollees’ grounding regarding knowledge in resolving disputes.

University of Canterbury Importance of language and writing competencies: Enrollees with science, mathematics, music or art

background will only succeed in law school if they possess good language and writing skills

Makerere University [Guidelines for all public Universities] There has been a shift in the requirements in recent past:

Up until early 2000s, subjects namely History, literature and Divinity were considered essential

Subsequently, any subjects done by enrollees in their A-Level were considered essential (MoES, 2005; 2006; 2007)

From 2012 onwards, outcome of pre-entry tests are adopted as a basis for admission

To investigate the relevance of prior studies in predicting competencies required to

excel academically in law school. Focus was made on A-Level grades obtained in

disciplines, namely, History, Divinity, Literature, Economics and Geography.

Objective of the study

Data and Methods

• Administrative records of 629 graduates in the enrollment cohorts 2005 to 2007

– over the four stipulated period of study – were adopted [N = 2485]

• Academic achievement [dependent] was assessed by CGPA obtained in the first,

second, third and fourth year of study

• Investigations were made by: grades obtained in the A-level subjects and/or

weighted score; students characteristics namely, sex, cohort, entry scheme and

Nationality

Characteristics of enrollees [2005-2007 Cohorts]Students’ Characteristics Overall Between

N Percentage (%) N Percentage (%)

Sex

Female 1465 58.9 371 59.0

Male 1020 41.1 258 41.0

Total 2485 100.0 629 100.0

Year of Enrollment

2005 843 33.9 215 34.2

2006 887 35.7 225 35.8

2007 755 30.8 189 30.1

Total 2485 100.0 629 100.0

Nationality

Non-Ugandan 224 9.0 58 9.2

Ugandan 2261 91.0 571 90.8

Total 2485 100.0 629 100.0

Entry Scheme

Government 993 40.0 249 39.6

Private (Day) 1220 49.1 312 49.6

Private (Evening) 272 10.9 68 10.8

Total 2485 100.0 629 100.0

Weighted Score [on Admission]

N Mean Std. Dev Min Max

2023 51.8 3.04 44.1 56.5

Table 4: Summary statistics of the combined weighted score

The questionable aspect is the whether enrollees were competent for law school?

Note. The maximum possible score is 60

Mean weighted score suggests that the enrollees were highly competent in undertaking a bachelor’s study.

With regards to subjects done at A-Level

Academic Achievement

Year of Study Mean Std. Dev Min Max

First 3.01 0.33 2.20 4.00

Second 3.04 0.31 2.23 4.05

Third 3.12 0.30 2.23 4.13

Fourth 3.19 0.29 2.30 4.30

Table 1: Descriptive summary of academic achievement

A “Lower second” class of degree obtained by a considerable number of graduates on the program

Note. Estimates are based on CGPA

Predictors of academic achievement

Note. Assessment based on panel data approach – Random Effects Model (RE) using MLE

Independent Variables Model 1 Model 11

Coef. Std. Err p-value Coef. Std. Err p-valueNationality

Non Ugandans† 1.000 . . 1.000 . .Ugandans -0.067 0.040 0.096 -0.094 0.059 0.113

Year of Enrollment 2005 † 1.000 . . 1.000 . .

2006 -0.081 0.028 0.004 -0.099 0.033 0.0032007 -0.089 0.028 0.001 -0.158 0.031 0.000

Entry Scheme Government † 1.000 . . 1.000 . .

Private -evening -0.089 0.038 0.018 -0.117 0.055 0.033Private-day -0.055 0.042 0.198 -0.083 0.049 0.097

Sex Female † 1.000 . . 1.000 . .

Male 0.001 0.024 0.696 0.041 0.026 0.112Literature

N/A † 1.000 . . . . .A 0.101 0.036 0.005 . . .B 0.058 0.043 0.180 . . .C 0.030 0.059 0.608 . . .

D++ -0.017 0.095 0.860 . . .

Predictors of academic achievement [Cont.’]

Note. Assessment based on panel data approach – Random Effects Model (RE) using MLE

Divinity Coef. Std. Err p-value Coef. Std. Err p-valueN/A † 1.000 . . . . .

A 0.065 0.029 0.026 . . .B 0.029 0.036 0.409 . . .C -0.062 0.058 0.278 . . .

D++ -0.075 0.159 0.639 . . .Economics

N/A † 1.000 . . . . .A -0.122 0.094 0.196 . . .B -0.252 0.094 0.008 . . .C -0.330 0.095 0.001 . . .

D++ -0.317 0.098 0.001 . . .Geography

N/A † 1.000 . . . . .A 0.057 0.040 0.152 . . .B 0.042 0.039 0.276 . . .C 0.007 0.051 0.894 . . .

D++ -0.068 0.068 0.316 . . .History

N/A † 1.000 . . . . .A 0.214 0.085 0.012 . . .B 0.169 0.087 0.052 . . .C 0.140 0.097 0.150 . . .

D++ -0.092 0.211 0.665 . . .Weighted Score . . . 0.026 0.009 0.003constant 3.220 0.054 0.000 3.331 0.065 0.000

Significant variations in academic achievement (CGPA) were noted by enrollees

characteristics namely, nationality, enrollment cohort, and entry scheme (p < 0.05).

1. Students on government entry scheme had a higher CGPA compared to those on

the private evening arrangement

2. Enrollees in the 2006 and 2007 cohorts had a lower CGPA compared to those in

2005

Summary of the Findings [Student’s characteristics]

Could it be that students in the recent cohorts are increasingly taking on A-Level subjects they can easily pass rather than those they require to excel academically in law school ?

Significant variations in academic achievement were noted by performance in

disciplines namely literature, divinity, economics and history (p < 0.05).

1. Students who obtained grade A in literature had higher CGPA compared to their

counterparts who did not take the subject at A-Level

2. Students who obtained grade A in divinity had higher CGPA compared to their

counterparts who did not take the subject at A-Level

3. Students who obtained grades A and B in history had a higher CGPA compared

to those who did not take the subject at A-Level

4. Students who obtained grade B and below in economics had a lower CGPA

compared to those who did not take the subject at A-Level

Summary of the Findings [A – Level subjects]

Significant variation in academic achievement was noted by the weighted score

adopted on admission to law school(p < 0.01).

Summary of the Findings [Weighted score]

2.5

33

.54

4.5

Pa

rtia

l p

red

icto

r+re

sid

ua

l of cg

pa

45 50 55 60weight

Fractional Polynomial (1),adjusted for covariates

Line of best fit is not steep enough – weighted score weakly predicts CGPA

1. Using ANY best done of all A-Level subjects as “Essential subjects”

(MoES, 2005; 2006; 2007) to determine competence of

candidates to law school

2. The most competent candidates are the ones with the highest

weighted score adopted on admission to law school (MoES, 2005;

2006; 2007)

Implications

Questionable aspects

Performance in A-level subjects namely History, Divinity and Literature

predicts competencies required to excel academically in law school

Is the issue A-Level results or guidelines adopted on admission to law school ?

What about the pre-entry test to law school ?

Is performance in the test predicted by grades obtained in History, Divinity and Divinity ?

Students’ characteristics N Percentage (%)Gender

Female 253 31.0 Male 562 69.0 Total 815 100.0

Enrollment cohort 2012 461 56.6 2013 354 43.4

Total 815 100.0Nationality

Non-Ugandan 17 2.1 Ugandan 798 97.9

Total 815 100.0Entry scheme

Government 156 19.1 Private (Day) 511 62.7

Private (Evening) 148 18.2 Total 815 100.0

Prior qualification a A-Level 612 75.1

Bachelors’ degree 102 12.5Others 101 12.4

Total 815 100.0

Enrollees admitted on the basis of the test [2012-2013 Cohort]

Enrollees admitted on the basis of the test [Grades Obtained]A-Level subjects N Percentage (%)Literature

N/Aa 504 61.8A 99 12.2B 140 17.2C 63 7.7

D+ 9 1.1Total 815 100.0

Divinity N/A 378 46.4

A 130 16.0B 158 19.4C 99 12.2

D+ 50 6.1Total 815 100.0

Economics N/A 230 28.2

A 77 9.5B 148 18.2C 172 21.1

D+ 188 23.1Total 815 100.0

Note. N/A denotes student who did not do a subject at A-Level

Enrollees admitted on the basis of the test [Grades Obtained]

Geography N Percentage (%)N/A 610 74.9

A 51 6.3B 73 9.0C 59 7.2

D+ 22 2.7

Total 815 100.0

History N/A 238 29.2

A 112 13.7B 333 40.9C 112 13.7

D+ 20 2.5

Total 815 100.0

Contrary to enrollees admitted on the basis of their performance in A-Level, the

highest proportion of students obtained Grade B in the subjects.

Performance in the Test

Year of Study N Mean (95% CI) Min Max

First 379 2.65 (2.59 – 2.69) 0.60 3.90

Table 3: Descriptive summary of performance in the test

Figure is lower than FYCGPA of enrollees admitted using grades obtained in A-Level

Note. Estimates are based on First Year CGPA

Enrollment cohort N Mean (95% CI) Min Max

2012 446 59.2(58.6 - 59.8) 50 83

2013 348 67.1(66.4 - 67.8) 50 89

Academic AchievementTable 4: Descriptive summary of academic achievement [2012 Cohort]

Combination of admission test and Undergraduate GPA predicts first-year GPA in

law school (Anthony, Harris & Pashley 1999; Dalessandro, Stilwell, & Reese 2005;

Evans, 1984; Norton, Suto, & Reese, 2006; Wightman 1993)

Predictors of performance in the TestIndependent variables Model I a Model II b

Coef. Std. Err p-value Coef. Std. Err p-valueGender

Male† 0.00 . . 0.00 . .Female -1.09 0.204 0.000 -1.24 0.666 0.062

Enrollment cohort 2012† 0.00 . . . . .2013 7.72 0.206 0.000 . . .

Nationality Non Ugandans† 0.00 . . 0.00 . .

Ugandans 5.18 0.963 0.000 2.28 2.667 0.291Entry Scheme

Government† 0.00 . . 0.00 . .Private -evening -10.00 0.244 0.000 -10.36 0.859 0.000

Private-day -5.27 0.294 0.000 -4.59 1.044 0.000Prior qualification

A-Level 0.00 . . 0.00 . .

Bachelors’ degree 3.36 0.835 0.000 -1.03 2.040 0.612

Others -3.90 0.840 0.000 -5.49 2.035 0.007Literature

A† 0.00 . . 0.00 . .B -0.18 0.482 0.585 -0.24 1.129 0.826C 0.54 0.607 0.197 0.40 1.331 0.760

D++ -0.72 1.243 0.401 -1.63 2.246 0.467N/Ac -0.27 0.499 0.431 -0.54 1.225 0.657

Note. Assessment is based on a quantile (median) regression

Predictors of performance in the Test [Cont.’]Divinity Coef. Std. Err p-value Coef. Std. Err p-value

A† 0.00 . . 0.00 . .B 0.09 0.308 0.768 0.39 0.873 0.649C -0.36 0.365 0.321 -0.11 1.188 0.920

D++ 0.18 0.450 0.687 0.24 2.121 0.909N/A 0.90 0.329 0.006 0.54 0.997 0.582

Economics A† 0.00 . . 0.00 . .B -1.45 0.353 0.000 -2.11 1.144 0.066C -1.36 0.354 0.000 -2.01 1.171 0.086

D++ -1.27 0.368 0.001 -1.91 1.258 0.130N/A -1.18 0.690 0.088 -2.76 2.281 0.226

Geography A† 0.00 . . 0.00 . .B 0.18 0.462 0.694 0.95 1.769 0.589C 0.18 0.500 0.717 0.78 1.812 0.666

D++ 0.27 0.642 0.671 1.42 2.159 0.510N/A 1.09 0.418 0.009 0.75 1.616 0.640

History A† 0.00 . . 0.00 . .B -0.54 0.287 0.058 -1.65 1.284 0.199C -1.09 0.362 0.003 -2.05 1.415 0.148

D++ 1.00 0.630 0.113 0.49 1.962 0.799N/A -0.63 0.600 0.290 3.50 2.649 0.186

FYGPA . . . 0.70 0.522 0.180Constant 61.90 0.919 0.000 66.15 3.000 0.000

Performance in the test was HIGHER among:

1. Males (p < 0.01)

2. Ugandan nationals (p < 0.01)

3. Bachelor’s degree holders at enrollment compared to those with A-Level (p <

0.01)

Performance in the test was LOWER among:

4. Privately sponsored day and evening students compared to government

scheme (p < 0.01)

5. Students admitted on the basis of diploma and mature entry compared to those

with A-Level (p<0.01).

Summary of the Findings [Characteristics]

Performance in the test was HIGHER among:

1. Enrollees who did not do Literature compared to those with grade A (p < 0.05).

2. Enrollees who did not do Geography compared to those with grade A (p < 0.05).

3. Enrollees who obtained grade A in Economics compared to other grades (p <

0.01)

Performance in the Test was LOWER among:

4. Enrollees who obtained grade C in history compared to those with grade A (p <

0.01).

Summary of the Findings [A-Level Subjects]

Grades obtained in History, Divinity and Literature do not predict performance in the pre-entry test.

1. Relying heavily on the outcome of the test to admit students

under the government entry scheme (Makerere University, 2011)

2. Relying heavily on the outcome of the test to admit students to

law school

Implications

Questionable aspects

Aptitude tests predict inherent mental rather than educational or

academic ability as applied to law (LNAT Consortium, 2010; Law School

Admission Council, 2013; 2001)

Although candidates for the test are determined by a pooled index of attainment in A-Level (any subjects), admission to law school is mainly determined by the outcome of the test

1. The outcome of the admission test does not

predict competencies required to excel

academically in law school

2. Performance in A-Levels subjects namely, History,

Divinity and Literature predicts competencies

required to excel academically in law school

Conclusions

A pooled index of attainment in prior studies - using ANY subjects done at A-level - should not be used as a basis for determining candidates to sit the pre-entry test

1. To obtain the most academically competent candidates

to law school, admission requirements should focus on

performance in the A-Level subjects namely History,

Divinity and Literature

2. Requirements to sit the admission test should be based

on performance in the subjects rather than a pooled

index of attainment in prior studies i.e. A-Level

Recommendations

Problem is NOT UNEB results but the guidelines for admission to

law school