Assessing QL: Double Trouble

Northeast Consortium for Quantitative LiteracyAmherst CollegeApril 29, 2006

Plan for today

• What I mean by assessment• What I mean by QL• QL forces in US society• Why US colleges & universities need to

educate for QL• Challenges of education/assessment

for QL• Changes needed• Can one play the game?• Assessment items

What is assessment?

Comparing student learning with the learning goals of an academic program or curricular block of an academic program.

The Assessment Cycle

• What should our students learn?• How well are they learning?• What should we change so that

future students will learn more and understand it better?

Answers three questions:

Why is assessment done?

• The University administration mandated it.

• The governing board mandated it.• The legislature mandated it.• The accrediting agency mandated it.• To evaluate academic programs.• To evaluate students.

• To evaluate faculty performance.

What are better reasons for assessment?

• To improve courses.• To improve academic programs.• To improve teaching.• To enhance student learning.

Kinds of Assessment

• Classroom assessment• Formative assessment• Summative assessment• High stakes assessment• Outcomes-based assessment• Alternative assessment• Educative assessment• Assessment cycle• Authentic assessment

Assessment at different stages

• Entrance stage – placement testing

• Teaching and learning stage – formative feedback

• End/exit stage – final grades, etc.

Assessment Cycle – Brief Version

• Set learning goals and objectives.• Design strategies to accomplish objectives.• Determine areas and methods of

assessment.• Gather assessment data.• Use the assessment data to improve

program.• Do it again.

Of course, there are epicycles when they are warranted.

Assessment Methods

• Tests• Surveys• Evaluation reports• Portfolios• Essays• Capstone courses• Oral presentations

• Individual projects• Focus groups• Journals• Senior theses• Senior seminars• Employer feedback• Interviews

References on Assessment

• http://www.maa.org/saum (huge bibliography)• Assessment Practices in Undergraduate

Mathematics, MAA Notes # 49, 1999. (Bonnie Gold, Sandra Keith, & Bill Marion, Editors)

• Supporting Assessment in Undergraduate Mathematics, MAA Report, 2004. (Lynn Steen, Editor)

• Assessment of Student learning in College Mathematics: Toward Improved Programs and Courses, AIR, 2006. (Bernard L. Madison, Editor)

Plan for todayPlan for today

•What I mean by QLWhat I mean by QL

What is QL/Numeracy?

• QL to be the ability to understand and use quantitative measures and inferences that allow one to function as a responsible citizen, productive worker, and discerning consumer. (Madison - Notices of AMS)

• QL is the ability to understand and use numbers and data in everyday life. (Madison – QL:Why Numeracy Matters)

• QL is the ability to identify, understand, and use quantitative arguments in everyday contexts. QL describes a habit of mind rather than a set of topics or a list of skills. (Deborah Hughes Hallett)

• QL is the capacity to deal with the quantitative aspects of life. (Lynn Steen)

• Numeracy is the ability to handle numbers and data correctly and to evaluate statements regarding problems and situations that invite mental processing and estimating. (Jan de Lange)

• QL relates to a cultural field where language and quantitative constructs merge and are no longer one or the other. (Bob Orrill)

• Other (working) definitions of QL in:– Cockcroft Report (1982) – International Life Skills Survey (ILSS 2000)– Programme for International Student Assessment (PISA 2000).

• QL is the knowledge and skills required to apply arithmetic operations, either alone or sequentially, using numbers embedded in printed materials; for example, balancing a checkbook, figuring out a tip, completing an order form, or determining the amount of interest from a loan advertisement. (US NALS)

• “Numeracy” being replaced by “functional mathematics” in UK

Literacy has two very different meanings and this difference creates problems for QL education.

1)Literacy is often viewed as minimum competency as in “He is barely literate.”

2)Literacy is sometimes viewed as a high level of learning as in literati, or the educated class or intelligentsia.

Two Kinds of Literacy

• Inert - Level of verbal and numerate skills required to comprehend instructions, perform routine procedures, and complete tasks in a routine manner.

• Liberating - Command of both the enabling skills needed to search out information and power of mind necessary to critique it, reflect upon it, and apply it in making decisions.

Lawrence A. Cremins, American Education: The Metropolitan Experience 1876-1980. New York: Harper & Row, 1988. (as quoted by R. Orrill in M&D)

Relatives of QLRelatives of QL• Mathematical literacyMathematical literacy• NumeracyNumeracy• Symbol literacySymbol literacy• Document literacyDocument literacy• Financial literacyFinancial literacy• Statistical literacyStatistical literacy• Quantitative reasoningQuantitative reasoning• Critical reasoningCritical reasoning• Analytical reasoningAnalytical reasoning• Scientific literacyScientific literacy• Symbol literacySymbol literacy• Technical literacyTechnical literacy• EthnomathematicsEthnomathematics• Functional mathematicsFunctional mathematics

Kinds & Levels of QL

• Critical QL – liberating QL• Functional QL – inert QL• Adult QL• Aging adult QL• QL for journalists• School level QL• College level QL• QL for social change • Environmental QL• Cultural/vocational QL – e.g. street

vending, gambling, carpentry, real estate, & farming

Mathematics vs. QL

Mathematics• Power in abstraction• Power in generality• Some context dependency • Society independent• Apolitical• Methods & algorithms• Well-defined problems• Approximation• Heavily disciplinary• Problem solutions• Few opportunities to practice• Predictable

QL• Real, metamorphic contexts• Specific, particular• Heavy context dependency• Society dependent• Political• Ad hoc methods • Ill-defined problems• Estimation is critical• Interdisciplinary• Problem descriptions• Many practice opportunities• Unpredictable

Why Numbers Count, (Lynn Steen, Editor) The College Board 1997

Mathematics and Democracy, (Lynn Steen, Editor), NCED 2001

QL:Why Numeracy Matters for Schools and Colleges, (Bernard Madison and Lynn Steen, Editors), NCED 2003

Achieving QL (Lynn Steen), MAA 2004

Mathematical Literacy by Eva Jablonka, Second International Handbook of Mathematics Education, Kluwer Academic Publishers, 2003, 77-104

Peer Review, Summer 2004

Alfred W. Crosby, The Measure of Reality: Quantification and Western Society, 1250-1600, Cambridge, 1997

Peter L. Bernstein, Against the Gods: The Remarkable Story of Risk, John Wiley & Sons, 1996

Pat Cohen, A Calculating People: The Spread of Numeracy in Early America, University of Chicago Press, 1982

QL Publications

•QL forces in US societyQL forces in US society

Plan for Plan for todaytoday

QL Forces in US

Constitutional mandate for a census Free market system -- minimally

regulated labor markets Democratic accountability Representation of diverse interests --

minority representation Informing political debate -- how much

for how many Confirming national identity -- what’s

counted is what counts De-regulation – more freedom to choose Environmental protection and

occupational safety Emphasis on wealth accumulation Heavy use of rapidly developing

technology

Computers have made data easy to gather and analyze. Data and numbers occur in many areas of

daily life.

• Health care• Retirement plans• Rents and mortgages• Investing• Credit rates and

plans• Insurance• Wages and salaries• Education results• Consumer protection• Sports results

• Voting and polling• Laws - regulation & de-

regulation• Court decisions• Minority

representations• Taxes• Federal & state budgets• Economic policy• Political debates• Risk of terrorism• Election procedures

Individual Decisions Decisions as Citizens

•Why US colleges & Why US colleges & universities need to educate universities need to educate for QLfor QL

Plan for Plan for todaytoday

High School

College

High School and College Enrollments as a Percent of 14-17 &18-24 Age Cohorts (1900-1995)

College

High School

20% of 4-year college seniors basic or below basic and only 34% scored as proficient on the NAAL in QL.

AIR, 2006