Upload
marv
View
27
Download
0
Embed Size (px)
DESCRIPTION
Evaluation & Assessment in the Undergraduate Curriculum. Kris Stewart NPACI Education Center on Computational Science and Engineering San Diego State University http://www.edcenter.sdsu.edu/ Professor, Computer Science http://www.stewart.cs.sdsu.edu/. Assessment as Collaboration. - PowerPoint PPT Presentation
Citation preview
ASCP Workshop 04June04 LA076 SDSU
Evaluation & Assessmentin the Undergraduate Curriculum
Assessment as Collaboration
Kris Stewart
NPACI Education Center on Computational Science and Engineering
San Diego State University
http://www.edcenter.sdsu.edu/
Professor, Computer Sciencehttp://www.stewart.cs.sdsu.edu/
ASCP Workshop 04June04 LA076 SDSU
What is Computational Science?
Science DisciplinePhysics, Chemistry, Biology, etc.
Computer ScienceHardware/Software Applied Mathematics
Numerical Analysis, Modeling, Simulation
Computational ScienceTeamwork and Collaboration
ASCP Workshop 04June04 LA076 SDSU
What is Computational Science?Recognized as separate discipline
E.g. Society for Industrial and AppliedMathematicshttp://www.siam.org/cse/and Osman Yassar,IEEE Computing in Science & Engineeringhttp://www.cps.brockport.edu
Should CSE be a separate major?(Computational Science [no, too general] or
Computational Physics [yes, natural extension])Should it be a separate department?
(How to support interdisciplinarity?)
ASCP Workshop 04June04 LA076 SDSU
• Numerical Analyst* led to• Supercomputing and Undergraduate Education
(SUE**) led to• Supercomputing Teacher Enhancement Program
(STEP***) led to• Education Center on Computational Science &
Engineering (EC/CSE)
Kris’ Faculty Background(Kris Stewart, Director, San Diego State University, California State University System)
*MS/CS SDSU 1979, JPL 1981, PhD UNM 1987, SDSU 1986** UCES (DoEnergy 1994) *** Smithsonian Research Collection (1996)
ASCP Workshop 04June04 LA076 SDSU
UCES Paradigm(thanks Tom Marchioro and the “crew”, 1994?)
Previous exposure to “assessment”
ASCP Workshop 04June04 LA076 SDSU
1998/99 Assessment by LEAD
Background• Workshop in Wisconsin April 1997 to learn
about assessment and make it real• NPACI starts 1 October97• EC/CSE requested assessment for 1998
Assessment as a Collaboration
Preparation for SDSU Campus Visit by Baine & Julie• Discussions at SC98- SuperComputing Fall98 Orlando• Email to SDSU faculty gather attitudes
ASCP Workshop 04June04 LA076 SDSU
Grand Challenges for HPC (Stewart & Zaslavsky, SC98)
1. Faculty system of rewards does not encourage teaching innovations
2. Lack of awareness of HPC technologies already used in research or teaching for different fields
3. Faculty & students unaware of benefits and accomplishments of HPC
4. HPC technologies considered too complex/inaccessible for undergraduate instruction
5. Sequential HPC-related curricula is absent 6. Curricula using very large data sets not widely available 7. Adjust to different learning styles when material is complex 8. Variety of platforms/software leads to fragmented curricula 9. School administration/support staff not ready for HPC 10. Specs of computers and networks below user expectations
We had been thinking about this (based on April 97 LEAD Workshop in WI)
ASCP Workshop 04June04 LA076 SDSU
Building the Community of Faculty
• These challenges are people-centric, not technology-centric and of interest to the community
• Systemic Change requires understanding the system and working within it
• Empower teachers (find the time), recognition (from chair/dean), support (student assistants)
ASCP Workshop 04June04 LA076 SDSU
Interviews on SDSU CampusLEAD applying info from email surveys (Spr 99)
Faculty skepticism:
• Convincing evidence that computer-based tools enhance teaching process?
• Knowledge of modern computational methods and availability?
• Incentive from department and insufficient tech support?
LEAD Interviews with V.P., Deans, Chairs
Faculty Fellows program identified as a target
OUTCOMES:
• our local infrastructure at SDSU took us more seriously
• survey instruments refined
• SWB value recognition
ASCP Workshop 04June04 LA076 SDSU
Assessment not just requirement
Rather, found to be• vital tool to assist in clarifying student and faculty
needs• improve prioritization skills• validation of focus on human factors to integrate
HPC (modeling & visualization) into undergrad curriculum
ASCP Workshop 04June04 LA076 SDSU
California Education Infrastructure
• K12 Education (standards based, performance based)
• Community Colleges (Freshman/Sophomore)– Vocational– University preparation
• California State University System (23 campuses)• University of California (9 campuses, Merced in
2004)• Independents
(Stanford U., CalTech, U. Southern California)
ASCP Workshop 04June04 LA076 SDSU
California Education Infrastructure(testbed for change)
IMPAC http://www.cal-impac.org/Intersegmental Major Preparation Articulated Curriculum ProjectCommunity Colleges and Four Year Universities
Charter Schools (Preuss School) preuss.ucsd.eduDr. Rozeanne Steckler and Dr. Mike Bailey“Fostering Scientific Curiosity in All Children”San Diego Supercomputer Center
ASCP Workshop 04June04 LA076 SDSU
Involving University FacultyInfrastructure for Change
• NPACI/SDSU Faculty FellowsLocal Support from College Deans and Department Chairs (participation buy-in and faculty recognition)
• SDSU Academic Advisors (across disciplines)• Professional Meeting (SC2001, SIAM, ACM,
SIGCSE, your suggestions?)
ASCP Workshop 04June04 LA076 SDSU
Grand Challenges for HPC (Stewart & Zaslavsky, SC98)
1. Faculty system of rewards does not encourage teaching innovations
2. Lack of awareness of HPC technologies already used in research or teaching for different fields
3. Faculty & students unaware of benefits and accomplishments of HPC
4. HPC technologies considered too complex/inaccessible for undergraduate instruction
5. Sequential HPC-related curricula is absent 6. Curricula using very large data sets not widely available 7. Adjust to different learning styles when material is complex 8. Variety of platforms/software leads to fragmented curricula 9. School administration/support staff not ready for HPC 10. Specs of computers and networks below user expectations
ASCP Workshop 04June04 LA076 SDSU
Building the Community of Faculty
• These challenges are people-centric, not technology-centric and of interest to the EPSCOR community
• Systemic Change requires understanding the system and working within it
• Empower teachers (find the time), recognition (from chair/dean), support (student assistants)
ASCP Workshop 04June04 LA076 SDSU
Faculty Fellows Fall 01 Synergyamong themselves and with their chairs and deans
People, Time, Support, Recognition …
ASCP Workshop 04June04 LA076 SDSU
NSF/EHRNational Science
Foundation/Education and Human Resources
Directorate
www.ehr.nsf.gov/EHR/RED/EVAL/handbook/
handbook.htm
LEADAssessment and
Evaluation1998 Formative for the
EC/CSEhttp://www.cae.wisc.edu/~lead/pages/produts/eot-
paci.pdf
ASCP Workshop 04June04 LA076 SDSU
EC/CSE involve CSU
Faculty Workshops
• Introduce Computational Science
• Host Biology Workbench (BioQuest)
Support at NPACI AHM02
Extend Faculty Fellows
ASCP Workshop 04June04 LA076 SDSU
Contact Us
Kris Stewart, Directorhttp://www.edcenter.sdsu.edu
[email protected] Sale
[email protected] Kirsten Barber
ASCP Workshop 04June04 LA076 SDSU
The Ed Center Project and its Evaluation
• Goal: incorporate computational science techniques in undergraduate curricula
• Differences from STEP: target audience, institutional arrangements
• Methods and Strategies:• Faculty Fellows• Workshops, presentations, in-house projects, trying out new
approaches in own teaching, tools development• Computational Science Olympics (curriculum from the bottom-
up!!)• Surveys, continuous assessment of faculty involvement and
learning outcomes• Evaluated by the LEAD Center in 1998-99
ASCP Workshop 04June04 LA076 SDSU
Introducing the EC/CSE
The mission of the Ed Center on Computational Science and Engineering? www.edcenter.sdsu.edu
Who are the people involved? www.edcenter.sdsu.edu/staff
Some of our projects: www.edcenter.sdsu.edu/projects/
Some of our activities: www.edcenter.sdsu.edu/news/
Some resources: www.edcenter.sdsu.edu/repository
ASCP Workshop 04June04 LA076 SDSU
Undergraduate Faculty: A Tough Target Group
• Obstacles: lack of time, tenure and review considerations, lack of awareness about available technologies
• Undergraduate faculty: – ¾ have used WWW often or sometimes (1997), but not in the
classroom (only 18% - 1998)– The gap between those NEVER using computers in the classroom,
and those using them OFTEN, is the largest for untenured faculty, increasing towards tenure review
– Only 12% of surveyed faculty saw themselves as having a use for HPC applications in courses (higher for Sciences and Engineering)
– 11% of faculty have students working with computer models OFTEN
ASCP Workshop 04June04 LA076 SDSU
Using computers in the classroom versus number of years as a faculty member (1997 Faculty Survey)
ASCP Workshop 04June04 LA076 SDSU
Primarily graduate faculty Primarily undergraduate faculty
College of Arts & Letters College of Sciences
1. 1 OFTEN2. 2 SOMETIMES3. 3 RARELY4. 4 NEVER5. 9 DK/REF (Missing values)
Students Using Computers in the Classroom (1997 Faculty Survey)
ASCP Workshop 04June04 LA076 SDSU
Strategies for Building Faculty Community
• Reliance on most enthusiastic and technically advanced instructors who are already using computing and modeling in classes
• The Faculty Fellows program:• Stakeholders:
– College Deans - Specific support
– Faculty - Compensation, and acknowledgement, of the value of the faculty members contribution
• Benefits – College
– Department (Faculty Fellows as discipline-specific spokespersons for EC/CSE)
– Faculty (as individuals)
– Ed Center on Computational Science and Engineering
– Building a special infrastructure for curriculum transformation: human, institutional, technical – is a requirement for successful introduction of advanced techniques (since they are more demanding on faculty time and efforts)
The problems and strategies are not that much different from STEP!!
ASCP Workshop 04June04 LA076 SDSU
Faculty Fellows during 1998-2000
• Faculty Fellows representing four departments from four colleges: Geological Sciences, Geography, Computer Engineering, Business Information Systems
• Bi-weekly meetings at the Ed Center• Faculty Fellows as “ambassadors” of computational
science• Partnership with LEAD for evaluation during 1998-99• Follow-on Activities (Susan Millar, LEAD)
– CATS (Classroom Assessment Techniques)– FLAG (Field-tested Learning Assessment Guide)– SALG (Student Assessment of Learning Gains)
ASCP Workshop 04June04 LA076 SDSU
More Strategies
• Trying the new approaches in our own teaching:– Teaching the Supercomputer class with group-based
problem-solving environments
– Real-time distance teaching with Web-based collaborative software (featured as Microsoft Case Study in Higher Ed.)
– Development of Sociology Workbench, a free on-line survey data analysis system that can be used for evaluation of student outcomes and other surveys
ASCP Workshop 04June04 LA076 SDSU
Channels for Influencing Pre-Service Teacher Preparation
• Use of advanced computing modules in general ed courses (e.g. Geol 303 “Natural Hazards”)
• Cooperation with College of Education faculty and students, esp. in Education Technology: focus on experimentation with new technologies in classroom setting
• Computational Science Olympics: supporting the “bottom-up” development of computational science curricula
• Providing on-line assessment technologies:– Sociology Workbench: http://edcenter.sdsu.edu
ASCP Workshop 04June04 LA076 SDSU
SWB Convenient Tool to Learn from Student Survey Data
• Online tool for “standard public data sets” or your own data set http://edcenter.sdsu.edu
• Small Sample, therefore only useful as feedback for the instructor
• Can be used with “forms” interface directly into SWB format, as in June ‘99 CSU Faculty Workshop
ASCP Workshop 04June04 LA076 SDSU
SWB as Analysis ToolView Student Comments (text)
•
ASCP Workshop 04June04 LA076 SDSU
SWB as Analysis ToolIsolate on Specific Survey Response
•
ASCP Workshop 04June04 LA076 SDSU
•
SWB as Analysis ToolExplain the Response on Learning with “doing more”
ASCP Workshop 04June04 LA076 SDSU
SWB as Analysis ToolExplain learning with “active participation”
ASCP Workshop 04June04 LA076 SDSU
Lessons Learned
• Institutional support required for program to be sustainable
• Individual reform-ready faculty is focus for support
• Infrastructure:– Build a Synergistic Environment (across disciplines)
for Faculty
– Continuous monitoring through interviews, surveys, discussions
ASCP Workshop 04June04 LA076 SDSU
What’s Next
• The approaches we described proved useful for two target audiences. We believe that the strategies and lessons learned can be extrapolated in a targeted effort to incorporate computational science technologies in pre-service teacher preparation
• This may be a funding opportunity?• We will be happy to contribute our knowledge and
share experiences…
ASCP Workshop 04June04 LA076 SDSU
References ASCP Workshop
•User-Friendly Handbook for Project Evaluation: Science, Mathematics,
Engineering and Technology Education, NSF 93-152 www.ehr.nsf.gov/EHR/RED/EVAL/handbook/handbook.htm
•Sid Karin: The Importance of Science Literacy in a Computing World (see
enVision Science Magazine, V.15 No. 2
•K. Stewart & I. Zaslavsky, “Ten Grand Challenges”, Supercomputing ’98 Orlando FL SC98 Conference Paper
•J. Foertsch & B. Alexander, “Integrating HPC into the Undergraduate Curriculum”, report by LEAD Center June 1999 http://www.cae.wisc.edu/~lead/pages/products/eot-paci.pdf
ASCP Workshop 04June04 LA076 SDSU
Evaluation & Assessment
• U. Wisconsin LEAD for 1998/99 Ed Center evaluation
• www.cae.wisc.edu/~lead/pages/products/eot-paci.pdf
• Follow-on Activities (Susan Millar, LEAD)CATS (Classroom Assessment Techniques)FLAG (Field-tested Learning Assessment Guide)SALG (Student Assessment Learning Guide)
ASCP Workshop 04June04 LA076 SDSU
Evaluation and Assessment of Classroom Practice
Where to start?
• User-Friendly Handbook for Project Evaluation: Science, Mathematics, Engineering and Technology Education, NSF 93-152 www.ehr.nsf.gov/EHR/RED/EVAL/handbook/handbook.htm
• Student Surveys - Need a compatible tool for instructor to examine results with
• Sociology WorkBench (SWB) developed by team of undergraduate computer science majors employed by the EC/CSE
ASCP Workshop 04June04 LA076 SDSU
Information Overload
• How can teachers help out?
• How can technology providers help out?
• Ans: personalize the linkage.
ASCP Workshop 04June04 LA076 SDSU
Information Overlab Lab
• Using the WWW extensively in the classroom, implies obligation to discuss Information Overload
• http://www.stewart.cs.sdsu.edu/infolab/index.html
Timeline of Technology
http://www.stewart.cs.sdsu.edu/infolab/timeline_tech.html
ASCP Workshop 04June04 LA076 SDSU
Computer Network Security
• Campus-wide “buy-in” recognizing
the use of high-speed networks
the responsibility for “secure network use”
• SDSU Academic Senate adopts interim policy• UCSD Acceptable Use Policy useful start
ASCP Workshop 04June04 LA076 SDSU
A Little History - May 10, 1869
• What was the major event?
• Http://memory.loc.gov• Thank you to the US Library of Congress for the
on-line exhibit preserving our “Memory”
ASCP Workshop 04June04 LA076 SDSU
ASCP Workshop 04June04 LA076 SDSU
Promentory Point, UTAH
•
ASCP Workshop 04June04 LA076 SDSU
How does this relate to us today?
ASCP Workshop 04June04 LA076 SDSU
Technologies to Motivate Learning
• Mr. Sid uses Wavelets for compression of Railroad Maps
• Sense of History must be tied to personal, student- based, “stories”
• University Faculty need encouragement this will “count” for professional recognition by their peers and administrators (Deans…)
ASCP Workshop 04June04 LA076 SDSU
ONLINE Resources for Undergrads
• The Education, Outreach and Training (EOT) of the PACIs http://www.eot.org
• feature several projects
• Ed Center on CSE • Shodor
Undergraduate Computational Physics www.orst.edu
• Minority Institutions, a Directory
• Student Biology Workbench (Bioinformatics)
• Underrepresentation in Computer Science (CDC)
ASCP Workshop 04June04 LA076 SDSU
Learning Technologies from the PACIs
• Data Intensive Computing Environment
• Digital Video/eTEACH
• Interaction Environments: Bays to Brains
• Neuroscience: Telescience/Tomography
• NISE College Level One Institute on
• Learning Technology
• Remote Collaborative Learning
•www.eot.org
ASCP Workshop 04June04 LA076 SDSU
This is Only the Beginning...
YOU ARE HERE
TIME
ASCP Workshop 04June04 LA076 SDSU
What’s Next
• Your comments and thoughts?
• SIAM Computational Science & Engineering Conference – Sept. 2000 – Washington D.C.
• SC2000 – November 2000 – Dallas TX
ASCP Workshop 04June04 LA076 SDSU
ASCP Workshop 04June04 LA076 SDSU
• NPACI Partnership Report• enVision quarterly science
magazine, especially June2000“Why the Future May Very Well Need Us”
• www.npaci.edu/enVision/director.html
• “Online” biweekly electronic publication, www.npaci.edu/online/
• www.npaci.edu
NPACI Sources of Information
ASCP Workshop 04June04 LA076 SDSU
References - EPSCoR - July 2000• This Presentation• http://www.stewart.cs.sdsu.edu/PPT/
epscor_20july00.ppt• Ed Center on Computational Science & Engineering• http://www.edcenter.sdsu.edu/
• Timeline of Technology Development
• http://www.stewart.cs.sdsu.edu/infolab/timeline_tech.html
• Education, Outreach and Training
• http://www.eot.org• Information Overload • http://www.stewart.cs.sdsu.edu/infolab/index.html