58
Backward Design: An Integrated Approach to a Systems Curriculum SIGCSE 2015 • Kirkpatrick et. al Backward Design An Integrated Approach to a Systems Curriculum Michael S. Kirkpatrick Mohamed Aboutabl David Bernstein Sharon Simmons

Backward Design - w3.cs.jmu.edu

  • Upload
    others

  • View
    3

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Backward Design - w3.cs.jmu.edu

Backward Design: An Integrated Approach to a Systems Curriculum SIGCSE 2015 • Kirkpatrick et. al

Backward DesignAn Integrated Approach to a Systems Curriculum

Michael S. KirkpatrickMohamed Aboutabl

David BernsteinSharon Simmons

Page 2: Backward Design - w3.cs.jmu.edu

Backward Design: An Integrated Approach to a Systems Curriculum SIGCSE 2015 • Kirkpatrick et. al

Page 3: Backward Design - w3.cs.jmu.edu

Backward Design: An Integrated Approach to a Systems Curriculum SIGCSE 2015 • Kirkpatrick et. al

Page 4: Backward Design - w3.cs.jmu.edu

Backward Design: An Integrated Approach to a Systems Curriculum SIGCSE 2015 • Kirkpatrick et. al

Backward DesignThe Long and Rewarding Path to an ACM 2013 Systems Fundamentals Examplar

Michael S. KirkpatrickMohamed Aboutabl

David BernsteinSharon Simmons

Page 5: Backward Design - w3.cs.jmu.edu

Backward Design: An Integrated Approach to a Systems Curriculum SIGCSE 2015 • Kirkpatrick et. al

Backward DesignThe Long and Rewarding Path to an ACM 2013 Systems Fundamentals Examplar

Michael S. KirkpatrickMohamed Aboutabl

David BernsteinSharon Simmons

(ish)

Page 6: Backward Design - w3.cs.jmu.edu

Backward Design: An Integrated Approach to a Systems Curriculum SIGCSE 2015 • Kirkpatrick et. al

Context - how we got here

Not listed:• CS 139: Programming Fundamentals• CS 228: Discrete Structures II• CS 345: Software Engineering• CS 430: Programming Languages• CS 474: Databases

Page 7: Backward Design - w3.cs.jmu.edu

Backward Design: An Integrated Approach to a Systems Curriculum SIGCSE 2015 • Kirkpatrick et. al

Context - how we got here

Not listed:• CS 139: Programming Fundamentals• CS 228: Discrete Structures II• CS 345: Software Engineering• CS 430: Programming Languages• CS 474: Databases

Page 8: Backward Design - w3.cs.jmu.edu

Backward Design: An Integrated Approach to a Systems Curriculum SIGCSE 2015 • Kirkpatrick et. al

Context - how we got here

Not listed:• CS 139: Programming Fundamentals• CS 228: Discrete Structures II• CS 345: Software Engineering• CS 430: Programming Languages• CS 474: Databases

Cross-listed with ISAT

Page 9: Backward Design - w3.cs.jmu.edu

Backward Design: An Integrated Approach to a Systems Curriculum SIGCSE 2015 • Kirkpatrick et. al

Context - how we got here

Not listed:• CS 139: Programming Fundamentals• CS 228: Discrete Structures II• CS 345: Software Engineering• CS 430: Programming Languages• CS 474: Databases

Cross-listed with ISAT

Switch to kernel emphasis

Page 10: Backward Design - w3.cs.jmu.edu

Backward Design: An Integrated Approach to a Systems Curriculum SIGCSE 2015 • Kirkpatrick et. al

Context - how we got here

Not listed:• CS 139: Programming Fundamentals• CS 228: Discrete Structures II• CS 345: Software Engineering• CS 430: Programming Languages• CS 474: Databases

Cross-listed with ISAT

Switch to kernel emphasis

Originally exclusively

HW focused

Page 11: Backward Design - w3.cs.jmu.edu

Backward Design: An Integrated Approach to a Systems Curriculum SIGCSE 2015 • Kirkpatrick et. al

Page 12: Backward Design - w3.cs.jmu.edu

Backward Design: An Integrated Approach to a Systems Curriculum SIGCSE 2015 • Kirkpatrick et. al

Backward DesignFixing the Problems that Certain New Faculty Introduced when Hired

Michael S. KirkpatrickMohamed Aboutabl

David BernsteinSharon Simmons

Page 13: Backward Design - w3.cs.jmu.edu

Backward Design: An Integrated Approach to a Systems Curriculum SIGCSE 2015 • Kirkpatrick et. al

Backward DesignFixing the Problems that Certain New Faculty Introduced when Hired

Michael S. KirkpatrickMohamed Aboutabl

David BernsteinSharon Simmons

Page 14: Backward Design - w3.cs.jmu.edu

Backward Design: An Integrated Approach to a Systems Curriculum SIGCSE 2015 • Kirkpatrick et. al

Context - how we got here

Other problems:• CS 350 bottleneck• Insufficient scaffolding• Redundancies between siloes• Training vs. education• Aging curriculum

Page 15: Backward Design - w3.cs.jmu.edu

Backward Design: An Integrated Approach to a Systems Curriculum SIGCSE 2015 • Kirkpatrick et. al

Context - how we got here

Other problems:• CS 350 bottleneck• Insufficient scaffolding• Redundancies between siloes• Training vs. education• Aging curriculum

Page 16: Backward Design - w3.cs.jmu.edu

Backward Design: An Integrated Approach to a Systems Curriculum SIGCSE 2015 • Kirkpatrick et. al

Identifysituational

factors

DesignBackward

Page 17: Backward Design - w3.cs.jmu.edu

Backward Design: An Integrated Approach to a Systems Curriculum SIGCSE 2015 • Kirkpatrick et. al

Backward Curricular Design

Identifysituational

factors

Page 18: Backward Design - w3.cs.jmu.edu

Backward Design: An Integrated Approach to a Systems Curriculum SIGCSE 2015 • Kirkpatrick et. al

Backward Curricular Design

Identifysituational

factors

Starting points:• What are our goals for OUR students?• What are our strengths?• What do we want to improve?• What limitations and constraints exist?• What is backward design?

Page 19: Backward Design - w3.cs.jmu.edu

Backward Design: An Integrated Approach to a Systems Curriculum SIGCSE 2015 • Kirkpatrick et. al

Backward Curricular Design

Defineobjectives

Identifysituational

factors

Starting points:• What are our goals for OUR students?• What are our strengths?• What do we want to improve?• What limitations and constraints exist?• What is backward design?

Page 20: Backward Design - w3.cs.jmu.edu

Backward Design: An Integrated Approach to a Systems Curriculum SIGCSE 2015 • Kirkpatrick et. al

Backward Curricular Design

Summativeassessmentsand metrics

Defineobjectives

Identifysituational

factors

Starting points:• What are our goals for OUR students?• What are our strengths?• What do we want to improve?• What limitations and constraints exist?• What is backward design?

Page 21: Backward Design - w3.cs.jmu.edu

Backward Design: An Integrated Approach to a Systems Curriculum SIGCSE 2015 • Kirkpatrick et. al

Backward Curricular Design

Formativeassessments/instructional

activities

Summativeassessmentsand metrics

Defineobjectives

Identifysituational

factors

Starting points:• What are our goals for OUR students?• What are our strengths?• What do we want to improve?• What limitations and constraints exist?• What is backward design?

Page 22: Backward Design - w3.cs.jmu.edu

Backward Design: An Integrated Approach to a Systems Curriculum SIGCSE 2015 • Kirkpatrick et. al

Situational factors

Page 23: Backward Design - w3.cs.jmu.edu

Backward Design: An Integrated Approach to a Systems Curriculum SIGCSE 2015 • Kirkpatrick et. al

Situational factors

Show-stopper constraints• Three-section, two-prep teaching structure• Virginia Community College System (VCCS) relationship• Faculty interests and staffing

Page 24: Backward Design - w3.cs.jmu.edu

Backward Design: An Integrated Approach to a Systems Curriculum SIGCSE 2015 • Kirkpatrick et. al

Situational factors

Show-stopper constraints• Three-section, two-prep teaching structure• Virginia Community College System (VCCS) relationship• Faculty interests and staffing

Other considerations• Prolonged CS1 sequence (assumes no background)• Infosec/Cyber-defense emphasis and strength• Student career aspirations and proximity to DC• Courses have reputations• Liberal arts university - credit hour questions

Page 25: Backward Design - w3.cs.jmu.edu

Backward Design: An Integrated Approach to a Systems Curriculum SIGCSE 2015 • Kirkpatrick et. al

Defining objectives

Formativeassessments/instructional

activities

Summativeassessmentsand metrics

Defineobjectives

Identifysituational

factors

Page 26: Backward Design - w3.cs.jmu.edu

Backward Design: An Integrated Approach to a Systems Curriculum SIGCSE 2015 • Kirkpatrick et. al

Defining objectives

Formativeassessments/instructional

activities

Summativeassessmentsand metrics

Defineobjectives

Identifysituational

factors

Starting points for objectives:• What vision do we have for our students?• How do we make our vision explicit and measurable?• Where are our students starting from?• What should they learn beyond just CS?

Page 27: Backward Design - w3.cs.jmu.edu

Backward Design: An Integrated Approach to a Systems Curriculum SIGCSE 2015 • Kirkpatrick et. al

Taxonomies galore

Bloom’s

Page 28: Backward Design - w3.cs.jmu.edu

Backward Design: An Integrated Approach to a Systems Curriculum SIGCSE 2015 • Kirkpatrick et. al

Taxonomies galore

Expl

anat

ion

Interpretation

ApplicationPers

pect

ive

Empathy

Self-knowledgeBloom’s

SOLO (Biggs)

UbD(Wiggins &McTighe)

Fink

Page 29: Backward Design - w3.cs.jmu.edu

Backward Design: An Integrated Approach to a Systems Curriculum SIGCSE 2015 • Kirkpatrick et. al

Taxonomies galore

Page 30: Backward Design - w3.cs.jmu.edu

Backward Design: An Integrated Approach to a Systems Curriculum SIGCSE 2015 • Kirkpatrick et. al

Taxonomies galore

Affective Cognitive

Page 31: Backward Design - w3.cs.jmu.edu

Backward Design: An Integrated Approach to a Systems Curriculum SIGCSE 2015 • Kirkpatrick et. al

Defining learning outcomes

Foundational1. What key information should our students under-

stand and remember in the future?

2. What key ideas or themes are important for studentsto comprehend and describe?

Application

3. What kinds of thinking (critical, creative, practical)are important for our students to learn?

4. What software development skills do students needto gain?

5. What skills do our students need to learn about man-aging complex projects?

Integration6. What connections should students recognize and

make within the curriculum or to professional life?

Human dimension7. What should students learn about themselves

as software developers, users of technology, oras students?

8. What should students learn about understand-ing others and how they interact with others(including through developed software)?

Caring9. What changes and values should students

adopt in regard to interests or ideas?

Learning how to learn10. What should students learn about how to work

with computer systems?

11. What should students learn about becoming aself-directed learner?

Table 1: Guiding questions for defining learning outcomes

plete a degree in a timely manner. In our current struc-ture, the prerequisite depth (i.e., the minimum number ofsemesters from the first required course to the last requiredcourse) was 5, which is at the high end to offer a feasible pathfor timely completion. Thus, if our final proposal increasedthis depth, we would need to have a strong rationale and in-stitutional acceptance of the curriculum would be unlikely.Figure 1 shows the relevant prerequisite structure (which is asubset of our overall curriculum) for our systems core. Thecourses in green are offered every semester, those in blueare spring only, and those in brown are fall only. The twocourses in grey are electives, with CS 457 as a corequisitewith CS 450.Another situational factor arose from one of our depart-

ment’s strengths: As one of the seven original National Cen-ters of Academic Excellence in Information Assurance Ed-ucation (CAE/IAE), our department has a long history ofoffering multiple courses in information security and cyberdefense. These courses require completing the systems coreas prerequisites. Lengthening the duration of the core couldmake it more difficult for students to enroll in these courses,thus reducing the impact of a central departmental priority.Lastly, the career aspirations of our students created an-

other significant factor. Like many other institutions, ourstudents overwhelmingly go into industry positions ratherthan graduate programs, though some opt for the latter.Furthermore, given JMU’s proximity to Washington, D.C.,and our emphasis on information security, many of our grad-uates go into positions with the federal government. Thisbias toward industry and government is reflected in our over-all curriculum, which emphasizes applied software engineer-ing experience. Thus, our goal for the systems core was tocreate a foundation of systems knowledge for all studentswhile providing ample opportunities for exploring certainsystems in greater detail, particularly for students with aninterest in post-graduate work.

3.2 Systems Curriculum GoalsAs a starting point for our curriculum re-design, we ap-

plied the Fink taxonomy [3] of learning goals to enumer-

ate all desired outcomes without consideration of feasibility.That is, we began by asking what we felt all of our studentsneeded to achieve without regard to how much time wouldbe needed to achieve these goals. At this stage, we did notconsider how our goals would impact the prerequisite depthissues for VCCS students or our security courses.

To define our desired learning outcomes, we adopted theFink taxonomy [3]. This model defines 6 areas of learn-ing outcomes as shown in Table 1. In Fink’s model, 3 ar-eas (Foundational, Application, Integration) roughly corre-spond to the cognitive domain concerns underlying Bloom’staxonomy [2]. Fink’s model incorporates 3 additional areasthat emphasize affective and metacognitive questions thathave no direct mapping to counterparts in Bloom’s taxon-omy. We found defining learning goals for the cognitive ar-eas to be a rather straightforward endeavor, as it is commonpractice to define cognitive outcomes. Through casual dis-cussions, we had developed an informal understanding of ourgoals and values. However, enumerating them allowed us toidentify mismatches in expectations and assumptions, thusmaking it possible to come to a consensus on these issues.

The affective and metacognitive questions led to consid-erably more discussion. Most members of the group werenew to this process and unfamiliar with Fink’s work; con-sequently, we found it necessary to spend time coming toan agreement on how to interpret terminology (e.g., whatit means to be a “self-directed learner” in computing, whatconstitutes “caring,”what role should a CS department havein addressing the human dimension). In fact, disagreementsin later discussions could often be traced back to differentviews about these questions. Furthermore, standard cur-riculum descriptions provide considerably less guidance indefining such objectives. For instance, the ACM 2013 has ashort chapter describing “Characteristics of Graduates” at abroad level, but does not define specific measurable objec-tives similar to those defined in the Knowledge Areas. As aresult, we found ourselves spending a significant amount oftime grappling with this discussion, and others consideringthis process should be prepared for potential conflicts arisingfrom these questions.

Page 32: Backward Design - w3.cs.jmu.edu

Backward Design: An Integrated Approach to a Systems Curriculum SIGCSE 2015 • Kirkpatrick et. al

Defining learning outcomes

Foundational1. What key information should our students under-

stand and remember in the future?

2. What key ideas or themes are important for studentsto comprehend and describe?

Application

3. What kinds of thinking (critical, creative, practical)are important for our students to learn?

4. What software development skills do students needto gain?

5. What skills do our students need to learn about man-aging complex projects?

Integration6. What connections should students recognize and

make within the curriculum or to professional life?

Human dimension7. What should students learn about themselves

as software developers, users of technology, oras students?

8. What should students learn about understand-ing others and how they interact with others(including through developed software)?

Caring9. What changes and values should students

adopt in regard to interests or ideas?

Learning how to learn10. What should students learn about how to work

with computer systems?

11. What should students learn about becoming aself-directed learner?

Table 1: Guiding questions for defining learning outcomes

plete a degree in a timely manner. In our current struc-ture, the prerequisite depth (i.e., the minimum number ofsemesters from the first required course to the last requiredcourse) was 5, which is at the high end to offer a feasible pathfor timely completion. Thus, if our final proposal increasedthis depth, we would need to have a strong rationale and in-stitutional acceptance of the curriculum would be unlikely.Figure 1 shows the relevant prerequisite structure (which is asubset of our overall curriculum) for our systems core. Thecourses in green are offered every semester, those in blueare spring only, and those in brown are fall only. The twocourses in grey are electives, with CS 457 as a corequisitewith CS 450.Another situational factor arose from one of our depart-

ment’s strengths: As one of the seven original National Cen-ters of Academic Excellence in Information Assurance Ed-ucation (CAE/IAE), our department has a long history ofoffering multiple courses in information security and cyberdefense. These courses require completing the systems coreas prerequisites. Lengthening the duration of the core couldmake it more difficult for students to enroll in these courses,thus reducing the impact of a central departmental priority.Lastly, the career aspirations of our students created an-

other significant factor. Like many other institutions, ourstudents overwhelmingly go into industry positions ratherthan graduate programs, though some opt for the latter.Furthermore, given JMU’s proximity to Washington, D.C.,and our emphasis on information security, many of our grad-uates go into positions with the federal government. Thisbias toward industry and government is reflected in our over-all curriculum, which emphasizes applied software engineer-ing experience. Thus, our goal for the systems core was tocreate a foundation of systems knowledge for all studentswhile providing ample opportunities for exploring certainsystems in greater detail, particularly for students with aninterest in post-graduate work.

3.2 Systems Curriculum GoalsAs a starting point for our curriculum re-design, we ap-

plied the Fink taxonomy [3] of learning goals to enumer-

ate all desired outcomes without consideration of feasibility.That is, we began by asking what we felt all of our studentsneeded to achieve without regard to how much time wouldbe needed to achieve these goals. At this stage, we did notconsider how our goals would impact the prerequisite depthissues for VCCS students or our security courses.

To define our desired learning outcomes, we adopted theFink taxonomy [3]. This model defines 6 areas of learn-ing outcomes as shown in Table 1. In Fink’s model, 3 ar-eas (Foundational, Application, Integration) roughly corre-spond to the cognitive domain concerns underlying Bloom’staxonomy [2]. Fink’s model incorporates 3 additional areasthat emphasize affective and metacognitive questions thathave no direct mapping to counterparts in Bloom’s taxon-omy. We found defining learning goals for the cognitive ar-eas to be a rather straightforward endeavor, as it is commonpractice to define cognitive outcomes. Through casual dis-cussions, we had developed an informal understanding of ourgoals and values. However, enumerating them allowed us toidentify mismatches in expectations and assumptions, thusmaking it possible to come to a consensus on these issues.

The affective and metacognitive questions led to consid-erably more discussion. Most members of the group werenew to this process and unfamiliar with Fink’s work; con-sequently, we found it necessary to spend time coming toan agreement on how to interpret terminology (e.g., whatit means to be a “self-directed learner” in computing, whatconstitutes “caring,”what role should a CS department havein addressing the human dimension). In fact, disagreementsin later discussions could often be traced back to differentviews about these questions. Furthermore, standard cur-riculum descriptions provide considerably less guidance indefining such objectives. For instance, the ACM 2013 has ashort chapter describing “Characteristics of Graduates” at abroad level, but does not define specific measurable objec-tives similar to those defined in the Knowledge Areas. As aresult, we found ourselves spending a significant amount oftime grappling with this discussion, and others consideringthis process should be prepared for potential conflicts arisingfrom these questions.

Page 33: Backward Design - w3.cs.jmu.edu

Backward Design: An Integrated Approach to a Systems Curriculum SIGCSE 2015 • Kirkpatrick et. al

Systems core themesTo reflect trends within the CS field, the JMU CS systems core courses should strive to produce the following characteristics of CS graduates:

• technical understanding of computer and network systems• appreciation of the interplay between theory and practice• system-level perspective (thinking in levels of abstraction)• understanding of how to identify and evaluate trade-offs in design and

implementation• ability to identify common problem patterns and apply appropriate solutions• demonstrable experience with large software projects• commitment to individual skill development, such as learning new languages• commitment to professional responsibility• understand the differences between systems and application programming• understand the hardware characteristics that dictate the requirements/features

of the controlling software (e.g., sampling frequency, signal propagation delays, arithmetic precision)

Page 34: Backward Design - w3.cs.jmu.edu

Backward Design: An Integrated Approach to a Systems Curriculum SIGCSE 2015 • Kirkpatrick et. al

Systems core themesTo reflect trends within the CS field, the JMU CS systems core courses should strive to produce the following characteristics of CS graduates:

• technical understanding of computer and network systems• appreciation of the interplay between theory and practice• system-level perspective (thinking in levels of abstraction)• understanding of how to identify and evaluate trade-offs in design and

implementation• ability to identify common problem patterns and apply appropriate solutions• demonstrable experience with large software projects• commitment to individual skill development, such as learning new languages• commitment to professional responsibility• understand the differences between systems and application programming• understand the hardware characteristics that dictate the requirements/features

of the controlling software (e.g., sampling frequency, signal propagation delays, arithmetic precision)

Page 35: Backward Design - w3.cs.jmu.edu

Backward Design: An Integrated Approach to a Systems Curriculum SIGCSE 2015 • Kirkpatrick et. al

Systems core themesIn support of this goal, the following principles should guide the systems curriculum:

• open-ended programming assignments that require analysis, design, and testing• opportunities to apply systems concepts to realistic problems• a combination of individual- and group-based projects• experience both reading and writing code• exposure to standard industry tools and techniques where appropriate• exploration of the design considerations underlying existing systems software• emphasis on developing skills for independent learning• flexible curriculum that serves the needs of students with varying technical

talents• appropriate coverage of fundamental computer and networking concepts

Page 36: Backward Design - w3.cs.jmu.edu

Backward Design: An Integrated Approach to a Systems Curriculum SIGCSE 2015 • Kirkpatrick et. al

Systems core themesIn support of this goal, the following principles should guide the systems curriculum:

• open-ended programming assignments that require analysis, design, and testing• opportunities to apply systems concepts to realistic problems• a combination of individual- and group-based projects• experience both reading and writing code• exposure to standard industry tools and techniques where appropriate• exploration of the design considerations underlying existing systems software• emphasis on developing skills for independent learning• flexible curriculum that serves the needs of students with varying technical

talents• appropriate coverage of fundamental computer and networking concepts

Page 37: Backward Design - w3.cs.jmu.edu

Backward Design: An Integrated Approach to a Systems Curriculum SIGCSE 2015 • Kirkpatrick et. al

Systems core themesEvery JMU CS student who satisfactorily completes all required systems courses should be able to:

• Summarize the technical foundations of how software executes on hardware, how parallel and distributed software communicate, what patterns and structures are used to construct systems and concurrent programs, and what layered abstractions support modern computing systems.

• Explain how bits can represent information, how instructions and communication can be seen as a sequence of state transitions, how mathematical models can describe system behavior, and how reactive programming practices used in systems programming differs from other approaches.

• Read and understand existing protocols, critically evaluate existing protocols, and select appropriate protocols.

• Describe the ways in which information can be exchanged between different parts of an existing system, select appropriate ways to exchange information in proposed systems, and implement systems that exchange information.

• Describe the architectural style/high-level design of a system using appropriate terminology, and evaluate the architectural style/high-level design of existing and proposed systems.

• ...

Page 38: Backward Design - w3.cs.jmu.edu

Backward Design: An Integrated Approach to a Systems Curriculum SIGCSE 2015 • Kirkpatrick et. al

Systems core themesEvery JMU CS student who satisfactorily completes all required systems courses should be able to:

• Summarize the technical foundations of how software executes on hardware, how parallel and distributed software communicate, what patterns and structures are used to construct systems and concurrent programs, and what layered abstractions support modern computing systems.

• Explain how bits can represent information, how instructions and communication can be seen as a sequence of state transitions, how mathematical models can describe system behavior, and how reactive programming practices used in systems programming differs from other approaches.

• Read and understand existing protocols, critically evaluate existing protocols, and select appropriate protocols.

• Describe the ways in which information can be exchanged between different parts of an existing system, select appropriate ways to exchange information in proposed systems, and implement systems that exchange information.

• Describe the architectural style/high-level design of a system using appropriate terminology, and evaluate the architectural style/high-level design of existing and proposed systems.

• ...

Core Tier 1 vs. Core Tier 2

Page 39: Backward Design - w3.cs.jmu.edu

Backward Design: An Integrated Approach to a Systems Curriculum SIGCSE 2015 • Kirkpatrick et. al

Systems core themesEnduring Understandings• Information = Bits + Context• The Semiotics of Systems• System Design Involves Tradeoffs• Systems Programs are a Foundation• Digital/Discrete Models of an Analog/Continuous World• The Computer as Other• Appearances can be Deceiving• From Abstract Specifications to Complete Implementations• Communication has Multiple Dimensions• Resources must be Shared• Reliability can be Elusive• Use the Right Tool for the Job

Page 40: Backward Design - w3.cs.jmu.edu

Backward Design: An Integrated Approach to a Systems Curriculum SIGCSE 2015 • Kirkpatrick et. al

Systems core themesEnduring Understandings• Information = Bits + Context• The Semiotics of Systems• System Design Involves Tradeoffs• Systems Programs are a Foundation• Digital/Discrete Models of an Analog/Continuous World• The Computer as Other• Appearances can be Deceiving• From Abstract Specifications to Complete Implementations• Communication has Multiple Dimensions• Resources must be Shared• Reliability can be Elusive• Use the Right Tool for the Job

Page 41: Backward Design - w3.cs.jmu.edu

Backward Design: An Integrated Approach to a Systems Curriculum SIGCSE 2015 • Kirkpatrick et. al

Assessment strategies

Formativeassessments/instructional

activities

Summativeassessmentsand metrics

Defineobjectives

Identifysituational

factors

Page 42: Backward Design - w3.cs.jmu.edu

Backward Design: An Integrated Approach to a Systems Curriculum SIGCSE 2015 • Kirkpatrick et. al

Assessment strategies

Summative assessments• Question bank for exams• Good for program assessment, too

• Common projects - 3-phase semester-long system building• Shared rubric encourages consistent expectations

Formativeassessments/instructional

activities

Summativeassessmentsand metrics

Defineobjectives

Identifysituational

factors

Page 43: Backward Design - w3.cs.jmu.edu

Backward Design: An Integrated Approach to a Systems Curriculum SIGCSE 2015 • Kirkpatrick et. al

Assessment strategies

Summative assessments• Question bank for exams• Good for program assessment, too

• Common projects - 3-phase semester-long system building• Shared rubric encourages consistent expectations

Formative assessments and activities• Primarily at instructor discretion• Challenge: flipped vs. non-flipped classroom structures

Formativeassessments/instructional

activities

Summativeassessmentsand metrics

Defineobjectives

Identifysituational

factors

Page 44: Backward Design - w3.cs.jmu.edu

Backward Design: An Integrated Approach to a Systems Curriculum SIGCSE 2015 • Kirkpatrick et. al

New courses

Page 45: Backward Design - w3.cs.jmu.edu

Backward Design: An Integrated Approach to a Systems Curriculum SIGCSE 2015 • Kirkpatrick et. al

New courses

Core Tier 1 vs. Core Tier 2

Page 46: Backward Design - w3.cs.jmu.edu

Backward Design: An Integrated Approach to a Systems Curriculum SIGCSE 2015 • Kirkpatrick et. al

New courses

Module Hours DescriptionC basics 6 Memory model, pointersCompiler and debugger use 3 GCC/clang, Makefiles, GDBCPU/memory organization 3 Registers and cache, localityBinary representation 4.5 Two’s complement, IEEE 754, arithmetic encodingvon Neumann cycle 3 Role of CPU and memory, load/store instructionsBasic circuits 6 Logic gates, adders, ALUs, control signalsThreads vs. processes 6 Fork vs. thread creation, unique memory spaceInterrupts and OS principles 4.5 Interrupts, system calls, user vs. kernel modeSystem software design, evaluation 4.5 Benchmarks, complexity, static/dynamic analysis

CS 261 Computer Systems I (required)Introduction to operation of modern interrupt-driven computer systems. Explores the representation of software and information in binary memory, the primary components of a CPU, multithreaded programming, and basic interactions with an Operating System. Prerequisites: Grade of “C-” or better in CS 159.

Page 47: Backward Design - w3.cs.jmu.edu

Backward Design: An Integrated Approach to a Systems Curriculum SIGCSE 2015 • Kirkpatrick et. al

New courses

CS 361 Computer Systems II (required)Intermediate exploration of modern interrupt-driven computer systems. Explores models of computation and complex systems, techniques for communication and synchronization of parallel and concurrent software, and the protocols that make up the Internet. Prerequisites: Grades of “C-” or better in CS 240 and CS 261.

Module Hours DescriptionArchitecture analysis, evaluation, design 3 P2P vs. client-server, layered architecture

State models 4.5 Notion of state, UML, FSMMathematical modeling 3 Basic systems theoryInformation exchange 6 Communication basics (blocking vs. non-blocking, IPC vs. sockets)Synchronization primitives and problems 6 Locks vs. semaphores, producer-consumer, readers-writers, dining

philosophers, deadlockParallel decomposition 3 Data vs. task parallelism, Amdahl’s law, fork-join pattern, libraries

Protocol analysis, evaluation, design 9 Protocols and services, timing and statechart diagrams, connections, push/pull, flow control, reliability, handshaking, metrics

The Internet model 4.5 HTTP, DNS, DHCP, TCP, UDP, IP, 802.3, 802.11, ARP

Page 48: Backward Design - w3.cs.jmu.edu

Backward Design: An Integrated Approach to a Systems Curriculum SIGCSE 2015 • Kirkpatrick et. al

New courses

CS 450 Operating SystemsIntroduction to the design and implementation of modern operating systems. Explores fundamental concepts of operating systems, memory management, virtualization, resource allocation, file systems, and system protection mechanisms. Course work includes a significant programming component. Prerequisites: Grade of “C-” or better in CS 361.

Module Hours Description

Thread and process management 4.5 Review of multithreading, OS structures for representing threads and processes, context switches

OS interface and IPC 4.5 Review of system calls vs. interrupts, pipes, shared memory, other forms of IPC

Synchronization implementation 6 Hardware support for implementation of semaphores, locks, spinlocks

Memory management 6 Paging vs. segmentation, virtual memory, demand paging, implementation of shared memory

Virtualization 6 Virtualization vs. emulation, trap-and-emulate, binary translation, hardware support for virtualization

Scheduling 3 Scheduling policies and evaluation

I/O and file systems 6 Interrupt-driven I/O, DMA, RAID, file system implementation and metadata

Security and protection 6 CIA model, access control mechanisms, malware defense mechanisms

Page 49: Backward Design - w3.cs.jmu.edu

Backward Design: An Integrated Approach to a Systems Curriculum SIGCSE 2015 • Kirkpatrick et. al

New courses

CS 456 Computer ArchitectureIntroduction to the design and implementation of modern CPU architectures. Explores hardware-based parallel execution, quantitative performance evaluation, I/O interfacing techniques, and hardware descriptor languages. Course work includes a significant programming component. Prerequisites: Grade of “C-” or better in CS 261.

Module Hours DescriptionAssembly language 6 RISC assembly language and decoding

Building a datapath 6 Logic gates, control unit, ALU construction, register banks, von Neumann implementation

Hardware descriptor languages 3 Verilog, VHDL, RTL

Pipelined datapath and hazards 6 Pipelined datapath and control, data hazards (forwarding vs. stalling), control hazards, exceptions

Memory hierarchy and cache design 6 Quantitative performance measures, cache mapping techniques, cache coherence protocols

Storage and I/O interfacing 4.5 Storage devices, bus protocols, I/O performanceInstruction-level parallelism 4.5 Branch prediction, dynamic schedulingData-level parallel architectures 3 Vector, SIMD, GPU architecturesThread-level parallel techniques 3 Hyperthreading, shared-memory multiprocessors

Page 50: Backward Design - w3.cs.jmu.edu

Backward Design: An Integrated Approach to a Systems Curriculum SIGCSE 2015 • Kirkpatrick et. al

New courses

CS 470 Parallel and Distributed SystemsIntroduction to parallel and distributed systems. Explores shared memory, cluster, grid, peer-to-peer, and cloud computing models along with parallel software patterns, distributed file systems, and performance considerations. Course work includes a significant programming component. Prerequisites: Grade of “C-” or better in CS 361.

Module Hours DescriptionParallel/distributed concepts 3 Amdahl’s law, critical paths, speedup/scalability, data/task decomposition,

applications, research challenges

Parallel patterns 6 Naturally (embarrasingly) parallel, nearest-neighbor, communication, producer-consumer, master-workers, pipelines, map/reduce

Parallel systems 9 Shared memory, SMP, SIMD, OpenMP, GPUs/co-processors, race conditions, mutual exclusion, deadlock, cache effects, dense/sparse matrices

Distributed systems 9 MPI, MapReduce, global address spaces, clusters, topologies, synchronization, collectives, clocks, NUMA, hybrid architectures, fault tolerance

Grid, P2P, and cloud computing systems 6 Heterogeneous systems, decentralized computation, consensus, IaaS,

virtualizationDistributed file systems 6 RPC, data replication, transactions, consistency

Parallel performance 3 Tools, measurement, scheduling/load balancing, contention, communication overhead, power usage

Page 51: Backward Design - w3.cs.jmu.edu

Backward Design: An Integrated Approach to a Systems Curriculum SIGCSE 2015 • Kirkpatrick et. al

Possible future coursesBreadth-first scaffolding as foundation for:• Advanced networking• Database design and implementation• Virtualization technologies

Page 52: Backward Design - w3.cs.jmu.edu

Backward Design: An Integrated Approach to a Systems Curriculum SIGCSE 2015 • Kirkpatrick et. al

ACM 2013 MappingACM 2013 KAACM 2013 KA Current Proposed

Algorithms and ComplexityAlgorithms and ComplexityDistributed algorithms (T1) none 470

Architecture and OrganizationArchitecture and OrganizationMachine-level representation of data (T2) 350 261, 456Assembly level machine organization (T2) 350 261, 456Interfacing and I/O strategies (interrupts) (T2) 350 456Memory architecture (T2) 350 261, 456Functional organization (ILP/datapaths) (E) 350 (some) 261, 456

Operating SystemsOperating SystemsOS overview (T1) 450 261OS principles (APIs, processes, interrupts) (T1) 450 261, 450Concurrency (T2) 450 361, 450Scheduling (T2) 450 450Memory management (T2) 450 450Security and protection (T2) 450 457/450File systems (E) 450 (some) 450System performance evaluation (E) none 261

Network-centric ComputingNetwork-centric ComputingIntroduction (ISPs, circuit vs. packet) (T1) 460 361Network applications (HTTP, sockets) (T1) 460 (most) 361, 466, 470Reliable data delivery (TCP, flow control) (T2) 460 361, 466Routing vs. forwarding (T2) 460 361, 466LANs (T2) 460 361, 466Resource allocation (T2) 460 361, 466Mobility (T2) 460 361, 466

ACM 2013 KAACM 2013 KA Current ProposedParallel and Distributed ComputingParallel and Distributed Computing

Parallelism fundamentals (T1) 450 (most) 261, 470Parallel decomposition (T1,T2) none 361, 470Communication and coordination (T1,T2) 450 (some) 361, 450, 470Parallel architectures (T1,T2) 350 (little) 261, 470Parallel performance (E) none 470Distributed systems (E) none 361, 470

System FundamentalsSystem FundamentalsComputational paradigms (T1) 350 (some) 261, 361, 456, 470Cross-layer communications (T1) 450 (little) 361, 450, 466, 470States, transitions, state machines (T1) 350 (some) 361, 456, 466, 470System support for parallelism (T1) 350 (little) 361, 456, 470Performance (T1) none 261Resource allocation and scheduling (T2) 450 450Proximity (T2) 350 450, 456, 470Virtualization (T2) none 450Reliability through redundancy (T2) 460 (little) 361, 456, 470

Page 53: Backward Design - w3.cs.jmu.edu

Backward Design: An Integrated Approach to a Systems Curriculum SIGCSE 2015 • Kirkpatrick et. al

Transitional plan

Page 54: Backward Design - w3.cs.jmu.edu

Backward Design: An Integrated Approach to a Systems Curriculum SIGCSE 2015 • Kirkpatrick et. al

Transitional plan

Fall 2015• CS 460 replaced by CS 361• Elaborate tango between CS 361 and CS 450 (OS)

Page 55: Backward Design - w3.cs.jmu.edu

Backward Design: An Integrated Approach to a Systems Curriculum SIGCSE 2015 • Kirkpatrick et. al

Transitional plan

Fall 2015• CS 460 replaced by CS 361• Elaborate tango between CS 361 and CS 450 (OS)

Spring 2016• CS 350 replaced by CS 261• First offering of CS 470 (Parallel and Distributed)

Page 56: Backward Design - w3.cs.jmu.edu

Backward Design: An Integrated Approach to a Systems Curriculum SIGCSE 2015 • Kirkpatrick et. al

Transitional plan

Fall 2015• CS 460 replaced by CS 361• Elaborate tango between CS 361 and CS 450 (OS)

Spring 2016• CS 350 replaced by CS 261• First offering of CS 470 (Parallel and Distributed)

Fall 2016• New curriculum offering fully in place• CS 261 and CS 361 now offered every semester

• First offering of CS 456 (Architecture)

Page 57: Backward Design - w3.cs.jmu.edu

Backward Design: An Integrated Approach to a Systems Curriculum SIGCSE 2015 • Kirkpatrick et. al

Lessons learnedToo many to name...• A well-defined vision statement works wonders• Revisit frequently, particularly for conflict resolution

• SMART objectives make writing assessments easier• Take the time to do this well

• Transition to new curriculum requires improvisation and flexibility• Build your own taxonomy• Fink and Wiggins/McTighe for aspirational structure• Hansen for enumerating points of emphasis• Bloom and SOLO for instructional objectives

• The process will take longer than you expect• Hidden assumptions, connections, requirements

Page 58: Backward Design - w3.cs.jmu.edu

Backward Design: An Integrated Approach to a Systems Curriculum SIGCSE 2015 • Kirkpatrick et. al

https://github.com/kirkpams/JMU-CS-Systems