Upload
inez
View
28
Download
0
Embed Size (px)
DESCRIPTION
Specification and Reasoning in SE Projects Using a Web IDE. Charles T. Cook (Clemson) Svetlana V. Drachova-Strang (Limestone College) Yu-Shan Sun (Clemson) Murali Sitaraman (Clemson) Jeffrey C. Carver (Alabama) Joseph E. Hollingsworth (IU Southeast) - PowerPoint PPT Presentation
Citation preview
Computer Science School of Computing Clemson University
Specification and Reasoning in SE Projects Using a Web IDE
Charles T. Cook (Clemson)Svetlana V. Drachova-Strang (Limestone College)
Yu-Shan Sun (Clemson)Murali Sitaraman (Clemson)
Jeffrey C. Carver (Alabama)Joseph E. Hollingsworth (IU Southeast)
This research is funded in part by NSF grants CCF-0811748, CCF-1161916, DUE-1022191, and DUE-1022941.
Computer Science School of Computing Clemson University
Part I: Overview
School of Computing Clemson University
About Clemson
School of Computing has about 600 undergrads and 200 grads
Clemson University has 17,000 students
Located in a town (also Clemson) with an official population of about 13,000 in South Carolina
Has a football stadium with capacity 85,000
3
School of Computing Clemson University
This talk
This talk is about undergrad courses, though the ideas have been used in both undergrad and grad courses for many years
4
School of Computing Clemson University
Goals of the SE Projects
Role of formal specifications as contracts in team software development and integration
Role of formal specifications in reasoning about software correctness
Other uses Specification-based test case design
5
School of Computing Clemson University
Intro. and Advanced Projects
Introductory projects involve only use of and reasoning with existing library components 2 weeks of lectures
Advanced projects involve development and reasoning about new components 3-5 weeks of lectures
Other variations
6
School of Computing Clemson University
Sample Courses and Institutions
Sample introductory projects Alabama (Software Engineering) Cleveland State (Software Engineering)
Sample advanced projects Clemson (Software Engineering) Denison (Independent study projects)
Other variations Depauw (Theory), NC State (Data
structures), Southern Wesleyan (Data Structures), Ramapo College (Programming Languages),…
7
School of Computing Clemson University
Clemson University Details
Two-course sequence Sophomore-Level, CP SC 215: Software
Development Foundations Junior/Senior-Level: CP SC 372: Software
Engineering Experimentation and assessment
Piloting (2007/2008) Institutionalization (2009 to present)
8
School of Computing Clemson University
Soft. Dev. Foundations Course
Intro to Java, object-based computing, software engineering, design patterns, etc. 10 weeks (includes usual materials and
projects) Intro to formal specifications and
reasoning (interspersed with above topics) 4 weeks
Uses RESOLVE-style specifications in a Java context 9
School of Computing Clemson University
Software Engineering Course
Software life cycle, process models, requirements analysis and design 8 weeks (includes usual materials and
projects) Transition from informal to formal
1 week Specification-based component
development and quality assurance 5 weeks
Uses RESOLVE10
School of Computing Clemson University
Clemson University Key Points
Specification/reasoning included in syllabi for two required courses for majors (2009)
Graduating student learning outcome modified to include “development of bug-free software according to specifications” (2012)
11
Computer Science School of Computing Clemson University
Part II: Example Projects
School of Computing Clemson University
RESOLVE
An integrated specification and programming language for verified, component-based software development
A Verifying compiler A github project A freely-available web interface to
use the verifying compiler: www.cs.clemson.edu/group/resolve
13
School of Computing Clemson University
Sample Intro Assignments
Generate VCs and prove the given Queue Remove_Last operation.
Make each of the following changes and explain what is unprovable. Comment out the first Dequeue
operation. Change the maintaining clause (loop
invariant) to #Q = <E> o Q. Change the decreasing clause
(termination progress metric) to |T|.14
School of Computing Clemson University
Sample Advanced Assignments
Implement and verify: Extension operations on Queues to
Insert_After and Remove_After Sequence_Template using
Queue_Template with extensions Multiple implementations of
Queue_Template satisfying given internal contracts
Assignments to put it all together Involve a dozen components and teams of 3
students15
School of Computing Clemson University
Sample Components
Stack_Template Queue_Template Preemptable_Queue_Template Sequence_Template List_Template Search_Store_Template Map_Template Prioritizer_Template … 16
Computer Science School of Computing Clemson University
Part III: A Web IDE Demo
School of Computing Clemson University
Getting Started
www.cs.clemson.edu/group/resolve Tab: Web IDE
Google: RESOLVE web IDE RESOLVE verifier Clemson RESOLVE
18
School of Computing Clemson University
Sample Intro Assignments
Generate VCs and prove the given Queue Remove_Last operation.
Make each of the following changes and explain what is unprovable. Comment out the first Dequeue
operation. Change the maintaining clause (loop
invariant) to #Q = <E> o Q. Change the decreasing clause
(termination progress metric) to |T|.19
School of Computing Clemson University
Elements of the Assignment
Queue_Template concept specification
A specification of Remove_Last Operation
An annotated implementation of Remove_Last operation
20
School of Computing Clemson University
Mathematical Modeling
Concepts provide mathematical models for programming objects
To write formal specifications, we need to model the state mathematically
Some objects we use in programming, such as Integers and Reals, have implicit models
For others, such as stacks, queues, lists, etc., we need to conceive explicit mathematical models
School of Computing Clemson University
Mathematical Modeling of Queues
Concept Queue_Template(type Entry; Max_Length: Integer);
uses String_Theory;
Type Family Queue is modeled by …
Operation Enqueue…Operation Dequeue……
end Queue_Template;
School of Computing Clemson University
Concept Queue_Template(type Entry; Max_Length: Integer);
uses String_Theory;
Type Family Queue is modeled byStr(Entry);
exemplar Q;constraints |Q| <= Max_Length;initialization ensures Q = ;
…
end Queue_Template;
Mathematical Modeling of Queues
School of Computing Clemson University
Operation Remove_Last (updates Q: Queue; replaces E:
Entry); requires |Q| /= 0; ensures #Q = Q o <E>;
A Specification of Remove_Last
School of Computing Clemson University
Procedure Remove_Last (updates Q: Queue; replaces E:
Entry);Var T: Queue;Dequeue (E, Q);While (Length(Q) /= 0)do
Enqueue(E,T);Dequeue(E,Q);
end;Q :=: T;
end Remove_Last;
An Implementation of Remove_Last
School of Computing Clemson University
Assume #Q = <10, 20, 30, 40>Dequeue (E, Q);While (Length(Q) /= 0)
T E QIter. 1 10 <20, 30,
40> Iter. 2 <10> 20 <30, 40>Iter. 3 <10, 20> 30 <40>Iter. 4 <10, 20, 30> 40 do
Enqueue(E,T);Dequeue(E,Q);
end;
Understanding Loop Invariant
School of Computing Clemson University
Assume #Q = <10, 20, 30, 40>Dequeue (E, Q);While (Length(Q) /= 0)
T E Q
Iter. 1 10 <20, 30, 40> Iter. 2 <10> 20 <30, 40>
Iter. 3 <10, 20> 30 <40>Iter. 4 <10, 20, 30> 40
Loop maintains this invariant: #Q = T o <E> o Q
Understanding Loop Invariant
School of Computing Clemson University
Procedure Remove_Last (updates Q: Queue; replaces E: Entry);
Var T: Queue;Dequeue (E, Q);While (Length(Q) /= 0)
maintaining #Q = T o <E> o Q;decreasing |Q|;
doEnqueue(E,T);Dequeue(E,Q);
end; …
An Implementation of Remove_Last
School of Computing Clemson University
Verification
Press Verify Button Generate and prove automatically a
series of verification conditions (VCs) Students understand why the VCs
arise Understand connections between
contracts, code, and proofs Can prove VCs
29
School of Computing Clemson University
Impact of Incorrect Code
The first statement, call to Dequeue, is removed.
Leads to one unprovable VC (Verification Condition)
VC 0_1: Base Case of the Invariant of While
Statement in Procedure Remove_Last: Remove_Last_Realiz.rb(8)
Goal: Q = ((empty_string o <E>) o Q) Givens: …
30
Computer Science School of Computing Clemson University
Part IV: Assessment
School of Computing Clemson University
Assessment Summary
For details See the paper See Drachova Ph. D. dissertation (2013,
Clemson), available at website A quick summary
Likert items to assess perceived benefits of web IDE (avg. 4.0/5.0)
Project grades (avg. from 80%); about the same as non-formal parts
RCI-item based analysis32
School of Computing Clemson University
What reasoning skills are necessary?Reasoning Concept Inventory
Boolean Logic Standard Logic Symbols, Standard Proof Techniques
Discrete Math Structures Sets, Strings, Numbers, Relations, and other mathematical theories as needed
Precise Specifications Mathematical Descriptions of Software interfaces for clients and implementers. Math models for structures. Pre and Post conditions for operations.
Modular Reasoning Each Module needs to be proven correct only once.
Correctness Proofs Mathematical Assertions equivalent to the correctness of the program.Application of Proof Techniques to the program
http://www.cs.clemson.edu/resolve/teaching/inventory.html
School of Computing Clemson University
Clemson RCI-Based Evaluation
34
Sem # Avg. % scoring 70% or higher
RCI 3.4.3 3 82 87% 85%RCI 4.2 2 56 76% 59%
RCI 4.3.1 1 24 77% 67%
3.4.3: Precise specifications 4.2: Design by contract 4.3.1: Internal contracts
School of Computing Clemson University
SE Project Benefits on RCI-Based Learning Outcomes
35
Avg. % scoring 70% or higher
Before project (mid-term)
59% 25%
After project (final)
84% 75%
RCI Topics: 3.4.2, 3.4.3: Precise specifications
Sample size: 24 students, Spring ‘12
School of Computing Clemson University
Summary
Students can practice formal specification and reasoning principles effectively using the RESOLVE web IDE and its verifier There are some benefits for even a
minimal intro (3 lectures plus project) Significant benefits with longer-term
exposure IDE and materials online
www.cs.clemson.edu/group/resolve