Think Tw Code On

8/11/2019 Think Tw Code On

1/5

IRON CODE : Think-Twice, Code-OnceProgramming

Mark W. BaileyComputer Science Department

Hamilton [email protected]

ABSTRACTTo become proficient programmers, novices must develop theskills of writing, reading, debugging, and testing code. We believethat learning to write short pieces of code correctly the first timehelps strengthen all of these skills. In this paper, we describe a typeof exercise, called I RONCODE, that helps develop the code-onceskill. We describe the exercise, the programming environment, itsimplementation, and our experiences using I RONCODE in a second

semester programming class.

Categories and Subject DescriptorsK.3.2 [ Computers and Education ]: Computer and InformationScience EducationComputer science education; D.2.3 [ Soft-ware Engineering ]: Coding Tools and Techniques; D.2.5 [ Soft-ware Engineering ]: Testing and Debugging testing tools ;

General TermsDesign, Reliability, Experimentation.

KeywordsCorrect Code, Program Reading, Program Writing

1 INTRODUCTIONAn important skill that beginning programmers must develop andmaster is the skill of writing correct code the first time . Unfortu-nately, the tremendous increases in processor speeds has shortenedthe edit-compile-run cycle to the point that students often rely tooheavily on testing. Frequently, students choose to try a codesnippet or hack over careful craftsmanship of a piece of code. In this paper, we present an exercise we use in a secondsemester programming course to develop this skill that we callthink-twice, code-once programming.

The ability to construct carefully a few lines of code is critical insoftware development. In teaching programming, we often focus

on clean and elegant solutions at the expense of emphasizing theimportance of correct solutions. This skill is used in not onlywriting code, but also reading and modifying code. How often dowe have to debug a piece of code by examination? Such exami-nation requires the same skills that writing bullet-proof codedoes: the ability to understand the interactions of two or more linesof code. The exercise we use to develop bullet-proof codingskills we call I RONCODE (in retrospect, we should have called it

KEVLAR CODE!).

2 RELATED WORKFor many years, instructors have used online environments for pro-gramming labs. Bowles used drills and programming quizzes totest programming skills [4]. More recently though, online exami-nations have been used in both closed and open environments [5, 7,8, 12]. We often evaluate student progress in courses using tests ordrills [2, 1, 3, 9, 10]. In addition, online quizzes have been used inintroductory programming courses [11]. Main and Savitch providesoftware with their text that will automatically test methods forcorrectness [6]. However, to our knowledge, there has never beenany experience with online code correctness exercises reported inthe literature.

3 SOLUTIONS IN TEN LINES OR FEWERIn order to develop strong code-once programming skills, we honea students skill using small programming problems. Each I RON-CODE problem asks the student to write a subprogram. Each prob-lem is one that might have been encountered in an introductoryprogramming course. An important characteristic of I RONCODEproblems is that a common solution can be written in ten or fewerlines.

We take many of the early I RONCODE problems from the standardC library. These include: string copy (strcpy), string comparison(strcmp), string length (strlen), and character search (similar to str-tok). These are good starters because students have used thesefunctions (or a variant) in their programming and may already befamiliar with their implementation. Additional problems are takenfrom classic introductory problems: sum the integers from M to N ,determine if a number is prime, sum of squares, etc . Finally, as thesemester progresses and student skills improve, we introduce prob-lems relevant to current data structures course material: find theminimum in a stack, determine the depth of a tree, etc . Often, theseare problems whose solutions have already been presented in classand have been coded recently by the students. These advanced

Permission to make digital or hard copies of all or part of this work for per-sonal or classroom use is granted without fee provided that copies are notmade or distributed for profit or commercial advantage and that copies bearthis notice and the full citation on the first page. To copy otherwise, orrepublish, to post on servers or to redistribute to lists, requires prior specificpermission and/or a fee.SIGCSE'05 , February 2327, 2005, St. Louis, Missouri, USA.Copyright 2005 ACM 1-58113-997-7/05/0002...$5.00.


2/5

problems not only further develop a students I RONCODE skills buthelp reinforce current concepts in the course.

4 IRONCODE LAB ENVIRONMENTWe introduce I RONCODE in data structures, our second semesterprogramming course. Our course introduces students to C++ aftera semester of programming in Java. Our course has a dedicatedthree-hour weekly lab, though weve used I RONCODE during lec-ture-only formats as well. Since we lecture in the lab, our enroll-ments are limited to the lab capacity of 26 students.

Students gather in a closed laboratory to solve the weekly I RON-CODE problem. Students work individually on online problems.Each I RONCODE problem asks the student to implement a C ++function. In a space provided on a web page, the student types theirsolution to the problem. When the student is satisfied with theirsolution, he submits it for evaluation. Figure 1 shows a snapshot of the I RONCODE programming environment.

During evaluation, the students solution is compiled and linkedwith a test harness. If the solution fails to produce an executableprogram, the student is notified and asked to make appropriatemodifications. However, if the solution produces an executableprogram, the solution undergoes a series of tests to evaluate its cor-rectness. The solution either passes (the test harness failed to iden-tify a problem with the solution) or fails. At this point, the studentcompletes the exercise by receiving the results. Most importantly,

if the students solution fails, the student may not modify the pro-gram to receive credit.

The preceding description will undoubtedly seem unreasonablyharsh to many readers. However, it doesnt tell the whole story.Students receive instructor support throughout the exercise. At anytime, a student may ask the instructor questions. In particular,questions similar to the following are both allowed and encour-aged:

What result should the function return if it is given an emptystring?

What is the question asking?

I dont know where to begin. Could I solve the problem bysearching through the array?

When is the increment expression of a for loop executed?

What does this compilation error mean? How do I fix it?

These questions help the students understand the nature of theproblem they are trying to solve and deal with the simple technol-ogy aspects of the exercise. We encourage significant interaction

throughout the exercise. There are, however, questions that wedeem inappropriate for the instructor to answer. These include:

Why doesnt my program work?

What does my solution return if the parameter is an emptystring?

Figure 1. The I RON C ODE programming environment.


3/5

Should I place this if statement inside or outside the loop?

Whats the value of variable x at this location?

Each of these questions focuses on the interaction of a sequence of two or more lines of code. Since the purpose of the exercise is todevelop exactly these skills, we gently encourage them to answerthe question themselves. Often, however, it is possible to guide the

student to a different, more appropriate, question that will helpthem with their solution.

5 IMPLEMENTATIONIn order to implement I RONCODE, we needed an environment thatallows students to edit and compile programs, but does not allowthem to run or test them. Since we were aware of no such system,we constructed a custom-built system for the purpose.

The I RONCODE environment is built using CGI (common gatewayinterface), a C ++ compiler, custom CGI scripts, and testing andsupport code for the I RONCODE problems. The I RONCODE frontpage presents the open problems. The student selects the appro-priate problem and is presented with an I RONCODE coding page.

This page always contains an English description of the problem, afunction signature (or C ++ prototype if you prefer), and an HTMLtext area for entering code.

The student progresses through the system by entering a proposedsolution to the problem. When complete, the student presses thesubmit button (this action is verified before proceeding). Uponsubmission, the content of the text area is uploaded to the webserver.

5.1 Results of Submitting a SolutionWhen a solution is submitted, it is compiled, run, and tested toevaluate its conformance to the problems specification. Themechanics for this process will be discussed in the next section.During the evaluation process, any of the following outcomes mayoccur:

The program doesnt compilethis is the simplest and mostcommon outcome. The students source has a compilationerror. The errors need to be displayed to the student so he canfix the problem.

The program doesnt linka linking error will occur if the stu-dent failed to provide the requested function, or if the studentfailed to provide an auxiliary function that he calls. Again, theerrors need to be displayed for the student.

The program runs correctlyin this case, the run of the pro-gram is displayed to the student. This always involves display-ing the different instances of calls to the function.

The program runs, incorrectlyin this case, the student hasfailed to correctly solve the problem. The run of the program isdisplayed. Each call to their subprogram is displayed, with acorrect result and the students result shown. Incorrect resultsare clearly marked for examination. In addition to the run, theversion of the function that the student was tested against isshown. We do this for two purposes. Usually, the student wantsto see a working version so he can learn from his mistakes. Fre-quently, this demonstrates a far easier solution than the studentproposed. We feel it is important for student to be exposed to

superb solutions to problems. Finally, in the event that the solu-tion is not superb (the solution has an error), this gives thestudent appeal material. Fortunately, this hasnt happened(yet).

The program runs, but terminates prematurely. This can occurfor many reasons. Any number of exceptionsmost notablydereferencing a null pointercan cause a program to termi-nate. Again, the program has not executed correctly, so the stu-dent doesnt receive credit (or another chance) for a correctsolution. This outcome is equivalent to the program runningincorrectly.

The program runs, but doesnt terminate (in a timely man-ner)one problem with small, execute-once solutions is thatinfinite loops are common. To overcome this problem, solu-tions are required to execute in a timely manner (a couple of seconds). After a predetermined length of time (set on a per-problem basis), the program is terminated. This outcome isequivalent to the program running incorrectly.

Any of the above, but the program has already been gradedafter a program has been graded (the program has executed atleast once), the student can continue to work on the problem.However, he will not get credit. This option was added aftermany students indicated that they wanted to polish their solu-tions (for no credit!).

5.2 Mechanics of Building a Test ProgramIn order to evaluate a students solution, it must be tested. SinceIRONCODE solutions always consist of one or more C ++ functions,we can repeatedly call the students function to determine correct-ness. Operationally determining correctness requires that for eachcall to the students function, the correct result must be known.This can be accomplished in at least two ways:

For each set of parameters, the correct result(s) may be

recorded. This can be handled using an array of records thatcontains the parameters and the corresponding results.

For each set of parameters, the correct result(s) may be com-puted. This requires a correct solution be available in the testharness.

It seems that either approach would work, but we felt that provid-ing a correct solution would be less error-prone than providing cor-rect results, so we opted for the correct solution approach.

Such a test harness will have a solution similar in structure to thecode in Figure 2. This approach will work, in theory, but fails foreven the simplest problems. For example, this approach limitsproblems to those that return a single result as a return value. Func-tions that return results through parameters cannot be included in

this test harness. In addition, functions such as C s string copy(strcpy) destructively modify one of the parameters causing subse-quent use of the same parameters to produce different returnresults (consider strcat for example). Further, incorrect executionof one function can impact the subsequent execution of other func-tions if the stack is corrupted. All of these issues therefore requirea more general solution than presented previously.

In our first implementation, we simply skirted the problems of developing a general test harness by writing custom test har-nesses for each problem. At first this sounds tedious, but often test


4/5

harnesses can be adopted or adapted from previous I RONCODEproblems. For example, the kernel of the test harness for reversingan array of characters is shown in Figure 3.

This illustrates the kinds of contortions that one must go through totest such a simple problem. First, notice that the correct solution iscalled prior to calling the students solution. This prevents the stu-dent solution from corrupting the stack of the correct solution (forthe current iteration of the loop at least!). Second, in the case of string reversal, the single parameter is destructively modified.Thus, a copy of the parameter must be made (strcpy) before callingthe first solution. A copy of the result must also be made, or at leastit must be placed in a different location. Finally, a type-specificcomparison operation (strcmp) must be performed to determine if the result is correct. This technique proves to be acceptable forsmall numbers of simple I RONCODE problems, but mounting amore ambitious program requires a more robust system.

A second-generation I RONCODE test harness system is in develop-ment. This system uses a complex collection of templated C ++classes to provide a general solution to this problem. In this sys-tem, a separate class is required for each set of problems with dif-fering arity (that is, functions taking four parameters use adifferent templated class than functions that take three). In additionto these classes, we have constructed a set of operators for copying

and comparison of common types. The system can model in, out,and in/out parameters and provides a mechanism for registeringinstances of tests to be run. Figure 4 shows the complete test har-ness for the reverse example.

6 CONCLUSIONS

We began the I RONCODE initiative because we found that studentswere exiting our first programming course with a firm understand-ing of programming through experimentation, but with deficien-cies in their understanding of fundamental programming languagesemantics. For example, it was common for students to misunder-stand the intricacies of the for loop. We have found that sincebeginning the I RONCODE project, students have responded to theclose instructor interaction that I RONCODE promotes. We have alsofound the environment helps the students quickly identify andstrengthen their weaknesses.

We have been experimenting with I RONCODE for several semes-ters. For the most part, the experiment seems to be a great success.Since our courses generally have fewer than 20 students eachsemester, student performance metrics are not statistically signifi-cant enough to evaluate I RONCODE as a pedagogical tool. Instead,we conclude with our observations of how I RONCODE is workingin our course.

When students first experience I RONCODE, they often feel quiteintimidated and anxious (we have tried to find ways to alleviatethis but to date have been unsuccessful). They feel this way eventhough I RONCODE is not used to determine student grades. How-ever, after a few exercises, most students rise to the challenge of

< for each student solution >if (student_soln(< parms >) !=

correct_soln(< parms >))

< signal incorrect solution >;< signal correct solution >;

Figure 2. A possible structure of a test harness.

for (i = 0; tests[i]; i++) { strcpy(solution, tests[i]); strcpy(theirs, tests[i]); solution_reverse(solution); start_alarm(2); reverse(theirs); stop_alarm(); if (strcmp(solution , theirs)) { errors++; }}

Figure 3. Test harness for reversing a string.

int main() { VoidOneParm TestRun(reverse_solution, reverse, "reverse");

char *tests[] = {"bob",

"bo", "a couple of words", "a couple", "BOB", "Next one is empty!", "", };

for (size_t i = 0; i < sizeof(tests) / sizeof(char *); i++) { TestRun.addTest(tests[i]); }

return TestRun.performTes ts();}

Figure 4. A templated test harness for reverse.


5/5

IRONCODE. The success rate for I RONCODE typically hoversaround 50%. This seems to be sufficiently high to encourage stu-dents but sufficiently low to challenge them to improve their I RON-CODE skills. By the end of the course, students are more confidentabout their programming-in-the-small skills and look forward tofuture I RONCODE challenges.

REFERENCES1. J. M. Adams and B. L. Kurtz. A state-of-the-art CS undergrad-

uate lab. In Proceedings of the SEI Conference on Software Engineering Education , pages 8594. Springer-Verlag, 1990.

2. H. P. B. Kurtz. Developing programming quizzes to supportinstruction in abstract data types. SIGCSE Bulletin , 21(1):6670, February 1989.

3. R. Bennett and J. Wadkins. Interactive performance assess-ment in computer science: the advanced placement computerscience (APCS) practice system. Journal of Education Com-

puting Research , 12(4):636678, 1995.

4. K. Bowles. A CS1 course based on stand-alone microcomput-ers. SIGCSE Bulletin , 9(1):125127, February 1978.

5. M. Joy and M. Luck. Effective electronic marking for on-lineassessment. In Proceedings of the 6th Annual Conference onthe Teaching of Computing and the 3rd Annual Conference on

Integrating Technology into Computer Science Education ,pages 134138. ACM Press, 1998.

6. M. Main and W. Savitch. Data Structures and Other Objectsusing C++ . Addison-Wesley, 2nd edition, 2001.

7. D. V. Mason and D. M. Woit. Integrating technology intocomputer science examinations. In Proceedings of the 29thSIGCSE Technical Symposium on Computer Science Educa-tion , pages 140144. ACM Press, 1998.

8. M. D. Medley. On-line finals for CS1 and CS2. In Proceedingsof the 6th Annual Conference on the Teaching of Computing

and the 3rd Annual Conference on Integrating Technology intoComputer Science Education , pages 178180. ACM Press,1998.

9. R. Sanford and P. Nagsue. Selftest, a versatile menu-driven PCtutorial simulates test-taking. Computers in Education Jour-nal , pages 5869, 1992 1992.

10. A. Walworth and R. Herrick. The use of computers for educa-tional and testing purposes. In Proceedings of Frontiers in

Education. 21st Annual Conference. Engineering Educationin a New World Order , pages 510514, 1991.

11. D. Woit and D. Mason. Enhancing student learning throughon-line quizzes. In Proceedings of the 31st SIGCSE TechnicalSymposium on Computer Science Education , pages 367371.ACM Press, 2000.

12. D. M. Woit and D. V. Mason. Lessons from on-line program-ming examinations. In Proceedings of the 6th Annual Confer-ence on the Teaching of Computing and the 3rd AnnualConference on Integrating Technology into Computer Science

Education , pages 257259. ACM Press, 1998.

Documents

Think Tw Code On