ACES: an interactive software platform for self-instruction and self-evaluation in automatic control systems

102 IEEE TRANSACTIONS ON EDUCATION, VOL. 46, NO. 1, FEBRUARY 2003

ACES: An Interactive Software Platform forSelf-Instruction and Self-Evaluation in Automatic

Control SystemsVassilios Petridis, Spyros Kazarlis, and Vassilis George Kaburlasos

Abstract—This paper presents an interactive-, menu-drivenprototype software platform, namelyautomatic control educationalsoftware(ACES), for self-instruction and self-evaluation in auto-matic control systems. ACES is used for enriching instruction inautomatic control at Aristotle University of Thessaloniki, Greece,in the Department of Electrical and Computer Engineering. TheACES platform includes theory with hyperlinks, a concept-graph,and a database with exercises. Students’ answers to exercises areevaluated automatically “on-line.” Furthermore, exercises can beproposed automatically by ACES. An instructor/supervisor cansupport in person the learning effort of a student, monitor theprogress of a student, and, also, tailor a course’s contents on themodular ACES platform. Two statistical hypothesis tests on bothattitude questionnaires and student marks in the final (written)exam confirmed that the employment of ACES in the educationalprocess can improve the performance of students in an automaticcontrol course although the attitude of students toward the coursedoes not change significantly with the use of ACES.

Index Terms—Control systems education, HTML, interactiveeducational software, self-evaluation, self-instruction.

I. INTRODUCTION

D IFFERENT authors have recognized that engineering ed-ucation is not keeping up with the rapid changes that take

place in the practice of engineering [1]. Moreover, the IEEE Ed-ucation Society has acknowledged that this is a period of rapidchange in engineering education, especially at the undergrad-uate level [2].

Lately, the ever increasing power and availability of com-puters, including the availability of both hardware and softwarehas encouraged efforts to use computers to enhance traditionalways of delivering education [3]. In this vein, the EuropeanUnion has launched a number of research projects for the de-velopment of information technologies for learning and training[4].

This work describes a prototype software platform for deliv-ering undergraduate education on automatic control systems.More specifically, the automatic control educational softwareplatform (ACES) has been developed for self-instruction andself-evaluation as detailed here. ACES is used in the Departmentof Electrical and Computer Engineering, Aristotle University of

Manuscript received March 31, 2002; revised April 11, 2002.The authors are with Division of Electronics and Computer Engineering,

Department of Electrical and Computer Engineering, Aristotle Univer-sity of Thessaloniki (AUTh), GR-54006 Thessaloniki, Greece (e-mail:[email protected]).

Digital Object Identifier 10.1109/TE.2002.808266

Thessaloniki, Greece, since 1999. Preliminary results have beenpresented in [5].

In electrical engineering various computer-assisted learningapplications have been reported for various courses, includingcircuit analysis [6], discrete-time systems [7], etc. Regardingautomatic control, in particular, instruction at an undergraduatelevel is a challenging task because of the fairly high mathemat-ical contents, which should bear, however, an immediate prac-tical utility [8].

With the advent of World Wide Web (WWW) new oppor-tunities arose for delivery of education. For instance, hyper-text markup language (HTML) provides a mechanism for al-lowing media rich representations to be made on the WWW,and methods for developing hypertext learning courses havebeen reported [9]. Regarding automatic control in particular, anoverview on Controls Education on the Web has been presentedin [10]. Interactive-learning software tools for automatic controlcourses have been developed in HTML [11]. Likewise, the workin [12] presents an interactive software tool for iterative rootlocus and Bode plot control system classroom design. Anothereducational software package for teaching automatic control viathe web is described in [13]. Also, the work in [14] describes anew technology that uses the web to teach feedback control; inparticular, a user can modify interactively a controller for a he-licopter model located remotely. To advance the state-of-the-artin computer-assisted delivery of education in automatic controlat undergraduate level, the ACES software platform has beendeveloped as detailed here.

The layout of this paper is as follows. Section II presents gen-eral characteristics of the software platform ACES. Section IIIdescribes technically in detail the various software componentsof ACES. Section IV discusses practical issues pertainingto employment of ACES and an example of using ACES isdemonstrated. Section V presents a statistical assessment ofACES. Section VI concludes by summarizing the contributionof this paper including potential future work. Three shortAppendixes A–C provide specific technical details.

II. ACES CHARACTERISTICS

A user-friendly software platform, that is ACES, was devel-oped in modules so as to facilitate maintainability. Seeking fora longer-term employment of ACES via WWW, most softwaremodules of ACES have been developed in the HTML program-ming language, and some in Visual C++. Moreover ACES runs

0018-9359/03$17.00 © 2003 IEEE

PETRIDISet al.: ACES 103

on PC platforms, including Windows 95/98/2000 and WindowsNT.

In order to maximize its utility, ACES is able to run either onstand-alone PCs or on network environments. In the latter caseclient versions of the software are installed on network worksta-tions, which can communicate interactively with the server.

ACES includes a theory module and a database with exer-cises which can be solved “on-line.” Also, marks to a student’sanswers can be delivered “on-line” automatically. That is,ACES can be used by the students for self-instruction andself-evaluation. More specifically, the student marks in anexercise are stored in a database together with a student’slearning state vector (LSV). LSV is a three-dimensional (3-D)vector of marks, which express a student’s competence in:1) designing; 2) comprehension of selected concepts; and 3)comprehension of specific algorithms. Based on a student’sLSV, ACES can propose exercises to a student. In this wayACES is able to address an individual student’s learning style.

Several tools for interactive system design have been incorpo-rated to visualize the effects of varying various control systemparameters, e.g., system poles and zeros. Furthermore, a Con-cept Graph illustrates relations between useful concepts in au-tomatic control.

It should be pointed out that ACES was not meant to dispensewith traditional instruction. Moreover, ACES was developed forcomputer-based enrichment of the educational process ratherthan for providing more homework problems.

III. T ECHNICAL DESCRIPTION ANDCONTENT

The software components of ACES are described technicallyin this section. Note that an Internet browser Active-X controlwas used by ACES to view either theory- or exercise- softwaremodules written in HTML. Furthermore, ACES requires an In-ternet browser, e.g., Internet Explorer.

A. Pull-Down Menus

The Main Menu Window includes several pull-downmenus. TheFile pull-down menu includes switches forconnecting-to/disconnecting-from a local server. TheViewpull-down menu specifies the appearance of the “toolbar”and the “status line;” from the latter menu it is also possibleto launch the browser for the pages of theory. TheStudentspull-down menu is used to register a student. TheTheorypull-down menu can activate both hypertext with theory andConcept Graph hypertext pages. TheExercisespull-down menuincludes buttons for displaying/searching/selecting exercisesand for displaying first, the list of solved exercises; second, astudent’s detailed marks in an exercise; and third, a student’sown LSV; also, from here a student can activate an interactivePole-Zero Design Window. Finally, theHelp pull-down menuincludes all conventional buttons for help. Most of the afore-mentioned buttons are also available as “stand-alone” buttonson the application’s floating toolbar for fast launching.

B. Theory Modules

The Theory Module includes six chapters on: 1) systemmodeling; 2) time response; 3) frequency response; 4) sta-

bility; 5) practical design issues and specifications; and 6)design methodologies. Links to other chapters are includedas well as an Appendix with useful mathematical formulasand procedures. The user can also access the Concept Graph(detailed below), click on to other Automatic Control websites, and email to the developers of ACES. The resizableTheory Module Window includes the conventional Back/For-ward/Refresh/Home/Search/Close/Help buttons for hypertextnavigation. A button for launching the Exercise Window is alsoincluded.

C. Exercises Modules

A database including 168 exercises is available. In particular,there are 9, 13, 21, 49, 22, and 54 exercises for chapters 1, 2,3, 4, 5, and 6, respectively. The difficulty level of an exercise isdenoted by an integer 1, 2, or 3. Moreover, an exercise is classi-fied in one of these categories:design, concept comprehension,or algorithm comprehension. Each category corresponds to anentry in LSV.

The Exercises Module Window includes subwindows for dis-playing both exercise contents and various useful attributes, in-cluding “student-defined filters” to shorten selectively the list ofexercises (Fig. 1).

After solving an exercise the student turns in his/her answersin a specified vector format. Then, ACES grades the answers andproduces a mark in the range according to the “grade ofcorrectness” of a student’s answer; that is, instead of giving fullcredit for correct answers and zero credit for wrong answers, afuzzy grading system was used as detailed in Appendix A. Themarks of solved exercises define theLSV, giving three differentmarks for: 1) designing; 2) concept comprehension; and 3) al-gorithm comprehension, respectively. The LSV is recalculatedeach time a new exercise is solved. The aforementioned exercisesolving procedure is demonstrated by an example in Section IV.

D. Pole–Zero Design

The user can place poles and zeros interactively on the scal-able complex plane at locations of his/her choice. Furthermorethe “gain” of a specific control system can be defined (Fig. 2).Then, with a single mouse click, ACES builds a system model;it calculates, and, finally, it displays the corresponding StepResponse, Bode, Nyquist Diagrams, and Root Locus on theSystem Response Window (Fig. 3). The user can insert-, move-,or delete- poles/zeros, and refresh the screen. Note that a pole(zero) can be inserted either by clicking the left (right) mousebutton on a specific location of the plane or by typing in spe-cific and coordinates. A pole/zero can be moved to anotherlocation by clicking on it and dragging it.

E. Concept-Graph

The Concept Graph was meant to illustrate intrinsic relationsbetween useful concepts in automatic control by a tree-like hy-perlink structure. The Concept Graph contains a total of 20pages; whereas, the tree structure has a maximum depth of fournodes/pages. The five “root nodes” of the aforementioned treestructure are labeled as: 1) System Categories; 2) System Char-acteristics; 3) System Response; 4) Stability; and 5) System De-


Fig. 1. The exercises module window. A selected exercise is displayed in the large subwindow. The “id numbers” of various exercises are shown in the top-rightsubwindow. Under the latter subwindow is shown the id number of a selected exercise. The five boxes in the top-left corner are either for displaying theattributesof a selected exercise or for defining a matrix of attributes used for filtering the list of exercises shown in the top-right subwindow.

Fig. 2. The pole-zero design window of ACES for inserting, moving, or deleting system poles (shown by an “X”) and zeros (shown by an “O”).

sign. A student can click on a keyword to branch onto a new pagewith illustrations. A student can also view relations of a conceptwith other concepts.

F. User Interface

Running in a windows environment, the User Interface isfully based on windows, dialogs, mouse input, and events. Allmenu selections are available through either mouse clicking orkeyboard commands. The basic buttons also appear in a floatingtoolbar for fast launching. The toolbar is initially docked underthe main menu, but can be reformatted and docked to alterna-tive locations. A status line displays useful information aboutthe state of the application and help hints. Most of the user input

is provided by single clicking on buttons and controls. A com-plete help system has been developed for all application mod-ules. Moreover, multiple module windows can be open at thesame time, e.g., Theory Window and Exercise Window.

G. The ACES Supervisor Application

As a supplement to ACES a separate application, namelyACES Supervisor, has been developed for the instructor/super-visor. The aim of ACES Supervisor is to enable both “on-line”monitoring of student progress and “on-line” definition of acourse’s contents. For instance usingACES Supervisor, the in-structor can view the “exercises solving record” of a student anda student’s LSV vector. The ACES Supervisor can be installed


Fig. 3. The system response window displays the step response (top-left),Bode diagram (top-right), Nyquist diagram (bottom-left), and Root locus(bottom-right), of the system whose pole(s) and zero(s) have been defined bythe user in the pole-zero design window (Fig. 2).

on any workstation that has network access to the lab server,e.g., in an instructor’s office.

IV. USING ACES

Various technical issues are addressed in this paragraph.ACES was installed on PCs using a customized “setup pro-gram,” which can be either downloaded via the Internet orcopied from an installation CD-ROM. ACES software isnot transferable from a PC to another by simply copying allapplication files. This setup was achieved by inserting certainregistry entries at “installation time” which (entries) are queriedat “run time.” Moreover, at “run time” the application runsa prelaunch diagnostic routine, which produces achecksuminteger number from all application files. More specifically,all bytes in all application filesare summed up resulting ininteger numberchecksum, which is required to be identicalto a reserved integer number in order for the application tostart. The aforementioned “checksum diagnostic routine” isvery fast, requiring specifically only about 5 s to sum up about7.5 Mbytes of data.

By connecting to a server, the software runs in a “client” modewhere all files are accessed from the server. There are at leastthree benefits from a single (server) copy of ACES: First, a stu-dent is not bound to sit at the same place during different lab-oratory sessions. Second, the instructor can monitor the perfor-mance of a student by accessing a single database file in theserver PC. Third, the instructor can update the contents of anydata or software module of either theory or exercises by re-placing a single file in the server PC. By disconnecting from theserver, ACES software enters the “stand-alone” mode where allfiles are accessed from local drives.

ACES was used in a “constrained educational environment”as explained in the following. In one year, students were in-structed using traditional classroom techniques. In particular,an instructor taught an automatic control course to one groupof students in the classroom two hours a week for one semester.In addition, there was a classroom tutorial two hours a week.Moreover, work was assigned to the students to be carried out

(a)

(b)

(c)

(d)

(e)

Fig. 4. Demonstrating various steps in the solution of exercise item 6.0.39using ACES. (a) Entries of LSV before solving an exercise. (b) A sampleexercise in the course, “classic automatic control.” Exercise answers arerequired in a specific (vector) format. (c) A student turns in his/her answers asspecified by ACES. Then a student activates executable program “evaluate”for automatic grading. (d) An overall grade is produced as explained inAppendix A. (e) After solving an exercise, the LSV vector is updatedaccordingly.

either in the lab or at home. Next year, an instructor taught thesame automatic control course to another group of students inthe classroom 2 h/wk for one semester. Likewise, there was aclassroom tutorial 2 h/wk. Finally, work was assigned to the stu-dents to be carried out using ACES either in the lab or at home.In both aforementioned years, the grades of students in the finalexam were determined the same way, that is, the students usingACES were not treated differently. The following example illus-trates in detail the employment of ACES for solving an exercise.

A. Example

A student’s LSV can be initially as shown in Fig. 4(a). Inparticular, Fig. 4(a) shows that a student has already achieved


8/10 in “designing,” 9/10 in “comprehension of concepts,” and6/10 in “comprehension of algorithms.”

Fig. 4(b) shows an exercise of type “algorithmic,” includinghints to a student for inserting his/her answers in a specificvector format. An exercise is shown in the largest subwindowof the Exercises Module Window (Fig. 1). A student may cal-culate the quantities requested using any tool of his/her choice.After solving an exercise, the student needs to click on the button“Solve Exercise” (Fig. 1), to pass the answers to ACES forgrading them. As soon as the button “Solve Exercise” is ac-tivated, certain tasks are carried out automatically by ACES.More specifically, the corresponding exercise’s name is used asa key index to retrieve a specific file that contains the solutionsof the exercise in question. A copy of the latter file is renamed“evaluate.exe” and stored in a prespecified location on the harddisk. The file “evaluate.exe” contains the correct answers of anexercise and instructions for grading a student’s answers.

After a student has calculated the answers in an exercise,he/she enters the answers in aspecific vector formatas specifiedin Fig. 4(b). Finally, the student types in the code word “eval-uate,” as shown in Fig. 4(c), then presses the “enter” key. Ex-ecutable file “evaluate.exe” starts running. Student answers arematched against the correct answers, and using “fuzzy gradingprinciples” as explained in Appendix A, an overall grade is pro-duced [Fig. 4(d)]. Finally, the LSV is updated so that no morethan one entry of LSV increases; whereas, the other two entriesof LSV remain the same [Fig. 4(e)].

The entries of vector LSV cannot decrease; they can only in-crease. The software which updates the entries/marks in vectorLSV was written so that a student needs to supply correct so-lutions to exercises in all categories and all levels of difficultyin order for his/her LSV vector to get high entries. In conclu-sion, execution of file “evaluate.exe” terminates, and softwarecontrol returns again to the Exercises Module Window (Fig. 1).The entries of vector LSV reflect a student’s “exercise solvingcompetence.” The entries of LSV can be updated only by ACESand only automatically. Nevertheless, either a student or the su-pervisor can view the LSV entries. Hence, based on a student’sLSV, a course of study can be decided/proposed by either a stu-dent or the supervisor. Moreover, ACES software itself has thecapacity to propose exercises automatically to a student basedon a student’s LSV vector.

From the literature “effective learning,” in a hypertext-basedlearning environment, calls for an active participation of stu-dents by browsing, selecting, searching, scanning, and tracing[15]. Experience with ACES has confirmed that students-usersof ACES should themselves be self-motivated in order to max-imize the effectiveness of ACES. Moreover, the authors’ expe-rience has shown that the plethora of software tools confusedsome students who were unwilling to makeactive choices. Inaddition, a few students were distracted (Internet surfing) duringa lab session. However, ACES was not designed for increasingthe student interest in a course, but rather it was designed for in-creasing a student’s understanding of course material. A statis-tical assessment has shown that ACES can affect the educationalprocess in a positive way as detailed in the following section.

V. STATISTICAL ASSESSMENT

The effectiveness of ACES was assessed statistically as ex-plained in this section.

A. Statistical Tests

The following two null hypotheses H0 and G0 were tested.

H0) Given the same resources, a group of students whoused ACES receive the same marks in the final(written) exam as a group of students instructed withtraditional classroom techniques.

G0) The student interest for the course is the same whetherthey use ACES or not.

Null hypothesis H0 was tested using the marks received bystudents in the final (written) exam; whereas, null hypothesis G0was tested based on student responses to customizedmultiplechoice“attitude questionnaires” handed out to the students theday of the final exam – the latter questionnaires are described inAppendix B.

Statistical testing of either hypothesis H0 or G0 was effectedbased on two populations of random samples. The first popula-tion corresponded to studentstaught by traditional classroom techniques; whereas, the secondpopulation corresponded tostudents who used ACES.

The nonparametric, rank-based Kruskal–Wallis statistical testwas used which assumes both independence and identical distri-bution within each sample. An advantage of the Kruskal–Wallistest is that it can be used for any distribution. Nevertheless, theKruskal–Wallis test assumes sample populations, which differonly in location but not in dispersion or distributional shape.Therefore, an alternative statistical test was employed, that is,“test A” which is described in Appendix C. In addition to theaforementioned advantages of the Kruskal–Wallis test, an ad-vantage of statistical “test A” is that it assumes arbitrary samplepopulations.

For the course “Automatic Control Systems,” the null hy-pothesis H0 was tested using populations of samples with sizes

and . The computed test-statistic value ofthe Kruskal–Wallis was significant at a “less than 0.1%” level.Moreover, using statistical “test A” the null hypothesis H0 wasrejected with confidence 99.9%, as explained in Appendix C.Therefore, hypothesis H0 is not statistically supported. Fig. 5shows two (percentage) histograms of student marks in thiscourse for two groups of students. In particular the histogramin Fig. 5(a) was produced from marks of a group of studentsinstructed in “Automatic Control Systems” using traditionalclassroom techniques; whereas, Fig. 5(b) was produced fromthe marks of another group of students instructed in “AutomaticControl Systems” using ACES. In view of the aforementionedstatistical rejection of null hypothesis H0, it is reasonable toconclude that the use of ACES has improved student marks.

Further, the null hypothesis G0 regarding the student interestin the course, “Automatic Control Systems,” was tested usingpopulations of samples with sizes and .For the Kruskal–Wallis the test-statistic value was significantat levels in the range 25%-50% for the various questions in theattitude questionnaire. Using statistical “test A” (Appendix C),


(a)

(b)

Fig. 5. Histograms of student marks (percentage-wise) in the course,“automatic control systems,” for two groups of students: (a) A group ofstudents instructed using traditional (classroom) instruction. (b) A group ofstudents instructed using the ACES software. An improvement of studentperformance in the final (written) exam has been inferred statistically whenACES was used in the educational process as explained in the text.

TABLE IVALUES FOR PERCENTILES z AND THE

CORRESPONDINGCONFIDENCEc = u

the null hypothesis G0 was accepted for the various questions inthe attitude questionnaire, in particular, the difference between

and was not statistically significant (see in Appendix Cand Table I). Therefore, it is reasonable to conclude that theuse of ACES in the educational process did not change studentinterest in the course, “Automatic Control Systems.”

For the course, “Classic Automatic Control,” the null hy-pothesis H0 was tested using populations of samples with sizes

and . For the Kruskal-Wallis the test-statisticwas significant at a level near 50%; furthermore, using statistical“test A” (Appendix C), the null hypothesis H0 was accepted asexplained in Appendix C and Table I. Hence, it is reasonableto accept the null hypothesis H0. Fig. 6 shows two (percentage)histograms of student marks in this course for two groups ofstudents. More specifically, the histogram in Fig. 6(a) was pro-duced from marks of students instructed in “Classic AutomaticControl” using traditional classroom techniques; whereas, the

(a)

(b)

Fig. 6. Histograms of student marks (percentage-wise) in the course, “classicautomatic control,” for two groups of students: (a) a group of studentsinstructed using traditional (classroom) instruction, and (b) a group of studentsinstructed using the ACES software. No change of student performance in thefinal (written) exam could be inferred statistically when ACES was used in theeducational process.

histogram in Fig. 6(b) was produced from the marks of studentsinstructed in “Classic Automatic Control” using ACES. In viewof the aforementioned statistical acceptance of null hypothesisH0, Fig. 6(a) and (b) jointly confirms that the use of ACES in theeducational process did not affect student marks in the course,“Classic Automatic Control.”

Further, the null hypothesis G0 regarding the student interestin this course was tested statistically using populations withsizes and . For the Kruskal-Wallis, thetest-statistic value was significant at a level around 25% forthe various questions in the attitude questionnaire. Furthermore,using statistical “test A” (Appendix C) the null hypothesis G0was accepted for the various questions in the attitude question-naire, more specifically, the difference betweenand wasnot statistically significant (see in Appendix C and Table I).Hence, it is reasonable to conclude that the use of ACES in theeducational process did not change the student interest in thecourse, “Classic Automatic Control.”

A remark is useful here regarding the applicability of “test A.”More specifically, the reader is cautioned not to be misled by theshapes of histograms (of populations) in either Fig. 5 or Fig. 6which (histograms) apparently do not correspond to a normalprobability distribution. Nevertheless, the average value of anyof the aforementioned populations is (for all practical purposes)normally distributedas explained in Appendix C; therefore, sta-tistical “test A” is applicable.

B. Explaining the Results

Regarding the null hypothesis H0, since the employment ofACES resulted in different final exam performances in two dif-ferent automatic control courses, an explanation was sought.The following explanation is postulated from an instructor’sviewpoint.


On the one hand, an improvement of student marks in thecourse, “Automatic Control Systems,” has been attributed tothe fact that the aforementioned course is basically concernedwith novel concepts and algorithms. It appears that the use ofACES in the educational process improved understanding ofboth novel concepts and algorithms resulting in a clear improve-ment in the marks. On the other hand, the course, “Classic Au-tomatic Control,” apart from novel concepts and algorithms, isalso concerned with hand-drawn diagrams, such as approximateBode plots. One can hypothesize that the use of ACES by it-self cannot improve students’ skill for hand-drawing diagrams.Therefore, it can be concluded that the latter is the reason theuse of ACES did not result in any statistically significant im-provement in student marks in the course, “Classic AutomaticControl.”

Regarding the null hypothesis G0, the above mentioned sta-tistical results confirm similar results reported in the literature[13], [16].

VI. CONCLUSION

Learning “on-line” is considered to be one of the fastest-moving trends in higher education [17]. In this paper, an interac-tive software platform, namelyACES, was presented for self-in-struction and self-evaluation in automatic control systems.

The effectiveness of ACES on the educational process wasassessed statistically as detailed in this work based on studentmarks in the final (written) exam and on student answers inthe multiple-choice attitude questionnaires in two different au-tomatic control courses. No special treatment or (grade) bonuseswere given to the group of students who used ACES. Results oftwo different statistical tests have confirmed that ACES can im-prove performance of students in the final (written) exam in anautomatic control course; whereas, the student attitude toward acourse did not change with the use of ACES.

From our experience it appears that ACES has motivated stu-dents for further study. That is, it seems that ACES has encour-aged students to study at home. Nevertheless, no relevant statis-tical study has been documented in the context of this paper.

With some additional work, the ACES platform can be devel-oped in the Java programming language and thus be launchedin the WWW. In the long run, the ACES platform could accom-modate educational software for other courses as well.

APPENDIX A

This appendix explains, by example, how “fuzzy gradingprinciples” have been applied; furthermore, the correspondingrationale is explained.

If the correct answer to a problem question is and thestudent turns in , he/she receives full credit (100%) in thisquestion. Any other answer than is wrong, and it shouldnot receive full credit. Nevertheless, it makes common sense toargue that a wrong answer in the neighborhood of the correctanswer such as “ ” might “deserve” some (partial) credit;whereas, another wrong answer far from the correct answer suchas “ ” does not “deserve” any credit. In this context, fuzzy

Fig. 7. Fuzzy sets with triangular membership functions are used by ACESto give potentially partial credit to a wrong answer. For the fuzzy membershipfunction shown above a student receives full (100%) credit for turning in thecorrect answerx = 7; whereas, a student receives partial (50%) credit forturning inx = 6:5. For turning in eitherx � 6 or x � 8 a student receives nocredit.

grading principles [18] have been applied as illustrated in thefollowing.

Consider a fuzzy set with isosceles triangular membershipfunction as shown in Fig. 7. Hence, for a student answer, ,full credit (100%) is given; furthermore, for , partialcredit (50%) is given (Fig. 7), etc. Moreover, for turning in an-swers either smaller than 6 or larger than 8, a student receivesno credit.

The exact shape of the fuzzy membership function employed(e.g., triangular, bell-shaped, etc.) and the corresponding span ofthe fuzzy membership function are user-defined. In the contextof this work triangular fuzzy membership functions have beenused with variable span.

In sum, “fuzzy grading principles” have been employed as ex-plained in this Appendix for giving students progressively largercredit for turning in answers closer to the correct one.

APPENDIX B

This appendix describes the attitude questionnaires used inthe courses “Automatic Control Systems” and “Classic Auto-matic Control.” An attitude questionnaire included 16 and 22multiple-choice questions for the courses, “Automatic ControlSystems” and “Classic Automatic Control,” respectively.

On the one hand, typical questions in the attitude question-naire for the course, “Automatic Control Systems,” included:“Do you find this course interesting?” “How well are youfamiliar with the concepts Controllability/Observability?”“How easily can you design a PID controller for a second-ordersystem, such that the closed-loop system satisfies certainsteady-state error, and overshoot specifications?” On the otherhand, for the course, “Classic Automatic Control,” typicalquestions in the attitude questionnaire included: “Do you findthis course interesting?” “How well are you familiar withdrawing a Bode Plot?” “How difficult is it for you to sketch aRoot Locus?”

For each multiple choice question a student was requestedto check in one of four boxes:little (1), enough(2), much(3),very much(4), where a number within parentheses was meantto quantify the corresponding linguistic term.


APPENDIX C

A statistical test, namely “test A,” is described in this ap-pendix for testing either null hypothesis H0 or G0. Test A isproposed here for overcoming a restrictive assumption of theKruskal-Wallis test, that is in particular, the assumption that“sample populations should differ only in location but not indispersion or distributional shape.”

For testing either hypothesis H0 or G0, two populations {}and { } of samples were available, having sizes and ,respectively, where population {} corresponded to studentstaught by traditional classroom techniques and population {}corresponded to students who used ACES.

Based on the Central Limit Theorem the average of a popu-lation of independent, identically distributed (i.i.d.) samples is,for all practical purposes, normally distributed, for populationsizes greater than 30 [19]. All the populations of samples usedin this work had considerably larger sizes than 30.

For each sample { } or { }, unbiased estimates forboth the mean and the standard deviation were cal-culated, respectively, [19] by ,

, and 2, re-spectively, for samples { } and { }.

Statistical “test A” is described in the following. Thenullhypothesisthat “samples { } and { } derive from the sameprobability distribution” was tested versus thealternative hy-pothesisthat “samples { } and { } derive from the differentprobability distributions with means and , respectively”.

It holdsfor real number . Hence, de-

noting by the percentile of the standard normaldensity [19], it follows, with very good approximationfor the population sizes considered in this work, that

implieswith confidence .

For two populations of samples {} and { } let there be. Then, if , it follows

that is different than with confidencelarger than .Confidencenumbers can be calculated from standard

Gaussian distribution tables. Selected values for percentilesas well as the correspondingconfidencenumbersare shown in Table I.

REFERENCES

[1] W. A. Wulf, “How shall we satisfy the long-term educational needs ofengineers?,”Proc. IEEE, vol. 88, pp. 593–596, 2000.

[2] Strategic Plan (2001). [Online]. Available:http://www.ewh.ieee.org/soc/es/plan.html

[3] M. A. Vouk, D. L. Bitzer, and R. L. Klevans, “Workflow and end-userquality of service issues in web-based education,”IEEE Trans. Knowl-edge Data Eng., vol. 11, no. 4, pp. 673–687, 1999.

[4] Information Technologies ESPRIT Theme: IT for Learning and Trainingin Industry (1998, Oct.). [Online]. Available: http://www.cordis.lu/es-prit/home.html

[5] V. Petridis, V. G. Kaburlasos, S. Kazarlis, L. Petrou, and G. Hassapis,“Simulation and hypertext: Software for instruction and practicein real-time systems” (in Greek), inProc. Panhellenic Conf. GreekEducational System sponsored by the Greek Ministry for Education,Athens, Greece, Sept. 21–23, 2000, pp. 200–206.

[6] B. Oakley, “A virtual classroom approach to teaching circuit analysis,”IEEE Trans. Educ., vol. 39, pp. 287–296, 1996.

[7] T. G. Engel and M. Jackson, “Discrete-Time analysis of linear and non-linear systems using analog circuit simulators,”IEEE Trans. Educ., vol.42, pp. 205–211, 1999.

[8] I. Levin and D. Mioduser, “A multiple-constructs framework forteaching control concepts,”IEEE Trans. Educ., vol. 39, pp. 488–496,1996.

[9] C. Chou, “Developing hypertext-based learning courseware for com-puter network: The macro and micro stages,”IEEE Trans. Educ., vol.42, pp. 39–44, 1999.

[10] S. E. Poindexter and B. S. Heck, “Using the web in your courses: Whatcan you do? What should you do?,”IEEE Contr. Syst., vol. 19, pp. 83–92,1999.

[11] B. Wittenmark, H. Haglund, and M. Johansson, “Dynamic pictures andinteractive learning,”IEEE Contr. Syst., vol. 18, pp. 26–32, 1998.

[12] R. C. Garcia and B. S. Heck, “Enhancing classical controls education viainteractive GUI design,”IEEE Contr. Syst., vol. 19, pp. 77–82, 1999.

[13] G. J. C. Copinga, M. H. G. Verhaegen, and M. J. J. M. van de Ven,“Toward a web-based study support environment for teaching automaticcontrol,” IEEE Contr. Syst., vol. 20, pp. 8–19, 2000.

[14] J. Apkarian and A. Dawes, “Interactive control education with virtualpresence on the web,” inProc. Amer. Contr. Conf., June 2000, pp.3985–3990.

[15] Y. B. Lee and J. D. Lehman, “Instructional cuing in hypermedia: A studywith active and passive learners,”J. Educational Multimedia and Hyper-media, vol. 2, no. 1, pp. 25–37, 1993.

[16] T. R. Rhoads and N. F. Hubele, “Student attitudes toward statistics beforeand after a computer-integrated introductory statistics course,”IEEETrans. Educ., vol. 43, pp. 182–187, 2000.

[17] J. M. Tien, “Individual-centered education: An any one, any time, anywhere approach to engineering education,”IEEE Trans. Syst., Man, Cy-bern. C, vol. 30, no. 2, pp. 213–218, 2000.

[18] J. R. Echauz and G. J. Vachtsevanos, “Fuzzy grading system,”IEEETrans. Educ., vol. 38, pp. 158–165, 1995.

[19] A. Papoulis,Probability, Random Variables, and Stochastic Processes,third ed. New York: McGraw-Hill , 1991.

Vassilios Petridisreceived the diploma in electrical engineering from the Na-tional Technical University, Athens, Greece, in 1969, and the M.Sc. and Ph.D.degrees in electonics and systems from King’s College, University of London,U.K., in 1970 and 1974, respectively.

He has been a consultant with the Naval Research Center, Greece, Director ofthe Department of Electronics and Computer Engineering, and Vice-Chairmanof the Faculty of Electrical and Computer Engineering with the Aristotle Univer-sity of Thessaloniki, Greece. He is currently a Professor with the Department ofElectronics and Computer Engineering, Aristotle University of Thessaloniki. Heis a coauthor of the monographPredictive Modular Neural Networks: Applica-tion to Time Series(Boston, MA: Kluwer Academic, 1998) and of a chapter en-titled “Learning and decision-making in the framework of fuzzy lattices” in thebookNew Learning Paradigms in Soft Computing, L.C. Jain and J. Kacprzyk,Eds. (New York: Physica-Verlag, 2002). He is the author of five books on con-trol and measurement systems. He is also an author or coauthor of more than50 research papers published in scientific journals and of more than 70 researchpapers published in conference proceedings. His research interests include con-trol systems, machine learning, intelligent and autonomous systems, artificialneural networks, evolutionary algorithms, fuzzy systems, modeling and identi-fication, robotics, and industrial automation.

Spyros Kazarlis was born in Thessaloniki, Greece, in June 1966. He receivedthe Dipl. Eng. degree and the Ph.D. degree, both in electrical engineering, fromthe Aristotle University of Thessaloniki, Greece, in 1990 and 1998, respectively.

Since 1986, he has been working as a computer analyst, programmer, andlecturer for public and private companies. Since 1990, he has been working asa researcher with Aristotle University. Since 1998, he has also been a professorwith the Technological Educational Institution (T.E.I.) of Thessaloniki. His re-search interests include evolutionary computation (genetic algorithms, evolu-tionary programming, etc.), constrained optimization, artificial neural networks,software engineering and computer technology.

Dr. Kazarlis is a Member of the Society of Professional Engineers of Greeceand the Research Committee of EVONET.


Vassilis George Kaburlasosreceived the Diploma degree from the NationalTechnical University of Athens, Greece, in 1986, and the M.Sc. and Ph.D. de-grees from the University of Nevada, Reno, in 1989 and 1992, respectively, allin electrical engineering.

He has made contributions to technological projects, funded both publiclyand privately, in the U.S. and in the European Union. He has authored or coau-thored more than 40 scientific research papers. Also, he is coauthor of a bookchapter entitled “Learning and decision-making in the framework of fuzzy lat-tices,” in the bookNew Learning Paradigms in Soft Computing, L.C. Jain andJ. Kacprzyk, Eds. (New York: Physica-Verlag, 2002). He is currently an Asso-ciate Professor with the new Department of Industrial Informatics, Technolog-ical Educational Institute of Kavala, Greece. His research interests include thedevelopment of computationally efficient techniques for industrial applicationsbased either separately or jointly on numeric and nonnumeric data.

Dr. Kaburlasos is a Member of several professional, scientific, and honorsocieties around the world including ACM, INNS, the Technical Chamber ofGreece, Sigma Xi, Phi Kappa Phi, Tau Beta Pi, Eta Kappa Nu.

Documents

ACES: an interactive software platform for self-instruction and self-evaluation in automatic control systems