Session T1C

978-1-4673-2418-2/12/$31.00 ©2012 IEEE August 20–23, 2012, Hong Kong IEEE International Conference on Teaching, Assessment, and Learning for Engineering (TALE) 2012

T1C-7

A Procedural Approach to Assessing Undergraduate Students’ Project

Madhavan Shanmugavel and Veera Ragavan Mechatronics, School of Engineering Monash University Sunway campus

Bandar Sunway, Malaysia 46150 {madhavan.shanmugavel, veera.ragavan}@monash.edu

Abstract—A systematic procedure for evaluating undergraduate students’ project is presented. A mechatronics project is a synergistic fusion of various disciplines to design and develop a system and a set of specific requirements have to be established for each project component and the students’ achievements are evaluated against these requirements. The project is assessed and evaluated in an industry-like scenario. During the project span of 12 weeks, the third-year Mechatronics students are assessed for the following skills: proposal writing, project management, technical document writing, presentation, group management, risk assessment and use of Gantt chart to track the project. The proposed procedure was tested on two batches of student cohorts and to two competitive projects, and found successful.

Index Terms—mechatronics; assessment; student project; project evaluation; project requirement

I. INTRODUCTION A project is evaluated by various metrics: time of

completion, success of completion, time to market, sustainability, public acceptance, demand in market, major cost cutting, etc. The industrial project is driven by market demand, short and long term investments, and benefits. Mechatronics project is a fusion of various disciplines (mechanical, electrical and electronics, computation, embedded, and information) to produce a system. In modern times, many mechatronic products are available in the market. However, student level mechatronics project may not have the commercial appeal sometimes; but a systematic approach to the project, if followed by the students is always a path to success. The evaluation and success of students’ mechatronic project are measured based on the concept of an industrial product development criteria. In this paper, we present a systematic approach to evaluate the students’ project in terms of project proposal, feasibility, costing, project management, log book and journal maintenance, risk and time managements, communication, presentation, and possibility of commercialization.

The aim of the mechatronics project is to impart the students with real life skills such as project planning, improved inter-personal communication, role-play and teamwork. The students are motivated to acquire these skills and evaluated through the application of these skills in a group project to

design and build a mechatronic system. A typical mechatronic system will consist of hardware components such as a controller (usually a microcontroller) together with appropriate mechanical structures, mechanisms, sensors and actuators integrated to work as a synergistic whole using a software program.

The objectives of this approach are: 1) Prepare the students for semi-industrial environment; 2) Inculcate hands-on skills in project; 3) Time management; 4) Proposal writing; 5) Portrait the students as project managers, technical lead, program lead – equivalent roles in the industry.

The industrial approach of designing, assigning projects and evaluating the same based on constructive alignment is discussed in [1]. The idea enables the students to build their own knowledge upon their existing knowledge. Also, it empowers the students with active learning. Engaging the students in projects to learn by communicating each other with complementary skills and knowledge is called cooperative learning [2], [3]. It promotes active learning and critical thinking, learning by sharing information which promotes written and oral communications and build complementary skills. The team project provides a positive outlook on learning experience, and positive attitude towards the unit (subject). The students can attain higher levels in Bloom’s taxonomy.

Best practices, strengths weakness, and group performance in new product development in Industry well analyzed in comprehensive article [4]. Though the paper discusses cross-functional teamwork towards new product development, the methods, approaches, and outcomes are valid for academic project settings. The project based learning shows positive outlook in interpersonal skills, sense of responsibility, emotions, engineering thinking, and creativity and intuition [5]–[7]. In this paper, we present a procedure to evaluate the students’ project and details on various components used. The proposed approach is validated on students’ competitions and found successful.

The paper is organized as follows: The following section discusses the unit content of third-year mechatronics students’ project in the following section. It contains the synopsis, and learning outcomes. In Section III, the project evaluation procedures are discussed in detail for each phase of the project. Section IV discusses a case study of the proposed evaluation

Session T1C

978-1-4673-2418-2/12/$31.00 ©2012 IEEE August 20–23, 2012, Hong Kong IEEE International Conference on Teaching, Assessment, and Learning for Engineering (TALE) 2012

T1C-8

procedure and its outcome. The paper ends with conclusions in Section V.

II. UNDERGRADUATE-LEVEL THIRD-YEAR MECHATRONICS PROJECT

A. Synopsis of Unit Students will learn the planning and communication skills

required to undertake a group project. An introduction will be given to the evolution of mechatronic technologies, design tools and methodologies, concurrent engineering design support tools, mechatronic design process and requirement interpretation. The acquisition of these skills will be motivated and tested by applying them in a group project to design and build a mechatronic system. The mechatronic system will be based on a microcontroller together with appropriate mechanical structure, sensors and actuators.

B. Learning Outcomes The students should able to understand the role and responsibilities of a project manager, and also able to understand the design support tools, and their use in system development. The project management skills and techniques should be acquired and applied at various stages of the project. The students acquire the skills required for developing practical mechatronics systems, and able to develop alternate design and testing methods by understanding the interrelationships between the components. The students should reflect their acquired knowledge, and skills in the form of technical report. An effective communication and teamwork should be reflected by maintaining individual and group log book, journals, and presentation. The learning outcomes are assessed and evaluated worth 70% for project, and 30% for theoretical examination.

III. PROJECT EVALUATION

A. Project Evaluation The project evaluation is divided into three phases: 1) proposal and feasibility; 2) progress management; and 3) project closing. Each phase is carried out to evaluate the intended learning outcomes drawn in Section II(B). The evaluation criteria are made in a rubric. Evaluation of “other skills” like real-life competition, peer pressure, sense of responsibility, cope with change, response to challenges that cannot be evaluated in academic exam settings can only be evaluated through national and International competitions. Hence, the proposed approach is validated in students’ competitions. The competitions may not provide the students in acquiring knowledge from the books. But, they encourage the students to practice, and realize the theory through experiments, and acquisition of knowledge through sharing with team members.

B. Project Rubrics The first phase is evaluated at the first two weeks of the

project. The project evaluated for consistent progress, and time and risk management assessment in the second phase scheduled at the seventh week of the project. The third and

final phase of the project evaluates the project completion with the final project report, along with the professional attitude of the students in achieving the project’s objectives.

The project proposal is refined and finalized in the first two credit weeks. The rubrics [8] with marking scheme are shown in Table I.

TABLE I. PROJECT PROPOSAL RUBRIC

S. No Rubric Max. Marks

0 Appearance, cover sheet, etc. 1

1 Introduction 1

2 Aim and objective of the project 2

3 Literature survey /state of art 2

4 Scope of work / problem statement 2

5 Tech. content / system design / modularity and layout

5

6 Planning and scheduling (task breakdown and person-in-charge (PIC), Gantt chart, resource planning / bill of materials)

5

7 Conclusion (different from Intro) 1

8 Benefits accrued 2

9 Reference section in the IEEE format 1

10 Language and communication 3

11 Innovative and executable 5

Total marks 30

The proposal will be evaluated by answering following questions:

1) Is the project idea proposed by the student?

2) is it innovative and executable?

3) Are the objectives, scope, roles and responsibilities and plan of work well defined?

4) Is the literature review adequate? System design, design alternatives evaluated?

5) Is the risk management plan and project plan and timeframe reasonable and realistic?

6) Does the student have a clear idea on how to go about carrying out the project?

The second phase of the project evaluates the project progress, and discussion, and journal and log book maintenance to track the project.

The progress report evaluates the following:

1) Has the student achieved the targets set?

2) Has the student identified the problems faced?

3) Is there a mitigation plan?

Session T1C

978-1-4673-2418-2/12/$31.00 ©2012 IEEE August 20–23, 2012, Hong Kong IEEE International Conference on Teaching, Assessment, and Learning for Engineering (TALE) 2012

T1C-9

4) Does the student have a clear idea on what is to be done to move forward?

5) System design, design alternatives re-evaluated?

The logbook, and journal maintenance evaluates the following components:

1) Has the student shown a good understanding on the concepts in the project?

2) Has the student shown initiative, diligence and an ability to work under minimal supervision?

3) Has the student shown a good attitude in the project? (e.g., punctuality, professionalism)

4) Has the student effectively contributed to the project? (e.g., teamwork, management, tech lead)

5) Is there a systematic approach to taking notes in the logbook?

6) Appearance of the logbook/journal.

7) Indication of dates when various tasks/events were carried out?

8) Notes of special visits to industries/companies/external institutions, etc. (if any)?

9) Written notes of discussion with supervisor(s)?

10) Written codes/algorithms/programs in logbook?

The third phase of the project evaluates the students’ presentation and professional attitude in completing the project. Also the final report and its quality is evaluated at this phase. It consists of four components: 1) presentation; 2) organization; 3) material content and achievement; and 4) demonstration, and questions and answers. Table II gives the details of rubrics followed during signing off the project.

TABLE II. OVERALL PROJECT EVALUATION RUBRICS

Continuous assessment Oral presentation

Fina

l Rep

ort (

10)

Tot

al M

arks

(70%

)

Proj

ect p

ropo

sal a

nd

feas

ibili

ty re

port

(10)

prog

ress

repo

rt (1

0)

Jour

nal/d

iscu

ssio

n w

ith

supe

rvis

or (1

0)

Pres

enta

tion

(4)

Org

aniz

atio

n (6

)

Con

tent

/ A

chie

vem

ent (

10)

Dem

o, Q

&A

(10)

7 8 7 2.5 6 10 8 9 58

8 7.5 8 3 6 9 9 8 59

The presentation component involves students’ style and attitude in communication and detailing the project. The presentation style for each student will differ for their role either as a project manager, or tech- or program-lead. The

marking will be based on voice whether the voice is clear and audible. Body posture (either closed or open), eye contact, use of gestures, movement during presentation are also assessed. Not but the least, the delivery of presentation is assessed whether sincere, enthusiastic, superficial, monotonous, unnecessary body movement are also evaluated.

The organization of report is weighted with six marks in which flow of information from chapter to chapter, section to section are evaluated for logical sequence. Visual aids in the form of figures, pictures, graphs, or videos, are also evaluated. These aids should be attached with relevant details. The organization of the report is mandatory with appropriate contents: motivation, introduction, literature review, simulation, experiment, analysis, results and discussions. Finally, the students should able to demonstrate individually their work to the audience and able to answer the questions relevant to the project work.

The procedure is adopted in various projects during the semester. We believe that the successful implementation of this systematic approach had enabled the students to win the Smart Car competitions consistently in 2010 and 2011. Our students were from third-year vs. final-year and Masters students of other universities.

IV. CASE STUDY The proposed evaluation procedure was tested on two tight

scheduled competitions: Freescale Cup. As a case study, we present the details of competitions, and planning which lead to victory to us in them.

A. Freescale Cup – A Brief Introduction The Freescale Cup, or previously known as Smart Car

Competition, is a high-speed, intelligent car racing. This is an effort by Freescale Semiconductor to introduce and promote engineering applications in the classroom, especially amongst universities. It is open to all public and private universities in Malaysia, and is also held in many countries around the world. The participants are from undergraduate to Masters level students. The maximum number of students in the team is restricted to three. The number of rounds and difficult levels are dictated by the sponsors and challenges may be abstract, or may not be known a priori.

In Malaysia the first Smart Car Competition [9] was conducted in 2008. In 2010, the competition was in collaboration with IEEE Malaysia. In 2011, Smart Car Competition was rechristened as The Freescale Cup. This competition caught the attention of MOHE (Ministry of Higher Education), and hence was included to be a part of PECIPTA 2011 [1]. The theme of the competition differs each year. In 2010, the theme was high-speed and quick completion of circuit in indoor under normal day-light conditions. The theme for year 2011 competition was “The Day Dragger” and “The Night Drifter”, in which environmental conditions differ. The track also includes obstacles such as turns, curves, slopes, tunnels, as well as uneven track sections.

The objective of the competition is to design and race a Freescale microcontroller based smart car which is independent

Session T1C

978-1-4673-2418-2/12/$31.00 ©2012 IEEE August 20–23, 2012, Hong Kong IEEE International Conference on Teaching, Assessment, and Learning for Engineering (TALE) 2012

T1C-10

of external control and capable of tracking a black line on a white track autonomously, and complete the track as fast as possible without losing smoothness of car movement while overcoming all the obstacles on the track. The competition is judged based on the sum of points covered by several criteria: tracking system, i.e. the ability to track the line, acceleration and speed control at turning, smoothness of car movement, and time taken to complete the track. Fig. 1 shows the smart car circuit, Fig. 2 shows the exploded view of the theme and the smart cars from Monash University is shown in Fig. 3.

Figure 1. Smart car circuit.

Figure 2. Smart car circuit – exploded theme.

Figure 3. Monash teams.

The following technical skills evaluated directly or indirectly:

1) Design of microcontroller peripherals and interfacing;

2) Microcontroller programming, fine tuning vision system;

3) Sensors, encoders;

4) Actuators, servo’s DC motors, etc.;

5) Path following controller, curve hugging, speed;

6) Environmental disturbances (vision system) system integration.

Every team is supplied with a basic kit which includes a freescale processor, chassis and other related components. The students have to design, build, and tune the system using the processor, components, and software. Under the tight time-line of two and a half months, the students have to prepare the smart cars for the competition. Out of six months duration, the first month involves registration and faculty training, and the last two and a half months spent on competitions. A careful planning and execution is crucial for every success of the project.

B. Project Planning and Challenges The project is well defined. But the normal time line of six

months is shrank into two and a half months. This provides the students a challenge to build a system, learning new microcontroller, and programming, testing and validating the system in a short time. Students have to scrutinize the time planning and management. Acquiring the knowledge of new system, executing the project planning, competing with heterogeneous teams (with Masters level students and final year undergraduate students), and sharing information through effective communication are achieved at various levels using the proposed procedure.

As a part of Bloom’s taxonomy, the project work develop psychomotor skills rank 3, the soft skills like communication, project proposal, project management develop cognitive skills of rank 2 and affective skills of rank 3. The two successive victories in the year 2010, and 2011 show that the proposed procedure is effective in preparing the students to get prepared to meet the challenges, time and project management in time-constrained competitions, and importantly ability to acquire new knowledge and realizing the same in short time.

V. CONCLUSION We propose an evaluation procedure for undergraduate

projects which was successfully integrated to the unit. The industry-like procedure helps the students to gain practical skills on project management, time management, knowledge sharing, improved communication and report writing. The proposed procedure was tested on two tight scheduled competitions: Freescale Cup and found success.

ACKNOWLEDGMENT We would like to thank Freescale Malaysia for their

support and sponsorship for Freescale Cup. Also, we would like to thank Mr. Paneerselvam and Mr. Bathamanathan for providing lab facilities. Finally, thanks to the students of Monash University who participated in the Freescale Smart Car Competition in 2010 and 2011.

REFERENCES [1] J. B. Biggs, “Enhancing teaching through constructive alignment,”

Higher Educ., vol. 32, no. 3, pp. 347–364, Oct. 1996.

Session T1C

978-1-4673-2418-2/12/$31.00 ©2012 IEEE August 20–23, 2012, Hong Kong IEEE International Conference on Teaching, Assessment, and Learning for Engineering (TALE) 2012

T1C-11

[2] L. Moreno et al., “Use of constructivism and collaborative teaching in an ILP processors course,” IEEE Trans. Educ., vol. 50, no. 2, pp. 101–111, May 2007.

[3] Center for Teaching and Learning, Stanford University, “Cooperative learning: Students working in small groups,” Speaking of Teaching, vol. 10, no. 2, pp. 1–4, Winter 1999.

[4] S. Holland et al., “Critical success factors for cross-functional teamwork in new product development,” Int. J. Manage. Reviews, vol. 2, no. 3, pp. 231–259, Sep. 2000.

[5] A. Mohan et al., “Professional skills in the engineering curriculum,” IEEE Trans. Educ., vol. 53, no. 4, pp. 562–571, Nov. 2010.

[6] S. J. R. Ivins, “Interdisciplinary project work: Practice makes perfect?,” IEEE Trans. Educ., vol. 40, no. 3, pp. 179–183, Aug. 1997.

[7] M. Frank et al., “Implementing the project-based learning approach in an academic engineering course,” Int. J. Technol. & Design Educ., vol. 13, no. 3, pp. 273–288, 2003.

[8] C. A. Mertler, “Designing scoring rubrics for your classroom,” Practical Assessment, Research, & Evaluation, vol. 7, no. 25, 2001.

[9] Freescale Semiconductor. (2011), The Freescale Cup Malaysia Page [Online]. Available: http://streetsmarts.freescale.com/profile/thefreescalecupmalaysia/.