COMPUTER SCIENCE (CS)

Abdul Razaque

This report provides indication that students are accomplishing end-of-program learning objectives and that graduates are achieving accomplishment outcomes set by the program.

Name of the program: Computer Network (Computer Science): conducted by Dr. Abdul Razaque

Year (e.g., AY17-18) of assessment report: 17/18 Date Submitted: 1 June 2018

Contact: arazaque@nyit.edu

1. GOALS Program outcomes:

Analytical and Critical Thinking: students can make decisions and solve the problems based on

the research, logic, and quantitative and qualitative and analyses of Correct and appropriate data and information.

Upon completion of the program, students will be able to: Describe the goals of conducting the survey and its overall procedure. Identifying the problem to be explored. Obtaining the information from existing literature. Describe considerations and possibilities for designing the sample. Describe steps in employing the survey. Analyzing the evidence. Describe implications, limitations, assumptions, and recommendations based on the analyses.

2. METHOD

Assessment of students’ course CSCI 345: Computer Networks was done. Total 229 students from the computer science department participated in the assessment. The participants were from three levels: Freshman, Sophomore and Junior. There were 90 Freshmen from F4-F7 cohorts, 89 Sophomore from S4-S7 cohorts and 120 Juniors from J4-J7 cohorts. A survey questionnaire consisting of 12 questions to assess the selected PLO was formulated. The survey was designed according to basic knowledge of all levels of students in the field of computer networks. The survey aimed to determine the students’ capability, knowledge, and awareness about Computer Network and Security. The tools and its rubric were designed and developed by the computer scientist and researchers that agree with the same way in each statement. The 5-Likert scale is applied with 11 Survey Questions; 1= Poor, 2= Below Average, 3= Average, 4= Above Average and 5= Excellent. The junior students were currently enrolled in the CSCI 345: Computer Networks, but Freshmen and Sophomores also did not have problems to understand survey questions which were related to basic idea of Internet and security and known by majority of the students using Internet. Sample survey questions were provided given in Table 1.

TABLE 1: Survey questions for PLO Questions # Survey Question Description

2 Internet is the best source of exchanging the educational information

3 Anti-Virus and Firewalls protect your data from attacks of virus and worms

4 Bus topology performs better for all types of data communication network

5 Local area network has faster data rate than Wide area network

6 Robust Algorithms support faster data transmission over the network

7 You prefer portable device such as mobile phone over Desktop computer and Laptop

8 Encryption protects all types of data over the Internet

9 Detecting vulnerabilities are more important than detecting threats in the cyber security

10 Special purpose educational mobile Apps are better than desktop mobile Apps

11 Ciphertext method protects data when sending over the Internet

12 Internet helps students and improves the learning process

The responses from all levels including all cohorts of Computer science are sorted out and evaluated on the following metrics. Trend of all levels for each scale

An average score obtained by all students on each question.

Percentage score of all students on each scale.

Comparison of three levels based on each scale.

3. ANALYSIS AND OUTCOMES

A. Trend of all levels for each scale

In Figures 1-2, the trend of all levels of student is shown for Questions-2 and 10. The statistical data show in the Figure-1, the junior students have 78% excellent choice as compared with freshmen and sophomores who chose 2% and 16% respectively. On poor scale there is 0% juniors, but Freshmen and sophomores are 4% and 5% respectively. In Figure-2, the junior students have 15% excellent choice as compared with freshmen and sophomores who chose 4% and 10% respectively. On poor scale, trend for Freshmen, sophomores and juniors have been observed as 12%, 11%, and 6% respectively. The results demonstrate that

junior students have sound knowledge and better understanding about the Questions-2 and 10.

Q 2: Internet is the best source of exchanging the educational information

Figure 1: Comparison of three levels on each scale for Question-2

Q 10: Special purpose educational mobile Apps are better than desktop mobile Apps

Figure 2: Comparison of three levels on each scale for Question-10

0%20%40%60%80%

100%

Poor BelowAverage

Average AboveAverage

Excellent

Q 2:

Freshmen Sophomores Juniors

0%10%20%30%40%50%60%

Poor BelowAverage

Average AboveAverage

Excellent

Q 10:

Freshmen Sophomores Juniors

B. An average score obtained by all students on each question

In Figures 3-5, an average score obtained by different levels is shown. Figure 3 displays the average score of freshmen for all questions. The results demonstrate that freshmen have lower average score for the questions 9 and 11 that is 3.52% and 3.5% respectively. On other hand, highest an average is for question 3 that is 4%. In Figure 4, an average score for sophomore is shown.

Figure 3: Average score for all questions obtained by Freshmen

Figure 4: Average score for all questions obtained by Sophomore

3.23.33.43.53.63.73.83.9

44.1

Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 Q11 Q12

Average of Freshman

0

1

2

3

4

Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 Q11 Q12

Average of Sophomore

Questions 5 and 8 have highest average that is 3.6% for both whereas the question 2 has lowest average that is 2.5%. In Figure 5, an average trend for juniors has been shown. As, an average score of junior students is higher than other two levels. Questions 2 and 3 have highest average score that is 4.3% and 4.3% respectively. The lowest score is 3.7% that is observed for the question 6. The results demonstrate that lowest average score of junior is even higher than highest score of freshmen and sophomores’ students. The results prove that junior has better average than freshmen and sophomores.

Figure 5: Average score for all questions obtained by Sophomore

C. Percentage score of all students on each scale

In Figures 6-8, percentage score of all students is shown on each scale: poor, below average, average, above average and excellent. Figure 6 shows the score of freshmen. As. 8% students selected poor, 17% students are below average, 45% are in the average, 22% are above average and 8% are in excellent scale. The statistical data show that maximum students select the average scale. In Figure 7, percentage for sophomores is shown. From the data to have been observed that 16%, 34%, 22%, 23% and 5% are obtained for poor, below average, average, above average and excellent scales respectively. Figure 8 shows the percentage score for junior students on various scale. As the data show that Juniors did not pick poor and below average option. 22% students chose average, 48% are in the above average and 30% picked excellent option. Based on this result, it is proved that juniors have much more knowledgeable than freshmen and sophomores.

3.33.43.53.63.73.83.9

44.14.24.34.4

Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 Q11 Q12

Average of Junior

Figure 6: Percentage score of all freshmen on different scale options

Figure 7: Percentage score of all sophomores on different scale options

8%

17%

45%

22%

8%

Freshman

Poor

Below Average

Average

Above Average

Excellent

5%16%

34%22%

23%

Sophomore

Poor

Below Average

Average

Above Average

Excellent

Figure 8: Percentage score of all juniors on different scale options

D. Comparison of three levels based on each scale

In Figure 9, three levels have been compared on different scales. The results demonstrate that Junior students picked Excellent and above average options more than Freshmen and sophomores. On other hand, freshmen picked average option more than other two levels. There is interesting point to be noted that juniors did not pick poor and below average. The overall performance of junior is much better than other two levels that proves the knowledge of the topic.

Figure 9: Comparison of three levels: juniors, Sophomores and Freshmen on different scale options

4. INTERPRETATION

0%0%22%

48%

30%

JUNIOR

Poor

Below Average

Average

Above Average

Excellent

0% 5% 8%

0%

16%

17%22

% 34% 45

%

48%

22%

22%30

%

23%

8%

JUNIOR SOPHOMORE FRESHMAN

COMPARISON OF THREE LEVELS (CS)

Poor Below Average Average

Above Average Excellent

Overall results for juniors, Sophomores and Freshmen prove that students have basic

knowledge of computer networks and security. The results demonstrate that Juniors have better understanding of Computer Network

and Security as compared to Sophomore and Freshmen. The main reason of Juniors’ better understanding is currently taking this course. So,

it proves that fresh knowledge of the students about domain/Course could bring substantial difference.

Overall results demonstrate that Juniors have selected right choice for each Question and having lower negative response from them.

5. IMPROVEMENTS PLANNED

A. Action Items

Empirical Instruction and professional enhancement events should be tailored to help the students to recognize applicable literature for their research papers and class projects. NUPT AND NYIT should collaborate to provide such environment to students. In addition, the students should have free access to scholarly Journals, magazines and conference papers.

B. Recommended improvements

Better audio equipment should be installed in the classrooms. Internet does not work most of the classes which makes difficult for the instructor to

demonstrate some videos or live examples as part of the discussion in the class. Internet availability in the classrooms are also very limited sometimes, leading to

the same problem as above. Too many students connected to the Wi-Fi network available in classrooms slows down the data bandwidth.

Overall scores of Juniors prove better understanding of Computer Network and Security. However, there is still room of improvement available that could be filled by taking following measures.

Facility of Laboratory should be provided to students to gain additional knowledge by conducting hands-on-experience.

Study tours to big companies such as Mobile-Unicom, Mobile China should be arranged for students.

Attendance in seminars should be compulsory for all levels: Freshmen, Sophomores and Juniors that could remove the gap with some extend.

Fundamental concept of Computer Network course-1 for Freshmen and Essential Computer Network components course-2 for Sophomore should be introduced

Table 2: Summary of Improvements Made in Response to Assessment Results in the

past few years

Year of Brief Name of Improvements Impact of Assessment Program Learning Implemented Based on Improvements

Results Goal (e.g., Writing) Assessment Results (report reassessment

results if available)

AY12-13 An ability to determine algorithmic efficiency, computability, and resource usage.

External activities for students involving programming, mathematics and mathematical modeling were continued and increased. Dr. Douglas Van Wieren took the responsibility for implementing and following through on relevant external activities with a specific schedule and elements.

NA

AY13-14 An ability to apply programming language concepts such as data models, control structures, language translation and testing and debugging in the development of software systems

a) The class size for programming language were reduced to 30. b) Students were exposed to more lab time/ programming assignments / activities and programming language classes were taught in the lab and not in a general classroom. c) Each programming class got a lab assistant who was helping the teaching faculty to monitor the students’ activity and at the same time helped students to learn some basic programming skills.

There was a 47% increase in the performance of the programming abilities of the students because of the implementation of the assessment recommendations.

AY14-15 An ability to apply programming language concepts such as data models, control structures, language translation and testing and debugging in the development of software systems

None of the recommendations were implemented

NA

AY15-16 An ability to apply programming language concepts such as data models, control structures, language translation and testing and debugging in the development of software systems

None of the recommendations were implemented

NA

AY16-17 An ability to analyze the local and global impact of computing on individuals, organizations and society. Providing continuous professional development, effective communication and understanding the ethical, legal and security issues

Valuable recommendations were suggested in report, but small recommendation was considered by providing the 2 lab assistants for lab courses.

NA

C. Brief Description of Faculty Engagement in the Current Annual Assessment

Report:

Professor Sonali helped entire team as coordinator during assessment and provided the previous years’ reports. Dean Keh’s suggestions also provided valuable suggestion for improvement. I met and chatted with assessment committee members several times and their feedback were incredible. Results from this assessment report were shared with all NYIT faculty during the Annual Assessment retreat day on May 14. Feedback has been constructive demonstrating that thanks to the good efforts of our students and faculties the PLOs are being met.

D. Annual Program Achievement Goals:

I do not have access to this data.

Physics for ECE & CS

Alfonso Reina

This report provides evidence that students are achieving end-of-program learning goals and that graduates are attaining achievement outcomes established by the program.

Name of the program: Physics for Electrical and Computer Engineering (ECE) and Computer Science (CS)

Year of assessment report: AY 2017-2018

Date Submitted: June 1, 2018

Contact: Alfonso Reina

The Statement of Program Learning Goals and Curricular Matrix are available at: http://www.nyit.edu/planning/academic_assessment_plans_reports.

I. Annual Program Learning Assessment:

1. GOALS CRITICAL/ANALYTICAL THINKING. Students make decisions and solve problems based on research, logic, and qualitative and quantitative analyses of appropriate and relevant data and information. Upon graduation students will be able to:

1. Identify and summarize the problem, issue, or question to be investigated. 2. Present existing knowledge, research, and/or views. 3. Design an inquiry process 4. Analyze research/evidence 5. Draw inferences and conclusions from analyses

2. METHOD Comments: the goal stated above was assessed in the context of the course of PHYS170: General Mechanics, which is taken by all ECE and CS students. The course level outcome relevant to the instrument chosen is: “Understanding and Applying Newton’s Laws of Motion”.

2.1. Instrument The Force Concept Inventory, a multiple-choice test of 30 questions, was selected for the study. The test is often used to assess learning outcomes of undergraduate physics courses in the USA. The test measures the understanding of the nature of forces and their role on the motion of objects. The distractors are common misconceptions identified by more than 1000 interviews of college and high school students [1]. Because our students are non-native speakers, three previously published versions of the tests were used: English [1], Simplified English [2] and Chinese [3]. Each student was assigned randomly one version. Sample questions of each versions are included in the appendix. A brief description is given below for each version:

• English: the test is written in college-level English, comparable to that used in common University Physics textbooks used in the USA. Diagrams are not included

in all questions, and it is expected that students can draw inferences from the text to make a mental model of the situation/context of the prompt [1].

• Simplified English: more diagrams are included to help the student to imagine the situation (Figure A1). Less text is used in questions with redundant or irrelevant information (Figure A2). The context of some questions is simplified without changing the learning goal being tested (Figure A3) [2].

• Chinese: the test is a direct translation from English [3].

2.2 Participants The test was given to 92% of students enrolled in ECE and CS during the academic year 2017-1018. This allowed to compare results between majors and across all levels. Freshmen are currently taking the subject assessed, while sophomores and juniors took the class one and two years ago, respectively. The number and percentage (of the total population) of students assessed are listed in Table 1. TABLE 1: Number of participants in assessment process

Major Participant Students Enrolled Percentage (%) ECE 263 269 98% CS 300 345 87% Total (ECE & CS) 563 614 92%

Participants by major, level and version of test taken TABLE 2: showing Major, level and version of test taken

Major/Level Total Participants English Simplified English Chinese ECE Freshmen 88 29 29 30

ECE Sophomores 89 34 29 26 ECE Juniors 86 39 21 26

CS Freshmen 89 36 28 25 CS Sophomores 102 35 34 33

CS Juniors 109 37 34 38 2.3 Test statistics used In all comparisons a T-test and/or ANOVA were used to determine the significance of the means when comparing the groups above. Tests for normality were done by the graphical method with quantile-quantile plots (QQ-plots).

3. ANALYSIS 3.1. Benchmark

The benchmark chosen to compare student performance was a multiyear study conducted at Georgia Tech with 5000 students enrolled in engineering, science and non-STEM programs who took physics courses with various teaching styles [4]. Based on this report, students perform within the 60 to 70% range after completing their physics course. We took this range to be the criteria for satisfactory performance. The results were compared three ways: among majors (ECE & CS), levels (freshmen, sophomores, and juniors) and among types of tests.

3.2 Results by majors and levels (Figure 1)

There was no significant difference between the performances of ECE and CS majors (p>0.78, t-tests among ECE and CS groups within each level), suggesting that the level of learning did not depend on which program students were enrolled in. There was a minor significant difference between juniors and the rest of the levels (p~0.05-0.10, t-test between Juniors and Sophomores or Freshmen for both ECE and CS majors) depicted in Figure 1.

Figure 1. Results by majors and levels. Benchmark for satisfactory performance is shown in red (60-70%). Black bar represents the 95% confidence interval of the mean for each group. ECE is shown in blue and CS in yellow. 3.3 Results by test version (Figure 2 and 3) Figure 2 shows the performance of students grouped by each version of the test. Each group (e.g. English, Simplified English, or Chinese) contains students from all majors and levels (see tables in the method section). Students perform better in the Chinese version, followed by the Simplified English version. The difference of performance between the Chinese and English versions is equivalent to 12 percent points, roughly four questions out of 30. The differences are significant, with a p-values below 0.05 as determined by t-test among each pair and ANOVA among the three groups. Figure 4 shows the comparison

between each test version within each program and by levels. The trend identified in Figure 3 is consistent except CS Sophomores.

Figure 2. Results grouped by test version. The benchmark chosen for satisfactory performance is shown in red. Black bars represent the 95% confidence interval for each group.

Figure 3. Results grouped by test version, levels and majors. The difference between language performance is consistent within levels and programs.

3.4 Item analysis (Figure 4) The percentage of correct responses were calculated for each question in the test, regardless of the version. This was done to identify possible weaknesses and learning gaps. Questions were flagged if the percentage of correct responses was below 60% (based on the benchmark selected of 60-70% as the acceptable range). A total of four questions did not satisfy the criteria (Questions #3, 8, 11 and 23, Figure A4)

Figure 4. Percentage of correct answers for each question of the Force Concept Inventory. The percentage was calculated using the whole population of students without grouping by majors, levels or test version.

4. INTERPRETATION: 4.1 Majors and levels (Figure 1) It is believed that students understand the concept of force well enough independently of their major or level. Freshmen and sophomores perform overall above the benchmark, while juniors perform within the benchmark. It is possible that juniors fall short compared to the other levels because they took this physics course two years ago 4.2 Test version and language (Figures 2 and 3) The data collected shows that test language plays a role in student performance. In average, a student would miss four more questions than a student taking the same test in English. Furthermore, a simplified version of the test improves the odds of a better performance, but it is not equivalent to the results that can be obtained by using the native language version. The test language effect is observed regardless of majors and levels, as shown in Figure 3. The only exception is the case of CS Sophomores, where both the Simplified English and Chinese versions have comparable results. Even in this case the English tests remains with the lowest performance. These results underscore the influence of language during standardize testing of English Language Learners (ELLs.) and the necessary attention needed while designing test items in physics to assess learning during midterms and finals. 4.3 Identifying knowledge gaps (Figure 4)

Only four questions were identified to be problematic to the overall student population. These four questions represent 13% of the content covered, which confirms the notion that students have a good grasp of the concept of forces. The four questions are shown in the appendix. Common themes within these questions include: • The prediction of the trajectory taken by an object after (or during) the action of a force

(Qs #3, 11, 23, Figure A4) • Identifying the force(s) from knowing the trajectory taken by an object (Q #8, Figure

A4))

5. IMPROVEMENTS - PLANNED:

5.1 Test Language and design: It is important to be aware of possible language interferences when assessing and testing students. We currently provide translation in Chinese of challenging vocabulary in midterms and finals. Also, it is a requirement for every physics instructor to provide diagrams in all problems in each test to avoid false negatives during testing. As our results show, simplification of language and the incorporation of diagrams may help mitigate invalid assessment results. Although we believe that testing must be done in the language of instruction, it is important to raise awareness among faculty about the obstacles English Learners may face during science and physics classes. It is also important to be aware of test designs that may not be suitable for English Learners. More detailed analysis may be needed in the future to validate the differences seen among test takers in English, Simplified English and Chinese. For example, it is necessary to interview students to make a detailed description of their thought processes while answering the same question in English and Chinese. It would be good to identify the type of “language traps” that are common among our students to be able to provide more detailed guidelines for test designs. 5.2 Integration of knowledge and skills Although student performance is strong within the concept of force, we believe that student learning can be improved by emphasizing more the connection between the action of a force and the trajectories and properties of motion of the objects affected by such force. Furthermore, it may be necessary that students predict such connections in both ways (from force to trajectory and vice-versa). It is not surprising that such questions tend to be the most challenging to our students. These questions demand students to predict the behavior of common phenomena (rocket propulsion, ice-hockey, and the speed change of a falling orange) but at a detailed level that goes beyond the analysis done by an average person while performing these activities. Moreover, the prediction of the trajectory taken by a rocket in questions 8 and 11 is only possible by drawing on mathematical models that consider Newton’s laws of motion. Students do not only need to be familiar on how a force

acts, but also need to be able to model such action mathematically to predict the shape of the path. This is a task that can only be accomplished by the integration of multiple skills and concepts. Therefore, we believe that future lessons could benefit from including more integration of concepts and skills.

II. Summary of Improvements Made in Response to Assessment Results in the past few years:

Not applicable. This is the first assessment report of physics in the Nanjing Campus.

III. Brief Description of Faculty Engagement in the Current Annual Assessment Report:

The results were processed during the beginning of May due to the narrow window of time given to perform the tasks of this year’s assessment. They will be shared with other physics instructors by the end of the academic year. The themes addressed in this assessment have been discussed multiple times during meetings with physics faculty while planning courses, midterms and finals. All instructors are aware of issues of language that we all must consider during the writing of tests.

IV. Annual Program Achievement Goals:

Not applicable to physics.

Please provide examples of readily available data on program student achievement (e.g., first-year retention rates, six-year graduation rates, average time to degree completion, certification exam pass rate, student satisfaction survey results, employer satisfaction results, % pursuing an advanced degree, % of job placement, etc.)

References

[1] Hestenes, David et al. Force Concept Inventory. The Physics Teacher. 30(3) 141. 1992

[2] S. Osborn Popp and J. Jackson, Can Assessment of Student Conceptions of Force be Enhanced Through Linguistic Simplification? American Educational Research Association 2009, San Diego, CA, 2009.

[3] Guo, Chenyue. Force Concept Inventory. Simplified Chinese Translation. Online: www.physport.org. American Association of Physics Teachers.

[4] M. Caballero, E. Greco, E. Murray, K. Bujak, M. Marr, R. Catrambone, M. Kohlmyer, and M. Schatz, Comparing large lecture mechanics curricula using the Force Concept Inventory: A five thousand student study. Am. J. Phys. 80 (7), 638 (2012).

Appendix

Figure A1. Sample question from the English, Simplified English and Chinese versions of the Force Concept Inventory. The Simplified English version utilizes diagrams and underlining of words

Figure A2. Sample question from the English, Simplified English and Chinese versions of the Force Concept Inventory showing reduction of text in the Simplified English version.

Figure A3. Sample question from the three versions of the Force Concept Inventory showing the simplification of the context implemented in the Simplified English version

Figure A4. Questions identified with performance below the benchmark chosen (60% of correct answers).