SIMCE TIC - · PDF file5 National Results SIMCE TIC Introduction SIMCE is the National System for Learning Outcomes Assessment of the Ministry of Education of Chile. It is mainly intended

3

SIMCE TICNational Results

2011 National Results

SIMCE TIC

National results 2011 SIMCE TICMinistry of Education

ENLACES, Center for Education and Technologywww.enlaces.cl

[email protected] Alameda Libertador Bernardo O´Higgins Avenue 10th floor

Santiago, 8340527Chile.

5


Introduction

SIMCE is the National System for Learning Outcomes Assessment of the Ministry of Education of Chile. It is mainly intended to improve quality and equity in education and report on student performance in the different learning areas. linking such performance to the school and social learning contexts. The specific purpose of the SIMCE TIC assessment is to determine ICT skills for learning among second-year high school students nationwide. The aim is to assess. in a virtual environment. students’ ability to resolve problems and accomplish school work in the digital world. and to discover the individual and contextual factors affecting outcomes.

Intended for all school system actors. this document provides results of the first SIMCE national assessment on Information and Communication Technologies (ICTs) of second-year high school students in 2011(15 years or more).

The document has three chapters. The first describes and characterizes the 2011 SIMCE TIC assessment. The second is a report on assessment results. And the third is a set of annexes that supplement the information provided in this report.

7


Content

Introduction 5

Executive Summary 9

Background 11

Chapter 1: Assessment Characteristics 12

1.1. Skills and Knowledge Test Assessment 13

1.2. Test Characteristics 14

1.3.Test Description 15

Chapter 2: Results 16

2.1. National Results 17

2.2. Regional Results 18

2.3. Results by Socio-Economic Group 20

2.4. Results by School Administration Type 23

2.5 Results by Gender 25

Chapter 3: Annexes 26

9


Executive Summary• The 2011 SIMCE TIC test was implemented from 3 to 30 November and given

to a representative sample of 10.321 second-year high school students from 505 schools throughout the country. The total number of valid records analyzed is equivalent to 92% of all assessed students.

• The purpose of this assessment is twofold: to determine the development level of ICT Skills for Learning reached by students and to discover the individual and contextual factors that influence achievement in the SIMCE TIC test.

• The test for measuring ICT Skills for Learning consists of 32 items answered entirely on a computer with software developed specifically for this purpose. It simulates a virtual environment and provides a desktop with commonly-used applications. such as a text processor. spreadsheet. web browser. and web tools such as e-mail.

• ICT Skills for Learning assessed are grouped into three dimensions: information. communication. and ethics and social impact.

• At national level. the percentage of test takers who attained the Initial Achievement Level of measured skills was 46.2%. while 50.5% attained the Intermediate Level and 3.3% the Advanced Level.

• The average national score totaled 249 points. The only Region with an average score significantly higher than the national mean was the Metropolitan Region. with 258 points.

• In terms of Socio-Economic Groups (SEGs). the Medium-High and High groups averaged scores above the national mean (276 and 298. respectively). whereas Medium-Low and Low groups scores fell below the national average (234 and 218. respectively).

• In the analysis of results by School Administration Type1 . average scores in Private/Subsidized schools and Private/Paid schools was higher than the national mean (257 and 298. respectively). Municipal school average scores fell below the national mean. with 227 points.

• The average total score for girls. 250 points. was marginally higher than the score for boys (247 points).

1 Schools are classified according to Administration Types:Private/Paid Schools are managed by private individuals or private legal entities. Funding is provided by parents and/or tutors. Private/Subsidized Schools are managed by private individuals or private legal entities. Funding is provided by the government and/or parents and tutors. Municipal Schools are managed by Municipalities or Municipal Corporations. Funding is provided by government.

10

SIMCE TIC National Results

11


BackgroundThe Ministry of Education. through the ENLACES Center for Education and Technology. has become the main driving force behind a public policy aimed at breaching the digital gap in school populations and to foster pedagogical use of Information and Communication Technologies (ICTs) within school contexts. Owing to various strategies in the past twenty years to promote the use of ICT in schools throughout the country. the amount of available technological equipment in schools has grown significantly. and as a result. so has access and use of digital resources in the Chilean school system.

Similarly. strategies implemented in recent years were designed to promote digital competence among teachers and school leaders. mostly favoring the use of computer tools at schools for pedagogical purposes. The main goal of these strategies is to help students develop essential skills for living in an information society that demands cognitive skills commensurate with a highly technological world.

More specifically. these skills include mastering available technological tools as well as the ability to deal with and resolve cognitively complex problems. And. in addition. to interact with others on the basis of ethical behaviors adjusted to the legal framework of technological environments.

Given this context. the need arose to create a measurement system to diagnose the development of these skills in the Chilean school system. And. moreover. to study the conditions in which skills are better developed. centering all assessment efforts on researching the functional use of ICT and its relationship with cognitive processes that favor a student’s independence and initiative in the learning process.

The assessment was carried out from 3 to 30 November 2011. It was given to a representative sample of 10.321 second-year high school students from 505 schools of all administration types throughout the country.

The results of student development levels obtained by the measured skills will enable teachers and school leaders to reflect on the integration of ICT in teaching practices. and to plan strategies to enhance ICT inclusion in the teaching-learning process.

Chapter 1 Assessment Characteristics

13


1.1. Skills and Knowledge Test Assessment

The test is designed to measure the level of ICT Skills for Learning among second-year high school students. These skills include mastering available technological tools. plus the ability to address and resolve cognitively complex problems. and to interact with others based on an ethical behavior adjusted to the legal framework of technological environments. This is expressed in a student’s ability to resolve problems and accomplish school work in a virtual environment with commonly-used computer applications.

Skills are divided into three dimensions: information. communication. and ethics and social impact. Overall. the Information dimension refers to the ability to access. understand. apply. and create information in a technological environment. The Communication dimension refers to the ability to make information known by using technology (to communicate. to transmit). The Ethics and Social Impact dimension refers to the ability to analyze interactions within a virtual context. while recognizing the possible consequences of using technology in a person’s life and/or in the life of others. and of taking action in this technological world based on ethical implications and its impact on oneself and on others2.

2 See Annex N°1for details on ICT Skills for Learning.

14


1.2. Test Characteristics

1.2.1. Test Description

The entire SIMCE TIC test is done on a computer. The software. especially designed for this purpose. simulates a virtual desktop where students have commonly-used applications such as a text processor. spreadsheet. presentations editor. web browser and tools such as e-mail. In this virtual environment. students must solve various tasks and requirements. And. a cross-sectional topic on Ecology gives continuity to the 32 items in the test.

In this environment. students are guided by three virtual classmates. These virtual classmates introduce the different tasks students must work on throughout the test. and guide the process. The test has three types of questions: multiple choice. action questions. and open questions. Although there is no time limit for completing the test. students usually take about two hours on average to answer all questions.

1.2.2. Correcting the Test

When it comes to correcting the test. there are different strategies for the three types of test questions: multiple choice questions are rated automatically using software; action and output questions are rated manually by trained rubric scorers.

1.3. SIMCE TIC Results

SIMCE TIC provides national average scores. regional scores. and scores by school administration type. The calculation for average scores is based on student test scores; but since this is samples test neither student results nor school results are reported.

In addition. results are reported by Achievement Level of skills measured by the test. Achievement Levels describe the expected skills of students in the SIMCE TIC test. and their performance is classified according to one of these categories: Advanced. Intermediate. or Initial 3.

3 For more information on the Levels of Achievement defined for the SIMCE TIC 2011 Test. see document “Ni-veles de Logro 2° Medio SIMCE TIC 2011” (Levels of Achievement SIMCE TIC 2011 2nd Year) available at http://simcetic.enlaces.cl

15


Table 1 shows score ranges per Achievement Level.

Table 1: Score Ranges per Achievement Level

Achievement Level Cutoff Scores

Advanced 336 points or moreIntermediate 246 to 335 pointsInitial 245 points or less

The SIMCE TIC test Achievement Level cutoff scores were defined using the same standardized procedure used for this same purpose in other SIMCE assessments and in other countries. This procedure involved teachers. academics. and professionals of other fields (social sciences and engineering) all experts in education IT and who helped define Achievement Levels and cutoff scores4

4 For more information on the methodology for setting cutoff scores (Bookmark method). see “Niveles de Logro 2° Medio SIMCE TIC 2011” (2011 SIMCE TIC Second-Year Achievement Levels ) visit http://simcetic.enlaces.cl

This chapter provides SIMCE TIC test results of second-year high school students.

National results are the first to be discussed. followed by regional results. Both results are based on average scores and Achievement Levels. These results also take other factors into consideration. such as the Socio-Economic Group. School Administration Type. and student Gender.

Chapeter 2 Results

17


2.1. National Results

he average national score of second-year high school test takers was 249 points.

a) Achievement Levels Figure 2.1 shows second-year high school student distribution according to the Achievement Levels defined for the 2011 SIMCE TIC test.

Figure 2.1 Second-Year High School Student National Distribution per Achievement Level

Of the total number of second-year high school test takers at a national level. 3.3% were able to attain the Advanced Level. which is to say. the desired standard for this teaching level; 50.5% of students attained the Intermediate Level and 46.2% the Initial Level.

18


2.2. Regional Results

a) Average Scores

Figure 2.2 shows the average score in each region of the country compared to the 2011 national mean. The average score for the Metropolitan region and the region of Aysen (258 and 254 points. respectively) is higher than the national mean. The only region with significantly higher performance than the national mean. however. was the Metropolitan region5.

Conversely. regions with the lowest average scores include Maule and Los Rios (232 and 233. respectively).

Figure 2.22011 Second-Year High School Average Scores per Region.

Compared to National Averages

*Statistically significant differences compared to average national score.

5 The significance estimate is based on a Difference of Means Test. Test variance estimation is based on the (Jacknife) replication method. with 95% confidence level.

243 249 240 238 247 251 241 232 246 234 233 243 254 250 258Average Score

215

220

225

230

235

240

245

250

255

260

15 1 2 3 4 5 6 7 8 9 14 10 11 12 13

19


b) Achievement LevelsTable 2.2 shows nationwide student distribution per Achievement Level per region6.

The Metropolitan and Valparaiso regions have the highest percentage of Advanced Level students (4.5% for both regions). Similarly. Initial Level student percentage in the Metropolitan region was the lowest of all (37.8%). The region of Araucania. by contrast. has the highest percentage of students in the Initial Level (61.5%).

In comparison to student percentages at national level. the Table below shows the highest percentages of Advanced Level students in the regions of Tarapaca (3.4%). Valaparaiso (4.5%). Los Lagos (3.5%). Aysen (3.6%). and in the Metropolitan region (4.5%)7.

Table 2.2 2011 Second-Year High School Student Distribution per

Achievement Level per Region

Region Initial Intermediate AdvancedArica y Parinacota 50.2% 48.6% 1.2%Tarapacá 48.0% 48.6% 3.4%Antofagasta 54.0% ↑ 45.0% ↓ 1.0% ↓Atacama 56.0% ↑ 42.0% ↓ 2.0%Coquimbo 46.6% 52.3% 1.1% ↓Valparaíso 44.3% 51.2% 4.5%O’Higgins 55.1% ↑ 43.1% ↓ 1.8% ↓Maule 59.1% ↑ 39.9% ↓ 1.1% ↓Biobío 48.8% 48.2% 3.0%Araucanía 61.5% ↑ 36.9% ↓ 1.6% ↓Los Ríos 57.5% ↑ 40.4% ↓ 2.1% ↓Los Lagos 49.7% 46.8% 3.5%Aysén 40.7% 55.8% 3.6%Magallanes 44.6% 53.0% 2.4%

Metropolitana 37.8% ↓ 57.7% ↑ 4.5% ↑

↑ : The percentage of students in this specific Level is significantly higher than the national percentage of students in this same Achievement Level.

↓ : The percentage of students in this specific Level is significantly lower than the national percentage of students in this same Achievement Level.

6 See Annex 3 for more details on sampling distribution by region (number of valid analyzed records).7 Significance estimate based on a Proportions Test (Newcombe. 1998) with 95% confidence level.

20


2.3. Results by Socio-Economic Group

The methodology used to calculate the Socio-Economic Group of students is a multivariate cluster analysis, resulting in five student clusters8.

Variables used for this classification include: the mother’s level of education, the father’s level of education, regular monthly total income at the student’s home, and the school vulnerability index. The first three variables are gathered from the Questionnaire to Parents and Tutors of SIMCE TIC sample test takers. The fourth variable, the School Vulnerability Index (SVI), is calculated annually by JUNAEB, the National Scholarship and School Aid Board.

a) Scores by Socio-Economic Group

Figure 2.3 shows average scores by Socio-Economic Group compared to the 2011 national mean.

Overall, average scores appear to be higher in upper Socio-Economic Groups.

Average scores in Low and Medium-Low Socio-Economic Groups fall below the average national score (218 and 234, respectively), whereas average scores in the Medium-High and High Groups are above the national mean.

8 See Annex 2 for cluster characteristics based on variables

21


Figure 2.3 2011 Second-Year High School Student Average Scores per SEG,

Compared to the National Average

* Statistically significant differences

b) Achievement Levels per Socio-Economic Group

Table 2.3 shows student distribution per Achievement Level and per Socio-Economic Group.

In general, what this Table shows is that the higher the Socio-Economic Group, the lower the percentage of students in the Initial Level, and the higher the percentage of students at the Advanced Level.

234218 248 276 298

0

50

100

150

200

250

300

350

Low Medium Low Medium Medium High High

**

**

Average Score

22


Table 2.3 2011 Second-Year High School Student Distribution

per Achievement Level per SEG

Socio-Economic Group Initial Intermediate Advanced

Low 73.2% ↑ 26.6% ↓ 0.3% ↓

Medium Low 59.7% ↑ 39.6% ↓ 0.7% ↓

Medium 45.0% 53.7% ↑ 1.3% ↓

Medium High 21.8% ↓ 72.2% ↑ 5.9% ↑

High 8.5% ↓ 74.6% ↑ 16.9% ↑

↑ : The percentage of students in this specific Level is significantly higher than the national percentage of students in this same Achievement Level.

↓ : The percentage of students in this specific level is significantly lower than the national percentage of students in this same Achievement Level.

23


2.4. Results by School Administration Type

a) Scores by School Administration Type

Figure 2.4 shows average scores by school administration type, and a comparison between types.

The same Figure shows highest average scores in Private/Paid schools (298 points), with the lowest average scores (227 points), below the 2011 national mean, in Municipal schools.

Figure 2.4 2011 Second-Year High School Average Scores per School Administration Type

and a Comparison of Types

* Statistically significant differences

50

100

150

200

250

300

350

227 257 298Average Score

**

*

24


b) Achievement Levels by School Administration Type

Table 2.4 shows student distribution per Achievement Level per School Administration Type9.

This same Table shows in Municipal schools the highest proportion of students in the Initial Level, whereas the greatest percentage of students in the Intermediate Level are in both Private/Subsidized and Private/Paid schools.

Table 2.4 2011 Second-Year High School Student Distribution per Achievement

Level per School Administration Type

School Administration Type Initial Intermediate Advanced

MUNICIPAL 65% ↑ 34.4% ↓ 0.6% ↓

PRIVATE/SUBSIDIZED 38.1% ↓ 58.8% ↑ 3.1%

PRIVATE/PAID 9% ↓ 72.9% ↑ 18.2% ↑

↑ : The percentage of students in this specific level is significantly higher than the national percentage of students in the same Achievement Level.

↓ : The percentage of students in this specific level is significantly lower than the national percentage of students in the same Achievement Level.

9 See Annex 3 for a more detailed description of sample distribution (number of analyzed valid records) per school administration type.

25


2.5. Results by Gender

a) Scores per Gender

Figure 2.5 shows average scores by gender. Since the results for girls are marginally higher than for boys, the difference is not considered significant.

Figure 2.5 2011 Second-Year High School Student Average Scores per Gender

b) Achievement Levels per Gender

Table 2.5 shows there are no relevant differences in terms of gender in student distribution by Achievement Level.

Table 2.5 2011 Second-Year High School Student Distribution per Achievement Levels per Gender

Gender Initial Intermediate AdvancedBOYS 47.5% 49% 3.5% ↑GIRLS 44.8% 52.3% 2.9%

↑ : Shows the percentage of boys is significantly higher than girls in the same Achievement Level.

260

230

235

240

245

250

255

247 250GIRLSBOYS

Average Score

Chapter 3 Annexes

27


Annex 1: Table of ICT Skills for Learning10

Dimension Sub-dimension Skills

Information: Ability to access, understand, apply, and create information in a technological environment.

Information as a Source: Ability to obtain, manage, order, and understand information.

• Define the information required• Search for information• Select information• Evaluate information• Organize digital information

Information as Output: Ability to create new information in a technological environment using items available on the web. Refers to a superior cognitive capacity to synthesize, develop, and create new information.

• Integrate information• Understand information• Analyze information• Represent information• Create new information

Communication: Ability to make information known using technology. In other words, aside from accessing information, individuals are expected to communicate, transmit the information in a technological environment.

Communication: Specifically refers to the ability to transmit information effectively in virtual contexts, interact within them, knowing how to properly disclose information given diffe-rent contexts.

• Know how to transmit informa-tion to others

Ethics and social impact: Assesses the ability to analyze interactions in a virtual context; realizes the possible implications of using technology for the individual’s personal life and/or for the life of others, and takes action in this technological world, taking into consideration the ethical implications and the impact on oneself and on others.

Social Impact: Assesses the realization of the positive and negative impacts or implications of Internet use and other technological tools in people’s lives (both on individuals as well as on social groups). Considers the ability to reflect on such impacts.

• Responsible use of ICTs

10 Claro. Magdalena (2009) “Propuesta marco conceptual y listado de competencias TIC siglo XXI”.

28


Annex 2:Characterization of Socio-Economic Groups

The variables in the methodology traditionally used by SIMCE to create Socio-Economic Groups include: the Father’s Education, the Mother’s Education, Family Income, and the School Vulnerability Index. However, in the model used to characterize SIMCE TIC test takers, the variables are specific to each student, whereas the SIMCE methodology condenses the information as school averages.

Table 1 shows standard averages and deviations per classification variable per socio-economic group. As can be seen, the Low group has the lowest schooling and income levels, and the highest Vulnerability Index percentage. In contrast, the High group has the highest schooling, highest income levels, and the lowest Vulnerability Index.

Table 1: Characterization of Clusters

STUDENT CLUSTER

Mother’s Education Father’s Education Household Income 2012 Vulnerability Index

Av. Std. Dev.

Av. Std. Dev.

Av. Std. Dev.

Av. Std. Dev.

Low 6.39 2.69 5.76 2.39 162.163 110.539 81% 9%

Medium-Low 8.03 1.97 10.63 2.08 246.560 173.215 74% 12%

Medium 12.48 1.38 11.12 2.46 289.572 177.682 71% 12%

Medium-High 14.52 2.00 14.89 1.96 632.233 300.347 47% 16%

High 16.27 1.62 16.72 1.66 1.766.762 294.855 9% 15%

29


Annex 3: Student Distribution by Region and School Administration Type

• At National Level

Second-Year High School Student Enrollment Actual Sample

247.257 9.462

• By region

Region Total Actual SampleArica y Parinacota 3366 332Tarapacá 4447 356Antofagasta 8740 362Atacama 4440 395Coquimbo 10530 406Valparaíso 24429 752O’Higgins 13077 385Maule 15091 518Biobío 30050 948Araucanía 15151 659Los Ríos 5987 368Los Lagos 12324 463Aysén 1298 340Magallanes 2126 326Metropolitan Region 96201 2852Total 247257 9462

• By School Administration Type

School Administration Type Total Actual Sample

1. Municipal 94491 34182. Private/Subsidized 134483 50563. Private/Paid 18283 988

NATIONAL RESULTS 2011SIMCE TICMINISTRY OF EDUCATION

ENLACES, Center for Education and Technologywww.enlaces.cl

[email protected] Alameda Libertador Bernardo O´Higgins Avenue

10th floorSantiago, 8340527

Chile.

Documents

SIMCE TIC - · PDF file5 National Results SIMCE TIC Introduction SIMCE is the National System for Learning Outcomes Assessment of the Ministry of Education of Chile. It is mainly intended