SELCO Digital Education Program

SELCO Digital Education Program

Social Impact Assessment Guide

Evelyn NAMARA

JUNE, 2015

A report submitted to SELCO team to help in the social impact evaluation of the program.

Executive Summary This report analyses SELCO’s eSHALA Digital Education Program which is an intervention that gives

schools in rural areas a chance to operate multimedia programs using a solar enabled digital setup.

The report looks at key in-depth analysis of the program using a case study of two schools visited, and

highlights different stakeholder involvement and areas of improvement for various stakeholders. The

report also covers recommendations to improve metrics for measuring Impact for the Digital

Education Program.

Comprehensive Guide for eShala DEP Social Impact Assessment

Table of contents

1. Introduction 2

1.1 SELCO 2

1.2 Digital Education Program 2

1.3 Problem Statement 2

2. Social Impact Assessment Program 4

2.1 Stakeholder Analysis 4

2.2 Analysis of project context 6

2.3 Analysis of Project Objectives 9

2.4 Consultation with Stakeholders 9

2.4.1 Engaging Teachers 10

2.4.2 Engaging Students 11

3. Data analysis and key recommendations 12

3.1 On site assessments and recommendations 12

4. Creating Impact Assessment Metrics 16

4.1 An outlook on Outputs, What gets measured? 16

4.2 Measuring eShala DEP Outcomes 18

5. Conclusion 20

6. Abbreviations 21

7. Appendix: Stakeholder Questionnaires 22

1


Introduction

1.1 SELCO SELCO Foundation envisions a socially sustainable society. They seek to create avenues for asset

building, enhancement in quality of life and wealth creation that will uplift deprived sections of

society through sustainable development solutions.

SELCO foundation’s mission is to;

1. Systematically identify diverse needs of underserved communities, understand and define the

role of sustainability and energy in these communities.

2. Create and support innovative and sustainable solutions that positively impact well-being,

education and livelihoods and work towards the alleviation of poverty.

3. Foster ecosystem development in the social sector through holistic thought processes in

technology, finance, entrepreneurship and policy.

1.2 eSHALA Digital Education Program Sustainable energy practitioners have long noticed energy access affects rural and urban populations’

livelihood, generation capacity, education attendance, health outcomes and overall quality of life. In

line with SELCO’s mission, the Digital Education Program was born with a mission to give rural schools

a chance to operate independent of the grid malfunctioning.

The Digital Education Program targets schools in rural areas that demonstrate a need for quality

education but yet can not afford the cost that comes with incorporating multimedia in their education

programs with an end goal of achieving quality education.

1.3 Problem Statement

Initial feedback suggests that the goal of the eSHALA Digital Education Program is to improve quality

of classroom education at the same time helping students understand and retain concepts leveraging

on a different pedagogy that involves use of Multimedia. SELCO partnered with CLT India as the key

content providers to train and deliver multimedia content using a mini computer and a projector.

2


After one year of partnership with CLT India, an assessment to measure impact and look at areas of

improvement has not been made. No impact assessment has been done both to determine if SELCO

and CLT India are meeting the intended outcomes of the project. This report analyses the project after

one year and makes recommendations to improve metrics for assessing impact.

3


Social Impact Assessment Process

2.1 Stakeholder Analysis Since 2010, SELCO Foundation has expanded to create a holistic ecosystem to provide reliable energy

services, for the benefit of impoverished segments in society. The Foundation works to link the

benefits of sustainable energy to poverty eradication by collaborating with NGOs, local financial

institutions, education institutions and social enterprises.

Education is a big part of SELCO’s mandate and the Digital Education Program provides the

opportunity for SELCO to work with other stakeholders to bring about quality education in low income

populations using sustainable energy solutions. SELCO foundation funds 50% of the DEP

implementation which includes setting up and maintaining the solar equipment that powers the

multimedia setup, other stakeholders in the DEP project include;

● School Administration

SELCO primarily targets low income schools for the implementation of its DEP program. They

approach the school administrations of these schools and sell them the idea of involving multimedia

instruction in their schools. Each participating school has to fund 50% of the project for them to have

the installation done in their schools.

What schools get as part of the project

➔ Full multimedia installation consisting of a solar setup (panels, batteries), LED Projector and

mini computer.

➔ Training both on the technical aspects of the setup as well as training on how solar enables

their setup.

➔ Content for Maths and Science pre-loaded on the mini computer.

➔ Tablets to help teachers with their lesson plans and execution of lessons.

Expectations from Schools

➔ Ownership and overall safety of the set up

➔ Constant use of the system for subjects that are covered within the project

➔ Feedback to SELCO and CLT India about what improvements need to be done

➔ Monitoring impact in terms of children's learning outcomes and the potential that multimedia

brings to a classroom

4


● CLT India

CLT identified a problem in the education sphere within India which is that children face many

challenges while engaging in learning; the most intimidating of them all is not having a role to

participate in the learning process.

On the teachers’ front, as much as the other non-teaching duties come in the way of good

preparations, lack of subject expertise and teaching resources and over-crowded classes are few of

the reasons that limit the teachers to a delivery mode of lessons, rather than engaging children in

partnership for learning. It’s because of this background that CLT is expanding learning opportunities

and giving access to quality education for children in under-served communities.

Technology has been a fundamental enabling factor in CLT’s implementation process. To counter and

respond to shortage of elementary school teachers and resource-starved government schools, CLT

has engaged in sourcing teachers with subject expertise and digitally capturing their pedagogical

practices. CLT’s Digital educational content can be accessed via mobile devices such as Android

mini-PCs, Tablets and mobile phones.

CLT Partnership with SELCO

CLT India are the content partners for the DEP Project, since setups are done in low income

communities with no access to internet, CLT offers its offline content that is accessed through the

Android mini-PCs. They provide content for 5th - 10th grades based on NCF guidelines and customized

for Karnataka state syllabus and mapped to NCERT syllabus. For DEP project, they provide Maths,

English and Science content to schools.

What CLT offers as part of DEP project

➔ Android Mini-PCs pre-loaded with Maths & Science content

➔ Lesson plans for teachers

➔ Tablets for teachers to access their lesson plans

➔ Training for teachers on how to use the equipment and how to conduct classes using

multimedia

● Parents / Community

Having a multimedia setup within low income government schools builds long term relationship

within the community. Parents reap the benefits of having well empowered children who have access

to technology and whose learning outcomes improve with time.

Some of the benefits of using the multimedia setup is that children constantly take home science

experiments that they perform at home and parents get to see the tangible benefits of the program.

5


2.2 Analysis of project context The Digital Education Program is implemented in 120 schools across Bangalore and the outskirts.

SELCO identifies the schools according to need and location and provides 50% of the costs associated

with the setup. Each school selected is required to cover the other half of 50% to get the digital

multimedia setup in their schools.

This model allows for ownership and buy in from schools and in the long run leads to better

management and care of the equipment.

The multimedia equipment installed by SELCO involves the following;

● An LED projector

● A licensed mini computer which is android powered

● A solar setup consisting of panels and a battery pack

The schools are responsible for the overall safety of the equipment. The equipment is normally

installed in classes that are presumed to be the safest and the school allocates someone in charge of

the overall safety.

The LED projector and Mini computer

Figure 1: DEP installation in Kopal Vidya Kendra School

The mini computer is pre-loaded with content from CLT India who are the content providers /

academic partner for the Digital Education Program. The loaded mini computer is then attached to

6


the LED projector and is powered by solar. The setup also has speakers attached to the mini computer

to provide additional sound.

The installation is static which means that the equipment can not be moved around. The projector

projects on a projector screen in front of a classroom, there’s supposed to be a blackboard next to the

screen to allow teachers to explain concepts seen on the multimedia set up in depth.

Figure 2: The projector screen

The Solar setup

SELCO as an organisation has a focus on serving the unserved and underserved and one of their core

areas is energy access. They install and maintain the solar equipment at each of the partner schools.

One biggest aspect after the installation is the education bit. SELCO ensures that the teachers who use

the Digital Education equipment are trained both on the technical aspects of the set up but also the

fundamentals of solar technology.

In each corner where the solar batteries are kept, SELCO has a poster that explains how Solar works

and how the setup is powered. The teachers in turn educate the students who come for the

multimedia class about how solar works and how the equipment is powered.

This solar education creates great impact for SELCO by extending the knowledge and importance of

using sustainable energy in the minds of the students. SELCO also leaves behind contacts for

emergency situations but on a greater scale they train the teachers to be able to deal with any

troubleshooting needs they may encounter with the setup.

7


Figure 3: Poster explaining how solar works

The poster above explains how solar works in Hindi, and is a great tool for educating local

communities about the importance of using sustainable energy.

Setup concerns

Due to the fact that the setup is static, safety issues have to be looked at. How do the schools ensure

that there’s adequate safety for the equipment? Who takes ownership of safety?

Another fundamental aspect that should be noted from the set up is the need to keep the equipment

safe from dust. Since the equipment can not be un installed and kept safely, it harbours a lot of dust

that comes from the daily cleaning of the classroom. In the long run, if the equipment is constantly

exposed to dust, it may stop functioning properly.

8


2.3 Analysis of Project Objectives

The DEP project ultimately seeks to empower many low income schools with tools and resources to

enable them enhance and improve on the skills development of children who access the program. The

more schools empowered through the project, the bigger the impact in the community.

Many low income government schools are faced with outdated pedagogical methods of teaching and

as a result, students come out of such schools without any critical thinking skills, and they are not able

to attain jobs. Programs like DEP should help bridge the gap by enhancing teaching practices to help

get students curious and better at their skills due to the experiments carried out in class.

2.4 Consultation with Stakeholders

For this report, I consulted with different stakeholders to understand how the implementation of DEP

was on ground. SELCO provided enough background about their involvement and I visited two

implementing schools of DEP.

Case study: Gopal Vidya Kendra

This school is located in the village of Soladevana Halli, North 4 Block, Bangalore North District, and

has both Kannada and English mediums. The school was established in 2008 and is a private school

located in a low income settlement.

9


SELCO partnered with this school to bring the “Smart Class” to the community and empower students

with a better education and a fun way of teaching using multimedia.

Figure 5: One of SELCO’s DEP implementing schools

2.4.1 Engaging teachers I spoke with 2 teachers of Maths and Science. Both of them had high praises of how the smart class

has empowered them to be better teachers.

Below are some of the questions we discussed in our meeting;

- What their overall feelings were about incorporating multimedia in their class

- How relevant the content from CLT is compared to the syllabus

- How they prepare for their classes

- How much training they got before using the equipment

- Their views on lesson plans and how easy or difficult it is to prepare these

- Do they see any behavioral changes in the students since using smart class?

- Is there an improvement in Students’ grades since using multimedia?

Majority of the answers from teachers showed that they saw this as a positive inclusion to their

classes. They felt empowered using technology to conduct their classes and were certain this way of

teaching would bring about better results in students grades.

Positive outcomes

- Students appeared early for their classes because of the excitement of using an alternative

learning method

They see improvement in the student's participation in class

- The students are able to grasp concepts much better

10


The learning is self paced and the teacher can play the video so many times for them to

understand

- Availability of multiple examples helped reinforce learning outcomes

- “Learning by seeing” - helps students retain most of the knowledge

- They are able to go through their lesson plans and plan for classes using the tablets provided

by the project

- Content is easy to understand and it is engaging for the students

Areas of improvement

- More content needs to be added on the tablets - not the entire syllabus is loaded

- The teachers felt that more videos need to be added

- The sound used to record the videos is so low and sometimes students at the back of the class

are not able to hear

- They felt that the equipment should be open so that they can add more data or use the setup

for other educational activities

2.4.2 Engaging Students The students loved the inclusion of technology and wanted the smart class to be rolled out to all

subjects not just Maths and Science. The setup helped them understand concepts better and they

were able to play the video again if they didn’t understand the concept.

Below are some of the talking points I had with the students;

- Do they understand why the setup is important?

- How many of them understand solar - and how the equipment is powered

- How has technology enhanced their way of learning?

- What is the significant difference between using multimedia and their other “normal” classes?

- How have they improved in terms of learning?

- Are they able to carry out experiments done in class at home?

- Do they carry out experiments seen in videos by themselves?

A lot the responses to the following were positive with students saying that they enjoy the digital

classes more because they are interactive. A lot of them also pointed out that they tried out

experiments back home but some faced a challenge where they were not able to get the necessary

equipment to complete most experiments, in other words the equipment mentioned in videos is not

easily accessible in their communities.

My observations above are based on 2 schools I visited and both schools have only been using the

setup for 6 months.

A lot of improvements need to be made both on the setup point of view but also from the Content

provider's point of view. The next chapter digs in depth into the preliminary findings.

11


Data analysis and key recommendations

3.1 On site assessments and recommendations From data collected during the stakeholder engagements, a meeting with a SELCO representative

provided great insights into the eShala Digital Education program. The project is relatively new and

has been implemented for over a year in 120 schools.

● Number of trainings conducted;

The teachers are provided with one training from SELCO and one from CLT. SELCO is responsible for

setting up the equipment in schools and gives an initial demo of using the system to teachers. CLT

India who are the content providers then provide a much detailed training on the use the digital

content as part of their pedagogy to have the desired impact. Only one training is conducted at the

beginning.

There’s a need for teachers to have more additional trainings in intervals to refresh their memory.

From my conversations with teachers, there’s no clear strategy in place for training teachers who join

in later and missed the initial training. Both the school administration, SELCO and CLT India have to

work out a training program that addresses training needs for schools. Things to consider;

- Measuring the teachers retention of data through their lesson planning, execution and ability

to train new teachers

- A clear map of who handles training in house when the CLT team is not present

- How often will refresher trainings be carried out, who has to attend?

Training of teachers is key in contributing to better learning outcomes of students in a long run. The

more teachers are empowered in delivering digital content, the better for the students

● Execution of Digital Content using eShala

The setup consists of a projector, a projector screen and a blackboard. The teachers go through the

lesson plans using the tablets provided through the project. They study the content and plan on how

to execute it. The Mini-PCs attached to the projector have the class content loaded allowing teachers

to start up the projector and choose the lesson for the day.

This set up allows all students in class to watch videos and content projected on the screen. The

teacher plays videos and explains additional concepts on the blackboard.

One major concern repeated by most of the teachers interviewed is that the volume from the videos

is very low and thus most students are not able to fully grasp what is being viewed. This set up also

brings a few more questions that schools need to be keen on.

12


- All students have different learning levels, how do teachers ensure that ALL of them are on

the same page by the end of the class?

- How do teachers measure that learning is happening in real time? - Some students are shy to

put up their hands and ask for the video to be played again.

- Do teachers encourage students to ask questions to measure their knowledge retention?

- How do assessments after each lesson help teachers plan better for the next class?

Unfortunately without observing an actual classroom implementation, it is difficult to come up with

answers to these questions. SELCO and CLT India needs a mechanism in place that helps monitor the

above and make sure learning happens the way it is supposed to.

FIgure 2: Projected content from the Mini-PC attached on the projector

● Security and Safety of equipment

Safety and security of the equipment is key when looking at long term impact. If the equipment is well

taken care of, it increases its longevity and impacts many children’s lives. The schools are responsible

for the safety of the equipment and to ensure that the equipment is in good working conditions, this

involves not exposing the equipment to dust, harsh conditions and even petty thieves.

Both schools visited have a guard at the school who is responsible for guarding the school premises at

night. One school even has a CCTV camera setup to monitor the equipment. This covers a portion of

security concerns.

Computer systems in practice need to be secured in conducive atmosphere, they are not supposed to

exposed to dust and need to be serviced after a while to ensure that all inner components are well

kept and functional. The eShala DEP setup is a fixed installation that makes for cleaning of the

equipment relatively difficult. Also since the setup is fixed, it can not be uninstalled each day and kept

away, that makes the equipment exposed to dust from daily classroom cleaning.

13


One of the schools I visited is improvising by using a dark cloth to cover the equipment to protect it

from dust. While this is good effort, a better protective mechanism has to be considered by each

school. Each school needs to invest in a protective case with a lock for the setup, this not only helps

with the dust, but it also helps with security concerns should thieves break in.

Figure 6: A teacher covering the multimedia setup as a protection mechanism from dust

● Content providers analysis (CLT India)

CLT India provide the content used with the eShala DEP project. All of their content is pre-loaded on

the Mini-PCs and they are the only ones supposed to perform an update of the content. The teachers

raised some concerns regarding the content. Not all of the entire Syllabi is loaded and this means

some lessons have to be taught in a traditional way without aid of the digital setup.

More concerns from teachers show that they feel there’s need for more diagrams and animations.

The sound of most videos is low which is a big concern because then students are not able to

comprehend what they watch in videos to be able to understand better and this can in the long run

hinder their intended learning outcomes. Some of the equipment used in the video experiments in

science are not easily accessible by students and so it makes it hard for the students to re-do the

experiments back home or in the school labs.

Content is king, and how you deliver it is even more important. The role of CLT India is a very crucial

role in the success of effective implementation of the eShala DEP project. In these early stages, it is

fundamental for CLT to put in place mechanisms to gather feedback from implementing schools to

help them improve on their product.

14


There was no mechanism in place in the two schools I visited for which CLT India is using to collect

feedback from teachers. It is not even clear how often CLT visits the schools to monitor or fix issues

regarding content.

CLT needs the following in place in order to prioritise their effective implementation for better

learning outcomes;

- A clear feedback mechanism that allows teachers to report content related issues

- A timeline on providing feedback and answers to raised queries

- Plan regular visits to schools to observe teachers as they conduct classes to see if they can

pick up on some issues

- Volume issues need to be looked into as soon as possible

● Equipment physical setup

SELCO does a great job at installing the equipment. The equipment is installed at sealing level and

allowed to hang a few meters to allow teachers to operate it. (See, figure 1: for reference).

The setup design works well and allows for better security and limited disruptions.

However when it comes to science experiments, most teachers and students prefer the setup to be

more portable to allow them transfer the setup to a lab where they can easily work on the

experiments using the video references played. Because the setup is fixed, performing experiments

from another location becomes a challenge.

Should location of the equipment be well thought through before installation? The answer is yes. If

the best place to put the setup is the school lab, each school has to weigh the pros and cons of having

a setup in the lab. Better still, SELCO could come up with a different design setup where the

equipment can be transferred to the lab for experiments and then back to the smart class.

15


Creating Impact Assessment Metrics

Following the recommendations in the previous chapter, the cornerstone of CLT and SELCO’s impact

lies in improving metrics. Right now, there no measures in place to help track the necessary metrics

that lead to better project outcomes. Measuring the right impact outcome is key to the success of any

social project or program.

Let’s refer to Figure 4 below;

4.1 An outlook on Outputs, What gets measured? Outputs are the direct measurable results from SELCO’s installations. It is important to measure these

because they help map out the outcomes and then overall impact.

● Number of eShala DEP installations;

Measuring the number of installations leads to mapping the bigger impact SELCO makes in low

income schools. Each installation means more students empowered with access to digital content,

which in the long term helps with their learning outcomes.

● Number of students trained;

Each student trained using multimedia gains valuable skills that help them directly with improving

their grades and retention of data.

● Number of trained teachers;

The use of digital tools in a class puts a teacher at a higher level. Through this program, teachers get

trained in the use of multimedia and thus getting empowered to be better at their jobs. It is essential

to capture this impact as an output of the project.

16


Figure 7: - An overview of the theory of change and output indicators

Figure 8: Mapping the stakeholders affected by the project for each output indicator

17


4.2 Measuring eShala DEP Outcomes The outcomes are the specific changes in attitude, behaviour and skills obtained during the eShala

DEP project. SELCO is keen on achieving the following outcomes as part of the project.

- Improvement in skills development of students

- Improvement in grades attained in subjected taught using multimedia (retention of

knowledge)

- Improvement in the quality of classroom education

- Knowledge about sustainable energy and it’s importance

- Decreased Dropouts

For SELCO to effectively measure these, a few mechanisms have to be put in place in each

implementing school.

➔ Assessments:

In order to measure improvement in grades and retention of data obtained from the multimedia

classes, a series of assessments have to be taken to help measure each student’s level of academic

knowledge in subjects where smart class is used. It is essential to measure each child’s knowledge

level at the beginning of term (BOT), mid term (MT) and end of term (EOT). These assessments can

help in showing a progressive report of each individual student and can help in coming up with

conclusions of each student’s learning outcomes.

18


➔ Capture teachers & students testimonies:

Regularly capture testimonies of teachers and students who use the smart class. Testimonies can be

captured using video or any other digital media. Telling the story in the beneficiaries eyes is key in

communicating impact. Here are some of the questions teachers can answer;

- How do teachers feel empowered?

- What has changed for them since using digital content?

- How do they see this way of teaching improving their student’s outcomes?

The same can go for students, in addition to getting their testimonies, compare their grades with

other students from different schools who are not using smart class. How do these results vary? Can

the data obtained help come up with conclusions about the use of multimedia?

➔ Assess knowledge on sustainable energy practices:

For every student before using smart class, take their knowledge level about renewable energy and

capture findings in each student file. Revisit this file at the end of term and ask them about solar and

renewable energy after they have been exposed to smart class and the SELCO solar installation. How

much data do they now know? Can they name other sustainable energy practices?

➔ Measure soft skills:

Using smart class exposes a lot to students, the nature of instruction allows for students to be more

engaging than traditional classes. Find ways of measuring students behaviour changes in a smart class.

Key areas to look out for;

- Are students more excited about class? What are the average attendance levels compared to

other classes without smart class?

- How involved are students in class? Do they ask more questions than before?

- Are the students more curious to find out more on their own? - What projects have students

undertaken on their own?

Noticing and documenting these soft skills is fundamental in helping follow up each student's learning

curve.

Through measuring the above outcomes, it becomes relatively easy to know if the program is

achieving its intended objectives or if some things have to be tweaked to help achieve impact.

Teachers and School administration need to be trained on capturing the various data points to help

analyse learning outcomes and impact.

19


Conclusion

Measuring the right metrics is the cornerstone of every project’s success. For the DEP project,

ensuring every stakeholder knows what and how they are measuring their impact will enable the

program to impact many children in low income schools.

SELCO needs to align its goals with CLT India and develop tangible metrics that each of them can

measure on a regular basis to ensure project outcomes are met.

20


Abbreviations

SELCO - Key implementing partner of the eShala DEP project

DEP - Digital Education Program

eShala - Name of the Digital Education Program implemented by SELCO

CLT India - Official content partners of the Digital Education Program

BOT - Beginning of Term

MT - Mid Term

EOT - End of Term

21


Appendix: Stakeholder questionnaires

22


23

Documents

SELCO Digital Education Program