1
Analytical Support for Decision
Making Assignment
Group Hashtag
201266929
201273815
201269197
201285376
201273669
2
Contents Introduction ............................................................................................................................................................ 3
The Proposition .................................................................................................................................................. 3
The Problem ....................................................................................................................................................... 3
The Proposal ....................................................................................................................................................... 3
PART A DATA ANALYSIS AND RECOMMENDATIONS ............................................................................................ 4
Executive Summary ................................................................................................................................................ 4
Data Cleansing & Preparation ................................................................................................................................ 6
Background ........................................................................................................................................................ 6
Reorganising the data set ................................................................................................................................... 6
Initial work on the data set ................................................................................................................................ 6
Data Selection (including variable selection) ..................................................................................................... 7
Generating additional data ................................................................................................................................ 8
Visual Presentation ............................................................................................................................................ 8
Analysis of Data ...................................................................................................................................................... 9
Exploratory Analysis using Summary Statistics .................................................................................................. 9
Advanced Analysis Inference & Regression ................................................................................................... 13
Recommendations for Head of Department and Minister of Education ........................................................... 15
Part B .................................................................................................................................................................... 17
Value Tree ........................................................................................................................................................ 17
Scoring .............................................................................................................................................................. 18
Analysis ................................................................................................................................................................. 18
Structuring the Problem ................................................................................................................................... 18
Criteria ......................................................................................................................................................... 18
Alternatives .................................................................................................................................................. 19
Uncertainties ................................................................................................................................................ 19
Stakeholders ................................................................................................................................................ 20
External / Environmental ............................................................................................................................. 20
Weighting ......................................................................................................................................................... 21
Tier 2 ............................................................................................................................................................ 21
Tier 1 ............................................................................................................................................................ 22
Outcomes ......................................................................................................................................................... 23
Weight Sensitivity Analyses .............................................................................................................................. 25
Recommendations ....................................................................................................................................... 26
References ............................................................................................................................................................ 28
Appendix.29
3
Introduction
The Proposition
We are a group of students working for the Department of Education in Indonesia and our Line
Manager, the Head of the Department, has asked us conduct a quantitative analysis of Indonesia
with regards to its educational performance. Once we have analysed the data we are to provide a
detailed report for our Manager, who will then use this report to brief the Minister of Education.
This report will provide necessary information to the Minister in order for him to make informed
decisions on the future of education in Indonesia.
The Problem
Whilst our department provides guidance to the Minister of Education on social and economic
issues, that may affect the success and delivery of Education in the country, we have been asked to
conduct our analysis over several broad areas, utilising the data from the World Bank. These include:
Comparison of Indonesia with surrounding countries in the region
Comparison of Indonesia with developed countries
Comparison through time
The Minister has also asked us to include exploratory and inferential analysis within our report in
order for him and his department to better understand the process of data management so that he
may use this model as the basis for all future analysis going forward.
The biggest challenge that we face is trying to use and analyse the data provided by the World Bank.
Much of the data is either incomplete or irrelevant. Through a process of data cleansing we will
provide meaningful and coherent analysis for the Minister in his decision making process for
education in Indonesia.
The Proposal
The report is split into two main sections to make the findings of the report more easily understood
by the Minister and our Head of Department.
The first section of the report looks at the process of data cleansing, explaining how we converted
the data from the World Bank into a more meaningful, manageable database and how we
established the variables to be considered and our preparation process for using the data. We then
discuss how we selected, used and interpreted the visual displays of data and our use of summary
statistics in our analysis. Finally we summarise our selection and interpretation of the data using
inference, regression, normal distribution and confidence intervals.
4
We finalise this section of the report by providing some detailed recommendations to the Minister in
order to help him draw some meaningful conclusions regarding education in Indonesia.
The second part of the report discusses the Ministers options for the future of the country and his
desire to increase the number of graduates with analytical skills in Indonesia. He firmly believes that
Indonesia should have postgraduates with strong core subjects and has identified three key options
that are open to him:
Option A Send students to study abroad
Option B Bring teaching into Indonesia
Option C Develop courses within Indonesia.
The chosen outcome of these options will clearly have major political, social and economic
implications and as such, the decision must be the correct one for the Minister and his department.
For this part of the report we used a Multi Criteria Decision Analysis software tool called VISA.
Through a series of stakeholder role play scenarios, lengthy discussions and several variations of
scenario mapping, we have provided suitable information to the Minister that should help him in his
decision for the future of postgraduates in the country. By using this software tool we were able to
demonstrate the potential risks, successes and wrong decisions in all scenarios and therefore the
Minister would be suitably informed of all the variable choices open to him and his department. We
complete this section of the report with our recommendations to the Minister based on the
scenarios and variables that we have demonstrated to him and his department.
All supporting data and evidence is attached in the Appendices and has been referenced throughout
the report.
PART A DATA ANALYSIS AND RECOMMENDATIONS
Executive Summary
The report provides the details of the analysis conducted for the Ministry of Education to provide an
insight into factors associated with the enhancement of education. The analysis was conducted by
comparing the selected indicators of Indonesia against an equal number of regional and developed
countries. In order to undertake a comprehensive review, data for countries and indicators was
utilised from the World Bank (The World Bank Group, 2013).
Indonesia was compared with the selected countries on identified indicators using the most
available current data. The techniques utilised for the analysis were:
Visual Analysis Bar Charts
Visual Analysis Trend Charts
Ranking Tables using a balanced scorecard approach
Normal Distribution
Regression
5
Visual analysis of bar charts was used to analyse the position of Indonesia compared to the selected
countries, while trend charts showed the data pattern of the indicator. The indicators used for the
analysis could not completely facilitate the use of ranking tables. Some of the indicators provided
were disproportionate for comparison. However valuable information was gained on attempting to
analyse the ranking table. The use of normal distribution allowed certain assumptions to be made
regarding the current standing of Indonesia with respect to the overall population for a key
indicator. Linear regression analysis was performed to determine the level of relationship between
two educational indicators.
Indonesia was ranked top among the regional countries with respect to primary completion rate and
had caught up with the developed countries. We would suggest this is due to the government policy
of compulsory education for the first nine years. The focus on spending on education is however in
substantial decline. Spending policy does not follow a pattern in regional countries while the
developed countries have a pattern. Highly volatile trends for some indicators appear to
demonstrate a lack of long term strategy from the Indonesian government.
Indonesia's public spending on education as % of GDP is in the bottom quartile of the world
population requiring the Ministry of Education to review this figure and make strategies to improve
this. Although Indonesia has maintained a pupil teacher ratio competitive to developed countries
there is a requirement to introduce advanced training and teaching programmes into education.
The number of students attending secondary education is in decline. Qualitative research will
provide insights into the reasons behind the decline in the number of students attending secondary
education despite the fact this being mandatory for the first three years (AngloINFO, 2013).
It is critical for the analysis to be conducted on current, consistent and transparent data on
indicators directly related and supporting education. The report is limited to quantitative analysis.
Qualitative analysis on the indicators could provide an insight into the reasons for the trends
observed and complement the recommendation.
It is worth noting that data for certain indicators could be disputed. For instance, some percentage
figures were over 100%, even though based on the indicator description, this would be
mathematically impossible. As a result, it was important to constantly question the data and
conduct the analysis with a level of scepticism. Therefore, it is advised that the recommendations
made are also treated with care.
6
Data Cleansing & Preparation
Background Data cleansing is a process of reviewing data in its raw form and identifying, correcting and removing corrupt or inaccurate data. Inconsistencies and anomalies may be the result of incorrect data being reported, data not being available for certain countries or may be due to corruption in the transfer of data into the database. A significant amount of data was provided and this had to first be configured into a manageable and meaningful format, before it could be accurately grouped and analysed. Therefore, the first objective of the process was to reconfigure and present the data in a consistent and user friendly manner to allow analysis. The raw data was supplied for a number of countries in the East Asia region for a variety of different indicators.
Reorganising the data set The initial data set was in a format that was difficult to manage. To overcome this, data was sorted into filter fields to create the ability to select specific data samples which could be easily compared and measured against. The data was then sorted, which enabled the grouping of key indicators into alphabetical order. This brought together the data from each country, for each key indicator, into one block, making it easier to compare. This was further enhanced by adding an empty row between each set of indicators, which helped to visually highlight each section. Finally, the number format of the data was changed to two decimal places to make the figures more manageable. The results of this process can be seen in the Excel Appendix on the Orange worksheet (Hashtag, 2013).
Initial work on the data set When we first looked at the data set, we recognised that:
- A number of the key indicators did not seem relevant to the task - A number of countries had no data against a large number of the indicators, especially
between 1991 and 2000 - The only data provided for a developed country outside of the region was for the UK
By referring to the World Development Bank (WDB) data in Google Visualization we were able to identify additional indicators that were relevant to education. Furthermore, by using the WDB map and through trial and error of various analogies between Indonesia and several countries against different educational indicators, we identified a number of developed countries across the world, to provide a comparison point for Indonesia. This gave us 14 countries (regional and developed) and when added to our initial dataset, 37 possible indicators to work with. The new data was put into the same format as the original dataset and the years 1961-1990 removed to make both sets of data comparable. In the original data set, any seemingly unrelated indicators were removed. Finally, both sets of cleansed data were combined in a new spread sheet and separated into sections (Developed and Region). Indonesia was copied into the top cell of each section to allow for easy comparison in visual presentation. This cleansed data can be seen in the Excel Appendix on the Red worksheets.
7
Data Selection (including variable selection)
It was recognised that 37 indicators was too many to be used to provide manageable amounts of data for analysis and so this needed to be reduced. This was done by relevance of the indicator to this specific task and by the availability of data, across a number of countries. Four categories were established to help narrow down the indicators. These were:
- Spending on education - Areas of spend e.g. teachers - Measurable performance outcomes - Alternatives to public education
Through this process, the number of indicators was reduced to 17. The next step involved cross checking the available data for these indicators across each country.
There was a very low availability of consistent data for 1991-2000 and as it took time for the data to be gathered and reported, there was also minimal data from 2011-2012. The majority of the data set was most complete from 2000-2010 and so for the sake of comparison; we agreed to focus on this time frame. However, there were still indicators that had insufficient data attached to them. For example, data for levels of adult literacy would have been very useful for analysis but the data was sporadic at best across all countries and not reported at all for Indonesia. For this reason, we were unable to utilise it as a key indicator. By applying the same rationale to all indicators, we were able to further reduce the total number of key indicators to 13.
Figures 1 & 2 below summarise the final regional & developed countries chosen as well as the list of indicators which we felt were most relevant and had the best quality of data:
Whilst we wanted to compare the most current data, the gaps in the data set made this difficult. To overcome this we decided to select the most up to date, most complete data for each key indicator, from the data set. For example, when considering the indicator Primary completion rate, total (% of relevant age group) the year 2010 was selected as this column provided the most data to work with and therefore would create the most meaningful comparison for analysis.
Each key indicator was given a separate tab in the excel file and the data for all countries (categorised into regional and developed) was copied into each sheet. This created a workbook with the data organised and categorised into relevant, easy to read sections.
Regional Comparison Countries
Developed Comparison Countries
China Finland
Hong Kong, SAR Germany
Japan New Zealand
Malaysia United Kingdom
Philippines United States
Singapore
Indicator
Expenditure per student primary education % per capita GDP
Expenditure per student secondary education % per capita GDP
Expenditure per student tertiary education % per capita GDP
Public Spending on Education % GDP
Public Spending on Education % Government Expenditure
Primary Completion rate (%)
Primary Teacher numbers
Secondary Education Teacher numbers
Secondary Education Student numbers
Pupil Teacher Ratio primary education
Pupil Teacher Ratio secondary education
Progression to Secondary Education
Secondary Education Vocational Student numbers
Figure 1 Chosen comparison countries against
Indonesia
Figure 2 Chosen Educational Indicators
8
Generating additional data Using this newly created data set, a number of additional calculations were completed to support the analysis. This included finding values for the:
- Mean and median for each country (per key indicator). We ultimately decided that the mean would be the most useful and calculated this to give us an indication of the average.
- Standard deviation (to ascertain the variance of the data from the mean). - Ranking of countries. This was a useful way of ordering each country in terms of current
position (by key indicator) to compare against other countries. This was done for each group of developing and regional countries and for the sample as a whole.
This process involved a level of trial and error; with some calculations not providing meaningful results depending on the quality of data in the corresponding data set and where size of a country meant it was not easily comparable with others. The creation of graphs and charts was essential for analysing the data and helped with the identification of trends and issues.
Visual Presentation The visual presentation of data served two purposes; firstly, to aid the analysis and secondly, to allow clear communication of the findings, highlighting key points and aiding understanding of those being informed. By generating visuals of the data, it was easier to check for accuracy. For example, when viewing a series of graphs relating to expenditure per student, it became evident that one country was significantly greater than any other. This was cross checked with the data set and although the values matched, it was recognized that inaccurate reporting may have occurred. In terms of the analysis, using the interactive charts from Google Visualization was very beneficial for quickly bringing data to life, especially over a large time period. This helped us to select which areas to focus on for analysis. Through the creation of graphs with two Y-axes, it was possible to visibly compare, understand and analyse data. Utilizing tools on Excel, it was also possible to predict trends and possible convergence points in the future.
Due to the number of countries and years initially identified, the majority of the graphs generated presented too much data and so were incoherent and difficult to compare and to analyse. To overcome this, we reduced the number of countries being compared and adjusted the timeline to increase the amount of consistent data available per country.
Through the analysis of the data set, a significant number of graphs, charts and tables were generated, both as snap shots of the position in a particular year and as a trend over time. This was very helpful for analysis, but it was recognized that to include this level of information in a report would be confusing for the reader and that it may dilute the message being conveyed. Whilst all data was relevant, the most important data (positive and negative) was added to the main body of the report, with supporting information added to the Excel appendix. There were a number of other charts that were discarded altogether, despite the time invested in generating them. The lack of available data in some areas had a negative effect on some of the charts, causing confusion when looking at them. For aesthetic reasons, it was decided to remove the corresponding countries from the charts/ graphs however; these were retained in the data sets as evidence that they had not been
9
deliberately omitted to create a false picture. As a final aid to those reading the report, Indonesia was represented by the colour black in all charts/ graphs, to avoid confusion.
Analysis of Data
Once the data had been properly cleansed to the countries and indicators that we were interested
in, it was now appropriate to analyse the data in two ways:
Firstly, to compare and contrast the indicators at both a regional level (i.e. looking at how
Indonesia compared with other countries in the surrounding region) and on a global scale
(specifically how Indonesia compared with developed countries around the world). This
would involve the identification of trends between these countries and how they varied over
time. It would also involve the comparison of countries in a particular year to see how they
performed.
Secondly to use a variety of statistical tools. This consisted of inferential analysis to see if
assumptions could be made regarding Indonesias performance in relation to the total
population and specifically, trying to understand if any of the data could be applied to a
normal distribution to make assumptions regarding its performance relative to the overall
population. Finally, regression analysis was undertaken to understand if there were linear
relationships present between any two indicators for Indonesia and therefore if we could
make assumptions on what would drive improvements in any particular indicators.
Successful analysis of both of the above would enable us to make recommendations to the Head of
Department and the Minister of Education for recommended improvements.
Exploratory Analysis using Summary Statistics
The first part of the analytical process involved exploratory analysis of the data to present key
findings in relation to Indonesias performance in the key educational indicators versus regional and
developed countries.
Comparison of Regional & Developed Countries
The best way to compare Indonesias performance with other countries over time was using line
charts to view trends. In order to compare performance across countries at a particular point in
time, bar charts were created. These were ranked from high to low so that Indonesias rank within
the sample could be clearly identified.
The detailed results of this analysis for each of these indicators can be found in the Excel Workbook
with each of the relevant tabs highlighted in Green. For the purpose of this report, we selected 6
key indicators which would give the Minister of Education a balanced view of Indonesias
comparative performance by choosing 2 indicators where Indonesia appeared to perform better
10
than its peers, 3 indicators which indicated areas of concern and 1 indicator which on the surface
showed signs of positivity with an underlying concerning trend. These are summarised below.
Indicators showing positive trends for Indonesia versus peer countries
Indicator 1 Primary completion rate
The bar chart below shows Indonesia in a positive light for the indicator Primary Completion Rate.
Based on the 2009 World Bank data, it has the highest ranked value among both the surrounding
regional countries and the developed countries. This can be seen in Figure 3 below:
The trend charts in the Excel Workbook also show quite a sharp upward trend relative to these
countries and from this we can assume that there has been a strong focus to improve results in the
primary education sector in Indonesia over the past 6 years. This is an important fact to consider in
that it also determines the long term demand for a good tertiary education system and employment
opportunities for this population in the long term.
Indicator 2 Pupil teacher ratio
When looking at this indicator, it was recognised that the large majority of countries, including
Indonesia, showed a downward trend over time. In the context of this indicator, this was viewed as
a positive result, as we would assume that a lower pupil teacher ratio would imply a better quality of
education (Figure 4 below).
When comparing this within the region for primary education, Indonesia performs well amongst its
surrounding countries, with only the Philippines being a noticeable outlier to the general sample. In
comparison to the developed countries, Indonesia still has some catching up to do but has made
some significant improvements, particularly during the period 2006 to 2010. Finally, one area in
which Indonesia sees particular progress is in its pupil teacher ratio for secondary education, which
shows this as being very strong against the developed countries at an average of 12.4 over the
period 2006 to 2010.
Figure 3 Primary Completion Rate comparison
Figure 4 Pupil Teacher Ratio comparison
11
Indicators showing challenging trends for Indonesia versus peer countries
Indicator 3 Expenditure per student
The first indicator identified as being one of potential concern was Expenditure per Student as a % of
GDP per capita. For primary and secondary education, Indonesia sits in a relatively low position
compared to the surrounding regional countries (albeit in a better position than the Philippines and
surprisingly, Singapore). It also performs very poorly against the developed countries and was the
worst in this sample. Perhaps more worryingly, there was a downward trend in this indicator
between 2009 and 2010 for both primary and secondary education which was in contrast to an
upwards trend for both regional and developed countries, though it is difficult to determine whether
or not this was only a blip as the data for 2011 and 2012 was not available.
Figure 5 summarises some of these observations below for primary education, though the full suite
of charts can be found in the Excel Workbook.
Indicator 4 Public Spending as a % GDP
In a similar vein to Indicator 3, Indonesia performs in a very similar fashion between both regional
countries (i.e. relatively low position) and developed countries (i.e. worst performer) for Public
Spending as a % GDP. This is not surprising as these two indicators are very closely linked in that
they relate to public spending and the GDP of each country. Figure 6 shows this comparison below:
Indicator 5 Progression to Secondary Education
The main thing that jumped out of this indicator was the sudden downward trend between 2006
and 2007 and that at the point of the most recent data for this indicator in 2009, this situation had
not improved. In comparison to both the regional and developed countries, Indonesia appears to
fare the worst in the sample set. However, it is important that with a very small level of data across
Figure 5 Expenditure per student primary education comparisons
Figure 6 Public spending as a % GDP comparisons
12
a limited timeframe and across few countries, the above conclusions cannot be taken as concrete
evidence of Indonesia performing poorly in this indicator, although from the data given, we can
assume enough to be concerned. These trends are shown in Figure 7 below:
Indicator 6 Public Spending as a % Government Expenditure
Finally, we identified Public Spending as a % of total Government Expenditure as an interesting trend
as this did show Indonesia as the highest ranked compared to the developed countries with a decent
performance amongst the regional countries. It was however interesting to note the significant
variation in the trend for Indonesia, in particular compared to the developed countries and this
posed questions as to how consistent the spending policy on education was or if it was treated as
more of a fire-fighting exercise. This could indicate a lack of a long term strategic plan in improving
Indonesias educational system as the spending levels are very sporadic (see Figure 8 below).
Ranking Tables
We also decided to create ranking tables across each of the indicators to effectively create a
balanced scorecard of Indonesias performance ranked against the other sample countries. Each
country was ranked within each indicator and within each sample of regional and developed
countries. An overall ranking was also created across the 2 sample groups. The ranking tables
associated with this analysis can be found in the Excel Workbook with the relevant tab highlighted in
Yellow.
Figure 8 Public Spending % Government Expenditure
Figure 7 Progression to Secondary Education % students
13
After considerable debate, it was decided not to draw any firm conclusions from this analysis, as
some of the indicators were directly related to the size of the country and therefore not a fair
representation of performance. For example, China automatically ranked in 1st place for indicators
relating to numbers of teachers and pupils, simply due to the size of the country. Nevertheless, for
those indicators that demonstrated a fair comparison of performance, it was still interesting to
analyse how Indonesia compared across both sets of countries. Notably:
Indonesia ranked in 1st place across both sets of countries for Primary Completion Rate
Indonesia also ranked in 1st place for enrolment into primary and secondary private
education, however it was important to be aware of what private education meant within
the context of each countrys educational system as this could be misleading
Advanced Analysis Inference & Regression
Inference using Normal Distributions
We decided to look at a selection of indicators and determine their distribution across the
population of World Bank data. Specifically, we were looking for any instances where a Normal
Distribution was present. This would enable us to draw assumptions on the data in terms of
Indonesias performance versus the overall population.
Five of the indicators were chosen to determine whether such a distribution was present. The
relevant sheets and charts relating to this are marked in the Excel Workbook with Blue tabs. The
location of Indonesia within each chart was highlighted in red to quickly see its position within the
overall population.
Following this analysis, it was clear that only 1 of the chosen indicators could be approximated to a
normal distribution, namely Public Spending on Education as a % GDP, as can be seen in Figure 9
below:
Using this assumption to the Normal Distribution, the following conclusions could be drawn:
Mean average = 4.77; Standard Deviation = 1.86;
This gave a z score for Indonesia = (2.90-4.77)/1.86 = -1.00
Figure 9 Normal Distribution of Public Spending on Education as a % GDP
14
Using the Normal Distribution table, the corresponding proportion of the population where
public spending as a % GDP was less than that of Indonesia = 15.87%
This effectively placed Indonesia in the bottom quartile of the data, with 84.13% of the
population of countries spending more money as a % of their GDP on education
Clearly this is a powerful message for the Head of Department and Minister of Education to
understand when reviewing educational policy and future spending plans.
Future Projections based on Past Analysis using Regression Techniques
Regression is a technique of determining a relationship between two indicators. Using the graphical
pattern of the indicators and significance they had on education, we identified seven pairs of
indicators that could potentially have an influence on each other. The analysis for each of these can
be found in the Excel workbook under the White tabs. The decision to use linear regression
provided us with an opportunity to determine if the data available on the indicators had a direct
relationship with each other.
We started the analysis by plotting the data in a scatter chart to visually look for trends. Visual
analysis was followed by creating a trend line using excel function and extrapolating the correlation
coefficient. We used the correlations co-efficient comparison with 2/ and the confidence interval
of the slope, not to contain a zero as a check to ensure it there is valuable information provided by
performing the regression. We also used the R-Squared value to eliminate pairs due to the data only
being able to explain a low proportion of variability.
From the seven pairs analysed, two pairs had a relationship where the variance was above 75%.
They were:
Secondary Education Student Numbers and Secondary Education Teacher Numbers
Public Spending % GDP and Public Spending % Govt. Expenditure
Figure 10 shows a clear relation between number of students (which we used as the independent
variable) and teacher numbers (the dependent variable).
Figure 10 Linear Regression relationship between Secondary Education Student Numbers and Secondary
Education Teacher Numbers
15
Disappointingly, we could not find a clear relationship between Public Spending on Education and
Primary Completion Rate or between Expenditure per Student in Primary Education versus Primary
Completion rate. This would make it more difficult to find a link between public spending and
educational performance.
However, for those indicators where a linear relationship was found, the regression modelling
provided us with an opportunity of looking at future predictions with confidence intervals. Again,
the details of this can be found in the Excel Workbook for the regression analysis (White tabs). This
provided us with an opportunity to predict the course of the indicator and make appropriate
recommendations to the minister.
Recommendations for Head of Department and Minister of Education
Following analysis of the available data, we would recommend four priority areas for further
consideration and appropriate action:
1. Figure 6 demonstrates that Public Spending on Education as a percentage of GDP is consistently lower than many countries in the East Asia region and significantly lower than the developed countries. Furthermore, as outlined in Figure 9, only 16% of worldwide countries tracked by the WDB, report spending less on Education as a percentage of GDP than Indonesia. Over time, it is likely that this underinvestment will weaken the Education framework. To prevent this, the Department of Education must review and update the long term investment strategy for Education. This could be aided by comparing current spend plans to those of developed countries. The Department should also consult with industry to ensure that future employment needs are understood and addressed through future spend plans.
2. As demonstrated by Figure 10, a relationship exists between the number of teachers and pupils. Figure 4 highlights that the pupil teacher ratio has been reducing over the last 10 years, suggesting that class sizes are shrinking, which may increase the quality of learning. The Department of Education should consider the impact of an increasing population and an ageing teaching workforce on this ratio and ensure that a robust teacher training programme is in place, to maintain these class sizes. As new teaching methods evolve, it may be that there is a desire to decrease this ratio further. Through on-going investment in teacher training, Indonesia will be able to maintain and grow the number of qualified teachers.
3. From the Excel Workbook it can be seen that whilst there are comparably high numbers of young people completing primary education, there is a dip in the numbers going onto secondary school. As attendance at secondary school is mandatory for the first three years, it would be expected that the numbers accessing secondary would mirror that of the primary completion level. Some additional qualitative research is necessary to establish why this is the case and depending on the findings, action may need to be taken to encourage parents to send their children to secondary education in Indonesia (as opposed to going abroad for example) and to enforce the compulsory attendance of those young people who find unlawful pathways into work. Increasing the number of young people attending secondary school should have a positive impact on both the level of education in the adult population and the number of young people going onto tertiary education.
16
4. In order to accurately compare Education in Indonesia with other countries and identify trends, good practice and areas for concern, data must be captured and reported more consistently. Whilst there is some very useful data, there were a number of additional indicators which may have been of significant benefit, had the data been available. For example, data in relation to literacy rates of adults can be a good indicator of the quality of primary education. However, this data was not consistently reported. Similarly, whilst the number of young people accessing secondary education was available, there was no data to suggest how many complete and to what level. The Department of Education should give thought to other indicators that may be useful to measure the countrys progress in relation to Education, which can then be used to more accurately direct future strategic planning. Once agreed, monitoring and reporting frameworks can be developed to capture and analyse the information gathered.
17
Part B
The second major part of the exercise involved the use of V.I.S.A. to carry out a Multi Attribute Value
Analysis (MAVA) to determine what course of action the Minister of Education should choose when
deciding on strategies to improve post-graduate education. The key steps involved in this process
are outlined below.
Value Tree
Post Graduate Education was chosen as the main criteria. We then set our first and second tier
branches as the following and shown in Figure 11 below:
Project Cost
o Initial Investment
o On-going expenses
National Benefit
Students
o Repatriation rate of Graduates
o No. of Candidates
o Graduation Rate
Institution
o Tuition & Living Cost per student
o Quality of Education
o Prestige of Institution
We developed our tier 2 options from a much larger pool of inputs. By role playing as the different
stakeholders we were able cut to the core of our concerns. These inputs were then reduced to the
final number by going through a mini MCDA exercise and determining which of these were actual
concerns within the decision making process, within this context. This allowed us to create our tier 1
groups. It was interesting to note that National Benefit was, on the surface, involved in so many of
the inputs that we decided that this had to be a tier 1 input on its own.
Figure 11 V.I.S.A. Value Tree
18
Scoring
Scoring for each of the items was based on some light research for figures. For those items ranked
using High/Medium/Low we used high as the analogy for best i.e. in the case of initial investment,
Option A was scored as high as this would have been the cheapest, or best of the alternatives. For
the 5 point scoring each alternative was scored in relation to each option. The summary of this
scoring can be seen in Figure 12.
Analysis
Structuring the Problem
We used the CAUSE framework to outline and sanity check that all angles of the problem had been
explored.
Criteria
From the briefing document it was established that the criteria for this problem was to allow the
Minister of Education to evaluate the feasibility of a number of options, or the need for further
options for increasing the number of graduates with analytical skills to compliment core subjects.
Figure 12 Scoring of Criteria
19
Alternatives
The alternative outcomes analysed were the 3 suggested by the Minister and a 4th best of all worlds
option:
Option A Send Students to Study Abroad
Option B Bring Teaching into Indonesia
Option C Develop Courses within Indonesia
Option D Combination
Option D was created to capture the best elements of the first 3 options. It was envisaged that a
programme would be created in which students were initially sent overseas to study whilst the
academic and physical infrastructure for delivering high quality degrees was developed. This
development would begin by bringing in high quality academic staff that would eventually be
replaced by native staff as the programme matured.
Uncertainties
There were many uncertainties discussed from the outset, these uncertainties ranged from either
lack of particular knowledge or data such as funding sources, through to variables such as type of
courses sought:
Funding it was discussed that in all likelihood this would be a combination of government
expenditure, NGO funding (such as the ADB) and local fund raising from existing institutions
wanting to be involved.
Participants it was unknown what the scale of participation would be. Would these
programmes only be offered to the academic elite? Would a lottery type system be used?
Would such a system be available to all? It was decided that this was an uncertainty which
would be clarified better by narrowing the options and so could be discounted until after the
feasibility analysis of the current options.
Industry involvement it was obvious that Industry would be included in the discussion but
what was uncertain was what their role would be. Some of the options discussed included
industry as sponsors of students, industry as the beneficiary of higher calibre graduates, or
industry as an investor in areas such as research hosted by universities. Again it was decided
that even with Industry involvement during the discussion on options, the actual extent of
involvement would be better determined once the options had been narrowed down.
Type of education although broadly defined by the Minister as more analytical, the types
of courses being pursued would have an impact on things like length of study or capital
investment. The level of degree (i.e. Doctorate, Masters or Bachelor) was also uncertain.
20
Stakeholders
In the briefing document, the Minister suggested 4 types of stakeholder:
Government
Industry
Local Colleges
Students
On discussion it was determined that at this stage it would not be prudent to include local colleges
as their vested interests would always bias them.
It was discussed if any major stakeholders were missing from this group. During this discussion it
was concluded that Government as a stakeholder was too broad a term to encompass the divergent
priorities of different areas. This led us to define government stake holders as Government
(Education), Government (Finance), and Government (Public Relations).
Government Education
Government Finance
Government Public Relations
Industry
Students
External / Environmental
Consideration to external influences led us into a broad discussion on culture and graduate
opportunities around the world. Although by no means exhaustive, the list of external factors that
we felt covered the majority of possible sources of interruption to our decision process were:
Availability of international student places in foreign universities.
Political relationships with foreign nations.
Natural disaster diverting funding.
Competition for graduates within host nations or other nations.
Change in government through democratic or subversive means.
Culture clash leading to non-completion.
Game Changer technology leading to decrease in the use of traditional teaching institution.
As mentioned, these factors are not exhaustive but it was felt that these could offer a sufficient
threat to the stability of the decision making process that they had to be acknowledged.
21
Weighting
When discussing the weighting for each of the inputs, the group took on the roles of the
Stakeholders. The discussion and decision for each of the inputs is outlined below. In order to
increase efficiency the weightings were split into tier 1 and tier 2, with tier 2 being discussed first
and each grouped according to their parent. Each input in a family was weighted against the other
inputs within that family and not within the context of the whole tree.
Tier 2
Project Cost
o Initial Investment 82.5%
o On-going expenses 17.5%
The discussion surround this element was based around the upfront costs of the project. It was
agreed that regardless of the option chosen there would be a certain amount of on-going cost.
However this could not be discounted completely as there would need to be a balance between how
the total cost would be distributed. A figure of $100m for 10 years was suggested as the basis for
discussion. This led to discussion on whether $10m every year would return a better investment
than say $80m in the first 2 years, then $2.5m each year for the remaining 8 years.
The stakeholders had to be reminded that this discussion was to be on the weighting within the
decision rather than on the detail of any implementation of any of the options.
Students
o Repatriation rate of Graduates 7%
o No. of Candidates 61%
o Graduation Rate 32%
When examining this component the various stakeholders seemed to quickly reach agreement on
the repatriation rate very quickly. It was discussed that there was little that could be done to
directly affect this trend as international statistics are pretty robust in their stability.
A bigger disagreement came about when discussing the number of candidates. Whilst some stake
holders i.e. the student, were of the opinion that the pool of potential candidates should be a large
consideration, the counter argument was that there would always be candidates available and that if
selection criteria were added this would naturally reduce the candidate pool. Agreement was
reached between the Industry and Government stakeholders that in considering return on
investment, volume was a primary consideration.
In regard to graduation rates, it was discussed that with the widening of the criteria in order to
increase the pool size, it would be likely that the calibre of the candidates would decrease thus
effecting graduation rates. It was decided that it was important to balance the graduation rates
against the number of candidates and so was weighted accordingly.
22
Institution
o Tuition & living Cost per student 45%
o Quality of Education 48%
o Prestige of Institution 7%
In discussing the Institution components, all stakeholders agreed that the prestige of the Institutions
the graduates were coming from was of minor importance, so long as the quality of the degrees
being achieved was sufficiently high, as this would normally correlate with a more prestiges
Institute. This then led to a discussion in which the quality of the education was decided as a high
weighting factor. The Industry stakeholder argued that they needed quality graduates with good
degrees, the government argued that there needed to be a quality component in order to sell the
whole project and the student agreed that quality degrees would have more attraction to individual
students.
The discussion then naturally turned to the cost of quality degrees. As it had been decided that a
quality degree would normally be associated with a prestigious institute then there would be a cost
premium. It was agreed by all that when looking at individual programs, the cost of them would play
a large part when scaling up by numbers and needed to be kept as a major consideration.
Tier 1
Once each of the components of the tier 1 inputs was weighted, discussion turned to the relative
importance of these and the associated weighting. Involved in this discussion was again the
Government with a financial advisor and a PR advisor, Industry representatives and a student body
representative.
Project Cost 46%
National Benefit 35%
Students 2%
Institution 17%
Discussion began by all parties agreeing that the overall cost of the project was the primary
consideration as without the ability to finance, the rest of the inputs were moot.
Secondly it was argued by both Industry and the Government that there needed to be national buy
in to the program and that there needed to be tangible benefits to the country. At this point
Industry stated that its primary consideration was in getting good graduates so they did not really
care where they were taught so long as they were of high quality. Government argued that there
was a need for internal investment in any case and so each option should be considered with this in
mind.
In discussing the student input there was general consensus that candidates could be found in any
scenario and so in the overall decision making process this was a negligible consideration when
weighted against the other inputs.
The last input, Institutions, was discussed at some length. Government saw this linking in with
National benefit when a home grown option was discussed and that sending Indonesian students to
23
well know institutes around the world would raise the profile of the quality of Indonesian graduates.
The student representative also agreed that there should be a non-trivial weighting attached to this
in the decision making process.
A summary of the weightings applied to each criterion given the above can be seen in Figure 13
below:
Outcomes
The outcome of the value tree indicated that of the listed alternatives, based on the criteria
selected, their scores, and the weighting associated with them, that the most desirable outcome, by
a very small margin, was actually option B Bringing in teachers to Indonesia.
This can be seen in the score summary charts in Figure 14.
Figure 13 Weighting of Criteria
24
By viewing the score profile across all of the options we can clearly see that when each input was
taken individually option B came out on top twice, and even then it was joint top with option A in
both criteria.
If we allocated a score to the position of 4 being top and 1 being bottom we see that option B is
again the best outcome but that options C and D are tied, as can be seen in Figure 15.
Figure 15 Alternatives sense check ranking table
Figure 14 Score Summary Charts from V.I.S.A.
25
Weight Sensitivity Analyses
When working through the various criteria there were two critical inputs which showed there would
be a dramatic change in outcome if the weighting were to be changed. This first of these, Initial
Investment, had from the beginning felt like a critical factor.
We can see from Figure 16 that with the exception of option D, the three remaining options all have
remarkably steep slopes with all options intersecting at various points.
At our agreed weighting we can see that the intersect of option B and option D is extremely close,
this is reflected in the overall ranking of the options where we see option D being a very close
second place.
When taking a step back from this and looking at the tier 1 input, Project Cost, we see this aggressive
divergence again emerge, and the same rankings reflected, as shown in Figure 17.
It is obvious from both common sense and from the evidence seen that project costs are a huge
contributory factor in any decision making, and the weighting placed upon it can completely reverse
the preferred outcome.
Figure 16 Weight sensitivity analysis of Initial Investment vs Post Graduate Education
Figure 17 Weight sensitivity analysis of Project Costs vs Post Graduate Education
26
The next criterion which was deemed as critical was National Benefit. It was felt that any issue
which could impact a nation from top to bottom and would have large cost implications would be a
very decisive issue.
On running the sensitivity weighting we can clearly see that this is another criterion which,
depending on the weighting placed on it by the stakeholders, could radically change the preferred
outcomes. Again by looking at the steepness of the slopes we can infer that it would only take a
small amount of change in the weighting of the criteria to change the outcome of the decision as can
be seen in Figure 18.
With the current weighting given to this criterion we can see that we are already very close to two
cross over points which completely reverse the ranking of the 4 options.
Recommendations
Option B and option D are so close that it is recommended that a second iteration of the process is
undertaken covering these as the alternatives, with a further set of criteria developed, in order to
examine the root drivers in more depth.
Whilst option B gives a definitive course of action, Option D would require a greater amount of prior
planning. This additional time in preparing an action plan for Option D may then become the main
differentiating factor. If desire for quick implementation is a primary consideration then Option B
then becomes the clear outright best fit option.
It was interesting to note that Option B was not an option any of the stakeholders were initially
biased toward and this could either strengthen the integrity or raise question marks over the scoring
and weighting process. This would also increase the argument for a second iteration for the decision
making process between Option B and Option D, as the scoring and weighting could be reviewed
across the stakeholders again to ensure there was absolute clarity over what was important to each
stakeholder.
Figure 18 Weight sensitivity analysis of National Benefit vs Post Graduate Education
27
Another conclusion which is important to draw from this exercise is that biased opinion inherently
driven by human nature, cannot lead to a purely objective result. Although the importance of
including those biased players (as they are the stakeholders) is not in doubt, it is where their input is
applied that must be constantly considered and questioned.
This emphasises the importance of a facilitator figure to take charge of the process in order to
redress any bias which is inserted into the value tree build. This facilitator must objectify as much as
possible the reasoning behind any scoring not directly supported by independent fact and also in
allocating levels of relative importance.
In any case, the exercise acts as a valuable tool to aid the Minister in eliminating options which are
not feasible and improves the level of focus on those options that indicate a positive step forwards.
28
References
AngloINFO. (2013). Primary and Secondary Education in Indonesia. Retrieved February 14, 2013,
from AngloInfo: the global expat network: http://indonesia.angloinfo.com/family/schooling-
education/primary-secondary/
Hashtag, T. (2013). ASDM Assignment Excel Workbook. Glasgow.
The World Bank Group. (2013). World Bank Data. Retrieved January 20, 2013, from The World Bank:
http://data.worldbank.org/topic/education
29
Appendix 1 Task List
Group member
Task Owen Paterson Martin Provan Karen Taylor Wes beard
Chellappan
Thirunavukkarasu
PART A
Data Cleansing worked together as group worked together as group worked together as group
worked together as
group
worked together as
group
Identifying Gap in Available
Data worked together as group worked together as group worked together as group
worked together as
group
worked together as
group
Additional Data Collection worked together as group worked together as group worked together as group
worked together as
group
worked together as
group
Data Preparation &
Calculation worked together as group worked together as group worked together as group
worked together as
group
worked together as
group
Data Visualization worked together as group worked together as group worked together as group
worked together as
group
worked together as
group
Exploratory Analysis discussion of results discussion of results discussion of results Lead discussion of results
Inference Analysis discussion of results discussion of results discussion of results discussion of results Lead
Regression analysis discussion of results discussion of results discussion of results discussion of results Lead
Recommendations worked together as group worked together as group worked together as group
worked together as
group
worked together as
group
30
PART B
Problem structuring Facilitator role play role play role play role play
Scoring and weighting using software using software worked together as group
worked together as
group
worked together as
group
Analysis Lead worked together as group worked together as group
worked together as
group
worked together as
group
Recommendation Lead worked together as group worked together as group
worked together as
group
worked together as
group
Report Lead Proof read and comments Proof read and comments
Proof read and
comments
Proof read and
comments