Andrzej M.J. Skulimowski AGH University of Science and Technology and

Social ApplicationsForLifelongLearning

Multicriteria Selection of Sites for the European Projects

Supporting Local Information Society

Andrzej M.J. SkulimowskiAGH University of Science and Technology and International Progress & Bussiness Foundation

[email protected]


Social Applications for Lifelong Learning, Patra, Greece, 4-5 November 2010

Introduction1. Background information on the earlier projects: 1.1. The 6th FP Rural Wings project has been the selection and installation in Poland of 10 the pilot sites of bi-directional satellite internet in DVB-RCS technology (carried out in 15 further countries)

The Progress & Business Foundation, Krakow, Poland has been the National Coordinator for Poland responsible for the above tasks.

Timeframe: January 2006 – March 2010, Selection: 2006Installation and test runs: 2007-2010 (2 Phases: Phase 1 since March, 2007; Phase 2 since March 2008)1.2. Interreg IIIB CADSES TELEACCESS – project aiming at selection and operation of innovative rural telecenters (Poland and 5 European countries)Timeframe: June 2006 – June 2008, Selection: 2007Implementation: 2007 - present



Introduction (ctd.)

2. The motivation and the main issue of this paper How to assess the quality and appriopriateness of the pilot sites’ candidates, taking into account the outputs of earlier projects?

3. The approach used to solve the problem- Compare the initial selection-stage (ex-ante) and performance-based (ex-post) pilot sites’ rankings,

- Calculate the distance between these rankings, which evaluates the quality of initial selection fit,- Assess separately the appriopriateness of criteria, the pre-selcetion of target groups, the adequacy of the outranking procedure, the quality of data gathered



The pilot sites’ selection criteriaThe main three groups of initial criteria :

•Geographical situation of the site – according to the Project’s goals, the more remote and isolated the site, the more eligible for the Project, but the diversification of geographical locations and an even distribution of sites over the target areas should be taken into consideration as well.•Existing internet infrastructure – the pilot stations should be installed in areas without broadband facilities or – at least - the internet access provided by third parties should have lower transmission rates or be less reliable than the satellite one. On the contrary, other existing IT infrastructure, such as school computer labs, LAN etc. would be of advantage for the project. •Availability of potential end-users willing and capable to engage in the activities of the pilot station.



The pilot sites’ selection procedure(i) Needs assessment in order to choose most appropriate target

groups and areas. It resulted in a pre-selectionof sites and target groups, namely: A) The geographical situation of sites was restricted to those regions in Poland, where the problems with low broadband access was related to the topography and geographical isolation, i.e. to the Beskidy mountains in Southern Poland, then mountain counties in the Małopolskie and Podkarpackie Voivodships, fulfilling initial low-broadband-access assumptions were specifiedB) The target groups were restricted to:

- rural schools and public libraries, - remote research stations and national parks, - public access points in local government offices, - local cultural centres and telecenters, - touristic establishments in the mountains.



The pilot sites’ selection procedure (ctd.)Other categories of sites considered:

- policlinics - fire brigade centers in remote rural areas, (there existed already governmental plans to endow them with broadband infrastructure)

(ii) Data gathering using the questionnaires adjusted to the specific Poland’s needs discovered in Step (i).The questionnaires were mailed to the selected target groups (schools, national parks, local authorities, research establishments, business support organizations; since March 2006). It was supported by numerous personal and phone contacts and discussions.

(iii) The data gathered were verified during field visits and otherwise.



The pilot sites’ selection procedure (ctd.)(iv) A computational method was elaborated to transform the questionnaire data into the measures of subcriteria. (v) A multicriteria outranking method based on the reference sets approach was used to establish a preliminary ranking of sites:A) For each class of potential sites, i.e.: - rural schools and public libraries, - remote research stations and national parks, - public access points in local government offices, - local cultural centres and telecenters, - touristic establishments in the mountains, there was defined an ideal (maximally desirable from the point of view of the project’s goals) site model, represented by the desirable values of criteria.B) The sites were compared to the ideal models in the corresponding classes, those with the least values of the (distance-based) regret function were consideered as potential candidates



(vi) Final arrangements and rankingThe selected sites, in the order resulting from the above outranking procedure, were visited, the Rural Wings project presented to the appropriate local authorities and potential sponsors of sites. The technical feasibility of installing the DVB-RCS equipment at the selected sites, the results of negotiations, and the final assessment of the viability and usability of site derived from the field visits was taken into account by the Selection Committee to establish the final ranking

Then it followed (vii) The installation of satellite terminals according to the finally derived priority order

The pilot sites’ selection procedure (ctd.)



Rank Name of the Pilot Site Place, Region Type Users Employees with IT

background

Internet connection

available1 Arctowski

Research Station

St. George Island, Antarctica

a) Research a) researchersb) visitors

All researchers working at the Station

NONE

2 Babiogórski National Park

Zawoja, Małopolskie (Beskidy Mountains)

a) Wildlife reservationb) Research c) Tourism

a) scientistsb) visitorsc) local community members

2 IT-specialists responsible for maintaining the hardware and LAN

Dial-up connection 64Kb/sec (DSL)

3 Polish Polar Station – Hornsund

Hornsund Fjord,Svalbard

a) Arctic Research a) scientistsb) researchersc) visitors

All researchers working at the Station

512 / 256 Kb (Sat, high disturbances)

4 Polana Primary School

Polana, Podkarpackie (Bieszczady Mountains)

Primary School a) schoolchildrenb) farmers and other local community membersc) tourists

1 IT school tea cher responsible for maintaining the hardware and LAN

NONE

5 Hucul Horses Stud

Regetów, South-Eastern Małopolska (Beskid Niski Mountains)

a) Tourismb) Research, endangered species protection

a) touristsb) scientists c) farmers

1 IT-technician responsible for maintaining the hardware and LAN

Dial-up connection 64Kb/sec (DSL)

6 Magurski National Park

Krempna , Małopolska and Podkarpacie (Beskid Niski Mountains)

a) Wildlife reservationc) Tourismd) Research

a) scientistsb) visitorsc) local community members

1 IT-engineer responsible for maintaining the hardware and LAN

Radio ADSL connec tion (frequent disturbances)

7 Gładyszów Primary School

Gładyszów South-Eastern Małopolska

Primary School a) schoolchildrenb) farmers and other local community membersc) tourists

1 IT school tea cher responsible for maintaining the hardware and LAN

Ground-based ADSL connection

Results: initial ranking of candidates to become Rural Wings pilot sites in Poland



Phase I pilot sites’ selection – until June 2006

Implementation of the initial selection ranking:The Arctowski Research Station, ranked 1, could not get the satellite connection because of the technical infeasibility of the Atlantic Bird 1 B satellite to reach the St. George Island. Heavy snowfalls, that came as early as in September, made the installation of the D-Star terminal in Hornsund, Svalbard (ranked 3) impossible. Finally sites ranked 2 and 4: Babiogórski National Forest and Polana Primary School turned out to be successful winners of the phase I selection process. The other three sites (ranked 5-7) were retained for a later implementation, but none of them was actually installedA similar procedure to the above was performed for the 2nd phase pilot sites, yielding a list of eight pilot stations.



Polish satellite internet pilot sites selected within the Rural Wings project



ID

Place, host institution

Instal lation date

Geographi-

cal situation

Description and type of the pilot site

No. of inhabita nts/user

s

Learning goals to be achieved by this site and main target groups

POL01

ZAWOJA, Babiogórski National Park

2007.03.28

longitude: 1954,

latitude: 4965,

altitude: 650 m asl

Zawoja is a village situated close to the mountain massif of Babia Góra (1725 m asl). The pilot site serves the headquarters of the Babia Góra National Park (BGPN). The site is endowed with both in- and outdoor WiFi. Site type: Research Station & Wildlife Reservation

6200 / 2000

1. Better environmental education, enriching science learning and scientific activity in remote rural areas: high school and university students, teachers, ‘green schools’ participants 2. Learning at work: researchers 3. Supporting local administration in environmental issues (e-govt)Applications: Discovery Space (D-Space) - as a supplement to the own observations of the sky at the BNP, WebTV - to upload videos with observations of the Park's wildlife

POL02

POLANA, Polana Private Salesian School

2007.03.29

longitude: 2235,

latitude: 4918,

altitude: 639 m asl

Polana is a small village in the Podkarpackie Voivod ship, south-eastern edge of Poland. It is situated in the central area of the Bieszczady Mountains. Site type: rural school

400 / 80

1. Using internet access for learning at school. 2. Using internet access for learning at home.Applications: WebTV - schoolchildren use cameras to record school events, then upload them to the web, A visit to Cret@aquarium - this application was recommended to the teachers to supplement the regular biology and English lessons

POL03

WIŚNIOWA, The Private Secondary School for Adults in Wiśniowa

2008.02.26

longitude: 2012,

latitude: 4979,

altitude: 380 m asl

The Wiśniowa Municipality is located in Małopolska, 50 km to the south from Krakow It is surrounded by the mountains. Site type: rural school

1900 /40

1. Using internet access for learning at school2. Using internet access for learning at homeApplications: UNITE platform -used during English lessons, A virtual visit to Cret@aquarium - the latter application was recommended to the teachers by the NC f, it has supplemented the regular biology, geography, and English lessons





ID Place, host institution

Instal lation date

Geographi-cal situation


No. of inhabita nts/users


POL04

MT. LUBOMIR, The Astro nomical Observatory of the Jagiellonian University

2008.02.27

longitude: 2008,

latitude: 4975,

altitude: 904 m asl

Mt. Lubomir is situated in the north-eastern part of Beskid Makowski Mountains. The site is endowed with indoor WiFi for employees, and outdoor one for visitors. Site type: remote research station

1234 /100 1. Using internet access for learning at school.2. Using internet access for learning at work.Application: Discovery Space (D-Space) - to assist in design in a similar application that will allow to transmit astronomical data from Lubomir via internet and join the network of similar observatories. Further, the visitors of the Observatory that will use D-Space - mostly high school and university students and teachers - will be able to compare the views of astronomical objects from different telescopes

POL05

ROKICINY PODHALAŃS KIE, Raba Wyżna Telecenter

2008.02.28

longitude: 1991,

latitude: 4957,

altitude: 550 m asl

Rokiciny Podhalańskie is a village in the Raba Wyżna municipality, about 70 km south from Krakow, in the Beskidy Mts. The pilot site serves the local telecenter. The site is endowed with indoor and outdoor WiFi, reaching the nearby recreational area. Site type: rural telecenter

Raba Wyżna

Municipa lity:

13632, Rokiciny Podhalań

skie: 1450 / 250

1. Intensive e-government-oriented training programs 2. The use of municipal web service of Raba Wyżna3. Using internet access for learning at school4. Using internet access for learning at home and during leisure time 5. Using internet access for learning at work Applications: WebTV - trainees and the Telecenter's staff use cameras to record relevant events from Raba Wyżna, then upload it to the web. Electronic consultation concerning relevant local community matters proposed by municipal authorities

POL06

SKAWA, The Municipal Public Library in Skawa

2008.02.29

longitude: 1990,

latitude: 4962,

altitude: 514 m asl

Skawa is a village in Nowotarski county, Małopolskie Voivodeship, in the Beskid Zachodni Mountains about 70 km south from Kraków. Site type: rural library

4000 / 300 1. Using internet access for learning at school 2. Using internet access for learning at home3. Using internet access for learning at work Applications: A virtual visit to Cret@quarium - (app li cation was recommended to supplement the lec tures on ecology and biology that have taken place at the Library for the schoolchildren and their family mem bers, municipal web service of Raba Wyżna (e-govt)




ID Place, host institution

Instal lation date

Geographical situation


No. of inhabita nts/users


POL07

NOWY LUPKOW, Primary School in Nowy Lupkow

2008.11.06

longitude: 2205,

latitude: 49,15,

altitude: 592 m asl

Nowy Łupków is situated in the Podkarpackie Voivodship, in the Bieszczady Mountains. Site type: rural school

390 /60 1. Better education: Enriching science learning and scientific activity in remote rural areas2. Rural school teacher trainingApplications: WebTV, A virtual visit to Cret@quarium. Schoolchildren (only those oldest from the 6th class) use cameras to record school events, then upload the movies to the web

POL08

KĘTY, Secondary School No.9 in Kęty

2008.11.07

longitude: 1990,

latitude: 4962,

altitude: 514 m asl

The Community of Kęty is located in the Soła river valley, at the foot of the Beskidy mountains. Site type: county-level school

19500 / 300

1. Better education: Enriching science learning and scientific activity in remote rural areas2. Rural school teacher trainingApplications: Teachers` training seminars - actively promoted by the director of the school to enhance teachers knowledge in the area of computer technology. Xplora, UNITE platform, Experinet have been used to enhance natural science lessons; after the training provided by the NC ,the teachers from the school started an internal intensive "training-on-how-to-train" program

POL09

MYCZKOWCE Primary School in Myczkowce

2009.02.06

longitude: 2224,

latitude: 4926,

altitude: 364 m asl

Myczkowce is a village in the Podkarpackie Voivodeship in south-eastern Poland. It is situated approximately 73 km south-east of the regional capital Rzeszów. Site type: rural school.

510 /120 1. Better education: Enriching science learning and scientific activity in remote rural areas,2. Rural school teacher trainingApplications: A virtual visit to Cret@quarium, WebTV, e-govt-oriented training

POL10

HARKABUZ, Primary School in Harkabuz

2009.03.18

longitude: 1950,

latitude: 4932,

altitude: 809 m asl

Harkabuz is a village in the Raba Wyżna municipality, Malopolskie Voivodship, in the Beskid Zachodni Mountains. Site type: rural school

530 / 75 1. Better education: Enriching science learning and scientific activity in remote rural areas,2. Rural school teacher trainingApplications: A virtual visit to Cret@quarium, WebTV, municipal web service of Raba Wyżna (e-govt)



Implementation of learning scenarios in pilot sites

No. Scenario description

Satellite Internet Pilot Sites in Poland

PL01Babio górski

NP

PL02Polan

aSchool

PL03Wiś

niowa School

PL04Lubom ir

Obse rvator

y

PL05RabaWyżna TC

PL06Skawa Public Librar

y

PL07Nowy Łupkó

wSchool

PL08Kęty

School

PL09Mycz

kowceSchool

PL10Harka

buzSchool

1 Entrepreneurship education: A rural e-shop run by students

N P P N P P N XP P P

2 Access to education: A virtual music school for rural students

N P XP N P XP N P P P

3 Students broadcasting local affairs through their own WebTV programme

XP P XP XP XP XP XP XP XP XP

4 Better education: Enriching science learning and scientific activity in remote rural areas

XP P XP XP XP XP XP XP XP XP

5 Rural school teacher training P XP XP XP XP XP XP XP XP XP6 On-the-field personalized

communication and training services for farmers

P P P N XP P P N P P

7 Health emergency training P P D N P D P P P P8 Addressing change and

innovation competences in rural communities

XP P P XP XP P XP XP P P

9 Individual Learning (5 scenarios for different social groups in local community)

XP XP P P XP XP P P P P

10 Improving internet access to public services and the elect ronic office (e-government)

P P P P XP P P XP P XP

Overall ex-ante expectation of the dominant users’ scenario

Rese arch

IAP T-EL Rese arch

E-govt

IAP T-EL T-EL T-EL T-EL

Explanation of symbols used : XP - eXtensive use during the pilot phase highly ProbableP - Potential or Planned use at a later dateD - to be Determined, depending on a possible expansion of the target user groupN – No use expected during the project’s durationR - ResearchIAP- Internet Access PointT-EL - On site Teaching and E-Learning access



Criteria of evaluation of the Project’s results

No. Site name, Identifier

Mean dowlink rate

(kb)

Standard deviation of

downlink (kb)

Mean uplink rate (kb)

Standard deviation of

uplink (kb)

Overall technical

assessment [0…10]

Intensity of using

project’s dedicated

applications

Deviation from ex-ante

scenario assignment

(Tab.2)

Difference between the ex-post (and ex-ante ranks

2 3 4 5 6 7 8 9 101 ZAWOJA,

POL01 1031,795 486,723 68,375 38,71485 5 lower none 0

2 POLANA, POL02 902,9459 538,3644 N/A N/A 4 lower none -1

3 WIŚNIOWA, POL03 1133,273 508,9445 44,30006 7,32848 5 lower none -14 MT. LUBOMIR,

POL04 720,5812 562,4575 347,4125 304,2815 3 lower later launch 0

5 ROKICINY PODHALAŃS KIE, POL05

1417,943 1525,783 58,66087 49,04076 5 standarde-govt

additionally developed

1

6 SKAWA, POL06 1344,20 741,87 49,37 16,88 6 lower none 0

7 NOWY LUPKOW, POL07 1552,985 350,5986 112,845 31,50235 8 standard none 1

8 KĘTY, POL08 1323,77 665,22 88,01 47,22 7 higher none 1

9 MYCZKOWCE POL09 823,5455 467,8806 58,09091 71,14697 4 lower later launch -1

10 HARKABUZ, POL10 922,07 479,05 N/A N/A 3 standard later launch 0



Criteria of evaluation of the Project’s results in Poland (ctd.)

Remarks: 1. The overall technical assessment criterion (col. 7 above) was a result of rounding and normalizing to the scale [0,…10] of the linear combination of the absolute transfer rates (positive weights) with the unreliability of the link expressed by the standard deviation of the down- and uplink rates (with negative weights). 2. The highest mean values received for downlink in the site POL05 were accompanied by highest connection risk that reduced the technical score. 3. The criteria 7 and 8 were, in fact, correlated, since the main reason for a lower-than-expected use of dedicated applications were the lower than expected transmission rates, that made the use of applications via satellite more difficult



The final ranking of sites for the SoRuraLL project in Poland

Rank Site name and code TECH LEARN IS PROG

1. KĘTY, POL087 8 3 5 23

2. ROKICINY PODHALAŃSKIE, POL05

5 7 5 5 22

3. NOWY ŁUPKÓW, POL078 3 1 5 17

4. ZAWOJA, POL01 5 5 2 3 15

5. SKAWA, POL065 3 3 3 13

5. HARKABUZ, POL104 3 3 3 13

7. WIŚNIOWA, POL035 3 1 1 10

8. MT. LUBOMIR, POL043 2 1 3 9

8. POLANA, POL02 4 3 1 1 9

8. MYCZKOWCE, POL095 2 1 1 9

Abbreviations:TECH - Technical performance assessment, scale from 0 (lowest) to 10LEARN - Intensity of implementing learning scenarios, scale [0…10]IS - Experience in Interreg, LLP, or FP IS programmes, scale [0…5]PROG- Progress achieved during the pilot phase, scale [0…5] - Total score (sum of scores from cols 3-6)



• The installation and operation of the satellite internet access points in the DVB-RCS technology does overlap with the middle-term IS policy goals, as specified in the related policy documents, and with the statutory duties of dominating telecoms,• The satellite internet can provide safe, affordable e-infrastructure with geographically unlimited access, • The successful operation of the commonly accessible and school installations depends on the availability of comprehensible and common e-government, e-learning content and e-commerce services, • The growing role of e-health-related applications, that have to be made available to everyone, without geographical limits, is to be taken into account as well,

The recommendations and conclusions



• The satellite internet access can foster an extension of common intellectual sphere in the e-space and eliminate the digital divide,• The permanent surveillance of access points in rural areas and schools is necessary to monitor negative aspects of IS development, such as personal alienation through e-gaming and –gambling, dependence on internet etc., that can endanger schoolchildren and internet newcomers. • The development of the IS in rural areas based on the satellite internet will support a move from the traditional agriculture to the development of innovative, sustainable, environmentally and animals-friendly agriculture that supplies high-quality end products . • Other branches of rural economy, e.g. (agro)tourism, can benefit from the satellite broadband access.

The recommendations and conclusions



1. The ex-post evaluation of the initial ranking, based on the results of the pilot sites selection compared to the ex-post assessment criteria may allow to derive a learning scheme, which can be useful to assess the adequacy of the selection criteria, the adequacy of choice and correctness of using of the multicriteria outranking procedure applied in the selection process, and the credibility of data supplied by the applicants.

2. To make such a comparison possible, the actually measured criteria need to be transformed to the measures of the above goals, taking into account the stochastic character of the initial expectations.

3. The evaluation process does allow to derive a subset of relevant needs assessment criteria from a larger set of criteria considered at the selection phase of the project.

Conclusions regarding the multicriteria selection method



A similar approach can be applied to assess the adequacy of the multicriteria decision-making methods that served to derive the final (ex-post) pilot sites’ ranking. The procedure can be outlined as follows:1. Calculate the distance between the initial and final rankings (it reflects the efforts that would be needed to get the results anticipated, as seen ex-post.)2. Define an aggregated value function, describing the quality of assessment and quality of performance of selected sites.3. In case of repeating decision problems of the same kind, its outcomes can be used as inputs to the above-mentioned learning scheme.

This approach can be applied to any problems involving an initial ranking, and the performance of N selected objects based on this ranking

Conclusions regarding the assessment of multicriteria selection method (2)



Appendix: Internet transfer rates



Internet transfer rates












Thank you for your attention!

Further questions and inquiries: [email protected]

Documents

Andrzej M.J. Skulimowski AGH University of Science and Technology and