Evaluation of Guidance Services in Higher Education in Finland

Internat. Jnl. for Educational and Vocational Guidance 3: 159–175, 2003.© 2003 Kluwer Academic Publishers. Printed in the Netherlands.

159

Evaluation of Guidance Services in Higher Education in Finland

SIRPA MOITUS1 & RAIMO VUORINEN2∗1Finnish Higher Education Evaluation Council, Helsinki, Finland; 2Institute for EducationalResearch, University of Jyväskylä, P.O. Box 35, 40014 Jyväskylä, Finland (∗Author forcorrespondence: E-mail: [email protected])

Received: March 2003; accepted: August 2003

Abstract. The Finnish Higher Education Evaluation Council (FINHEEC) implemented anevaluation of guidance services in higher education in 2000–2001. Altogether 25 highereducation institutions participated in the evaluation. This article describes the evaluationprocess, findings and recommendations. The aim is to show how an evaluation process canserve the development of guidance policies and practices at the institutional and nationallevels. Additionally, the article demonstrates the use of evaluation methodology as a means ofenhancing communication between staff and students.

Résumé. Evaluation des services d’orientation dans l’enseignement supérieur enFinlande. Le Conseil d’Evaluation de l’Enseignement Supérieur Finnois (CEESF) a menéune évaluation des services d’orientation dans l’enseignement supérieur en 2000–2001.Vingt-cinq institutions d’enseignement supérieur ont participé à cette évaluation. Cet articledécrit le processus d’évaluation, les découvertes et des recommandations. L’objectif est demontrer comment un processus d’évaluation peut servir le développement de politiqueset de pratiques d’orientation aux niveaux institutionnel et national. En outre, l’article faitla preuve que l’utilisation d’une méthodologie d’évaluation est un moyen d’améliorer lacommunication entre le personnel et les étudiants.

Zusammenfassung. Evaluation der Berufsberatungsdienste an Hochschulen inFinnland. Der Finnische Rat für die Evaluation der Hochschulbildung (FINHEEC) führte2000–2001 eine Evaluation der Berufsberatungsdienste an Hochschulen durch. Insgesamtnahmen 25 Bildungseinrichtungen an dem Verfahren teil. Dieser Artikel beschreibt denEvaluationsprozess, die Ergebnisse und Empfehlungen. Das Ziel des Artikels ist es zuverdeutlichen, wie ein Evaluationsprozess dazu dienen kann, die Entwicklung von Richtlinienund der Praxis auf der Ebene der Bildungseinrichtung und auf nationaler Ebene zubeeinflussen. Außerdem demonstriert der Artikel den Nutzen von Evaluationsmethoden alsWeg zur Verbesserung der Kommunikation zwischen Lehrpersonal und StudentInnen.

Resumen. Evaluación de los Servicios de Orientación en la Educación Superioren Finlandia. El Consejo Finlandés para la Evaluación de la Educación Superior (FINHEEC)puso en marcha una evaluación de los servicios de orientación en la Educación Superioren el año 2000–2001. Un total de 25 instituciones de educación superior partciparon en laevaluación. En este artículo se describe el proceso de evaluación, los resultados obtenidosy algunas recomendaciones. El objetivo es mostrar cómo un proceso de evaluación puedeservir al desarrollo de políticas y prácticas de orientación tanto a nivel institucional como

160 SIRPA MOITUS AND RAIMO VUORINEN

nacional. Además, se demuestra la utilidad de la metodología de evaluación como un mediopara mejorar la comunicación entre el personal y el alumnado.

The challenges of guidance services

In the late 1990s policy makers in the higher education sector in Finlandstarted to pay attention to prolonged study periods and the length of thetransition from school to the labour market. The complexity of choices facingyoung people in education and employment stems not only from changes injob and career patterns, but also from the growing flexibility of the pathwaysfrom education to working life. In 1998 a national expert group appointedby the Ministry of Education noted that guidance is one of the key factors forHigher Education Institutions (HEI) in promoting relevant individual learningprogrammes and monitoring the completion of studies (Ministry of Educa-tion, 1998). According to the Finnish strategies on education for 1999–2004the aim is large-scale pedagogical renewal towards student-centred teachingmethods. Teaching, guidance and advisory services will be developed tosupport progress in studies and to shorten graduation times (Ministry ofEducation, 1999).

In order to support the implementation of these strategies, the NationalBoard of Education and the Finnish Higher Education Evaluation Council(FINHEEC) launched parallel processes for evaluating guidance provisionnationally from basic education to higher education in 2000–2002. Thisarticle focuses on the evaluation of guidance services in higher education in2000–2001 implemented by FINHEEC.

Evaluating guidance services

Guidance provision is a challenging subject for evaluation because of itsdiversity. First, the problem relates especially to the definition of guidanceand the difficulty of measuring its input and outcomes, since guidance is oftenembedded in the educational process. Because of different approaches in thedefinition of guidance, it is not possible to compare the results of differentstudies (Maguire & Killeen, 2003). Second, the factors that constitute thequality of guidance have been defined in various ways. Third, there maybe several reasons for carrying out an evaluation process (Plant, 2002; Watt,1998):

– political reasons: to justify the service,– funding purposes: to show that the service is worthwhile,

EVALUATION OF GUIDANCE SERVICES IN HIGHER EDUCATION IN FINLAND 161

– measure client progress: to assess implementation of planned objectives,– record what is happening: monitoring,– strategic planning: organisational development,– practice and policy development: assessing good practice; bench-

marking.One or all of these factors may be embedded in specific quality assur-

ance procedures. The quality assurance standards or guidelines have oftenbeen drawn up by the funding bodies, guidance managers and agencies,or by professional associations (QAA, 2003; Killeen, 1996; Hawthorn,1995; HEQC, 1995). Recently also the idea of client-oriented standards hasemerged.

Thus, the quality assurance frameworks in different countries may includethe following aspects to be measured:• Features of the provision about which information should be available:

– the aims of the service,– the content,– the list of audiences,– accessibility,– nature of policy statements,– the way in which quality is assured and developed,– referrals,– follow-up.

• Standards for inputs, processes and outputs:– materials,– staff qualifications,– activities.

An evaluation of institutional guidance services is a broad activity, whichmay concentrate on process or outcome evaluation (Killeen, 1996). Theoutcome evaluation covers number of difficulties to be considered: multiplegoals, unique clients, multiple methodologies, client autonomy and externaldetermination of outcomes. Being aware of these difficulties, the evaluationproject described in this article adopted methods related to qualitative processevaluation with the attempt of fostering a worthwhile change in guidancepractice and policies in HEIs. Rather than top-down judgemental processthe evaluation was regarded as a form of negotiation between the stake-holders demonstrating the approach of fourth-generation evaluation (Guba& Lincoln, 1989).


Evaluating guidance services in higher education in Finland

In Finland, careers information, guidance and counselling services areprovided mainly by two public service systems: student counselling offeredby the public education system, and the information, guidance and coun-selling services provided by the public labour administration. The goalof these services is to help individuals to make choices concerning theireducation, training and career planning from the perspective of lifelonglearning.

The HEIs provide guidance services for the students and applicants.The organisation of these services varies from one institution to another.Usually, the student affairs office offers information, advice and counsellingon studies, practical training, and Open University courses. A special finan-cial aid office deals with student welfare services. Career centres offer careercounselling, job search training, and information on vacancies and employersfor students close to completing their studies. Career services are also avail-able from joint WWW-pages provided by career services centres. Facultiesand departments usually offer the services of a student affairs secretary whois responsible for planning, co-ordinating and developing tutoring, trainingtutors and publishing study guides. Teachers, the student affairs office, thestudent unions and the Health Service professionals also play roles in theprovision and delivery of guidance and counselling in HEIs (CIMO, 2002).

The Finnish Higher Education Evaluation Council (see: http://www.kka.fi) is responsible for evaluating the quality of higher education.FINHEEC is an independent expert body established in 1996 at the initiativeof the Finnish Government. Their main task is to assist higher education insti-tutions and the Ministry of Education in evaluating and supporting institutionsin developing the quality of their activities. FINHEEC’s activities cover thewhole of the higher education system, which consists of twenty universitiesand thirty polytechnics.

This council organises evaluations of quality assurance and enhance-ment initiatives and institutional, programme and thematic evaluations. Sincethe beginning of its activities, 72 evaluation reports have been produced.FINHEEC focuses on the quality assessment. Furthermore, it providesadvisory and consultancy services in the implementation of the evaluations,develops evaluation methodology and disseminates good practices. For themost part, the evaluations are based on qualitative data, such as qualitativeself-evaluation reports produced by the HEIs. Quantitative data are usuallyused as a background for evaluations. Instead, the Ministry of Education usesquantitative data systematically in the annual target and budget negotiationswith the HEIs.


Methodology

FINHEEC launched an evaluation of guidance services in higher educationat the initiative of the National Union of Finnish Polytechnic Students. Theproject took about one year, starting with planning in April 2000 and endingwith the publication of the evaluation report in September 2001.

Framework for the evaluation

FINHEEC invited a steering group to define the aims, method and schedulefor the evaluation. In accordance with the FINHEEC’s overall mission themain goal of the evaluation was to support the universities and polytechnicsin developing their guidance systems. The specific aims were:

– to find out how guidance is implemented from both the student’s and theinstitution’s point of view,

– to appraise the matching of guidance strategies and practices,– to identify the critical points in guidance with regard to progress in

studies and learning,– to produce information about good practices and, on this basis, to

recommend measures for further development.Due to the fact that the guidance provision in HEIs varies considerably, the

steering group decided to pay attention to two viewpoints in particular: thestudent pathway (individual learning programme) and the guidance servicesin their entirety.

The student pathway model (Reisenberger, 1994; Lehtinen & Jokinen,1995; Tarkiainen & Vuorinen, 1998) was chosen since it covers all the phasesof the individual learning programme: first guidance before the studies begin;second guidance at the beginning of the studies; thirdly guidance during thestudies; fourth guidance at the end of the studies; and finally feedback andfollow-up systems relating to guidance. The entity of guidance in this contextmeans the entity of services offered to students during these phases within theinstitution and as a result of cooperation with guidance professionals outsideit. This entity had not been evaluated in Finnish higher education institutionsbefore.

In order to keep the project manageable, the steering group limited theproject to guidance relating to degree studies especially from the faculty’spoint of view. Thus, guidance relating to postgraduate studies, compulsorylanguage studies or Open University studies was excluded. Furthermore,other organisations providing student services, such as computer, library andhealth care services, careers services, placement and international exchangeswere not evaluated as such although relevant guidance was touched upon.The exclusion of careers services was partly due to the fact that the Ministry


of Education had recently conducted a national evaluation of these centres(Sakari, 2000). The results of this evaluation were available for the evalu-ation team. However, this exclusion appeared to be a limitation to the overallevaluation process.

Participants

As the aim of the project was highly developmental, the steering groupdecided to organise the evaluation on a voluntary basis. FINHEEC asked thehigher education institutions to register for the project and 15 polytechnicsand 10 universities volunteered. The HEIs could decide themselves whethera participant would represent a whole institution or part of it. Most poly-technics participated as whole institutions, while in the university sector theparticipants comprised nine faculties, nine departments and four subjects ordegree programmes.

The number of polytechnics that volunteered represents 50% of all poly-technics in Finland, so the sample can be considered representative. Likewisethe number of universities that volunteered represents 50% of all universities,but since the participants represent a random sample of fields taught at univer-sities, the results can be interpreted as an example of the current state ofguidance services in the university sector in different fields.

Apparatus

The material collected for the evaluation consisted of three elements:− self-evaluation reports produced by the staff and students,− written statements on each other’s self-evaluation reports produced by

the staff and students,− discussions during the site visits conducted by an external evaluation

team.The structure of the self-evaluation reports consisted of the following titles

(in parenthesis some examples of the subtitles):– entry stage guidance (e.g. aims, methods and liable persons),– student’s integration and orientation towards studies, institution and

study field,– study process (e.g. support of study skills and personal development,

monitoring student progress, giving feedback from student learning),– constructing a study plan (e.g. planning of the relevant subject combina-

tion, possibilities for individual learning programme, feedback collectedfrom alumni and potential employers),

– thesis/end-of-programme project,– transition to the labour market,


– planning and organisation of guidance services,– other matters relevant to guidance in the institution,– background information about the institution and self-evaluation

process.The guidelines for the self-evaluation required staff and students to

describe and evaluate current practices and possible development targets inguidance. In the guidelines FINHEEC emphasised that self-evaluation is acollective process representing different views and groups. The staff andstudent bodies could determine the way in which they organised the self-evaluation process. However, FINHEEC asked both the institutions and theirstudents to nominate a contact person to coordinate the self-evaluation. Themaximum length of a self-evaluation report was 20 pages.

After completing the self-evaluations, the staff and students were askedto exchange them and give their written opinion on each other’s reports atFINHEEC’s request. The staff and students were asked to list strengths anddevelopment objects based on the self-evaluation reports, to give their opin-ions as to possible reasons for the disparities in the staff’s and the students’reports, and to comment on the proposed development measures.

Before the site visits, the institutions were given instructions on the aimsand practical arrangements of the visit. The site visit programmes followeda similar scheme, which included presentations of guidance facilities andmaterial and other student services available at the institution/unit and athree-hour discussion for the staff and students

Procedure

The evaluation procedure consisted of five phases.

Pilot phase. Before starting the actual evaluation, the method was tested inthree pilots. The pilot phase showed no need for adjustment in the evaluationprocedure.

Training phase. Two seminars were organised to facilitate staff and studentcontacts with the method. The aim of the first seminar was to discuss the aimsand implementation of the project with the institutional participants. Nearly80 staff and student representatives participated in the seminar. The secondseminar organised by the central student unions was especially targeted tostudent contacts. This seminar gave an opportunity to discuss the evaluationparticularly from the point of view of the student.

The self-evaluation phase. The HEI staff and students drew up their ownseparate 20 page self-evaluation reports concerning guidance in their insti-


tution. While implementing the self-evaluation process, most HEIs andorganisations set up a project group, which collected the material by means ofinterviews, questionnaires, small group discussions, institutional discussionsor e-mail and compiled the self-evaluation report.

The local student organisations played a very crucial role in organisingthe student self-evaluation processes. Altogether nearly 800 staff and over1,200 students participated in the self-evaluation processes. Staff participantsrepresent 4.7% of the total number of staff in polytechnics and universities(N = 17,200) and student participants represent 0.5% of all higher educationstudents (N = 263,000).

In addition to the self-evaluation reports, the project included an addi-tional phase developed especially to support the communication betweenstaff and students. The staff and students exchanged their self-evaluationsand commented on each other’s report.

Site-visit phase. FINHEEC invited an external evaluation team to conduct sitevisits in selected HEIs and organise a discussion forum for staff and students.The evaluation team consisted of five members representing higher educationinstitutions, students and experts on the practical implementation of guidanceand on research relating to guidance.

On the basis of the self-evaluations, the external evaluation team choseseven HEIs for site visits. They were chosen to represent as widely as possiblethe participating units with regard to the stage of development of guidanceservices and background (size, study fields, geographical situation, language).Together with visits to the three pilot institutions, there were ten site visits tofive universities and five polytechnics.

The themes discussed during the site visits were chosen by the evaluationgroup case by case on the basis of the self-evaluation reports. The aim was toraise five themes in which the opinions of the staff and students diverged orwhich related to the quality of guidance in the unit concerned.

Discussions on a site visit included, for example, the followingthemes:

– How to bring the staff’s and students’ views closer together concerningthe role, construction and monitoring of individual study plans?

– What kind of personal, group and networked guidance do students needat different stages of studies?

– How could the feedback from students be used to better developteaching, study modules and guidance?

– Which mechanisms could be used to monitor the progress of studentsand to prevent dropout?


– How to make the guidance system better known to the parties concernedand how to develop a common view of guidance?

For scheduling and financial reasons, the evaluation group could not visitall the participating institutions. The team hoped that the 14 institutions,which were not visited, would arrange corresponding discussions and thusgain the best possible benefit from the self-evaluation reports. FINHEECissued instructions to the HEIs concerning the arrangements. Many HEIsarranged such an event on a voluntary basis and reported to FINHEEC.

Reporting phase. The evaluation team wrote an evaluation report based onthe material created in the self-evaluation process.

Results

Framework of the analysis

The evaluation team compiled the results of the evaluation into a reportpublished in September 2001 (Moitus et al., 2001). The report containsconclusions and recommendations concerning all the participating institu-tions and specific feedback to the ten institutions/units visited.

Due to the qualitative approach in the evaluation, triangulation (Patton,2002) was used as a main method in clarifying the institutional reports. Thecomposition of the evaluation team provided a platform for investigator trian-gulation. The members of the evaluation team were using multiple perspect-ives while interpreting the different sets of data. During the analysis processthe potential inconsistencies in findings were discussed in the team meet-ings and these discussions highlighted new parallels between the evaluationmethodology development and the phenomenon under study.

Findings

One aim in the evaluation project was to find out how HEI staff and studentsunderstand the concept of guidance. The evaluation showed that guidance isgenerally defined in terms of operations and organisation. It is seen to includestudent services (opintotoimistojen opiskelijapalvelut), careers services andplacement (ura- ja rekrytointipalvelut), international affairs (kansainväl-iset asiat), faculty and departmental guidance (tiedekuntien ja laitostenohjauspalvelut), and student health care services (opiskelijaterveydenhuolto).Another point of view, expressed by many participating universities, was tosee guidance as overprotecting, in that excessive guidance easily becomesnannying, which is not appropriate for higher education.


The evaluation showed that the aims of guidance are two-fold: institution-centred and student-centred. The institutional aims usually relate to quan-titative degree and other targets and therefore stress efficient and shorterstudy times and prevention of dropout. Student-centred aims in turn relate toempowering students in goal-oriented studies and their professional growthin their own fields.

The guidance resources were evaluated in terms of human and physicalresources. It turned out that the institutions had a fairly large number ofguidance and advisory personnel. The full-time guidance personnel includestudy secretaries (opintosihteeri), study counsellors (opinto-ohjaaja) andcareers counsellors (uraohjaaja). Guidance is also given part-time by theteaching staff (opetushenkilöstö), international affairs staff (kansainvälistenasioiden henkilöstö), careers services (ura- ja rekrytointipalvelut) and studentfinancial aid boards (opintotukilautakunnat), student health care psycholo-gists (opiskelijaterveydenhuollon psykologit), peer tutors (opiskelijatuutorit),alumni and mentors already active in working life, and chaplains (opiskeli-japapit).

The evaluation group noted that on the whole the physical guidance facil-ities were excellent in the institutions visited. In this context, the team alsoassessed guidance materials published in the form of study guides, on noticeboards, on the intranet and Internet, etc. Although there was a great deal ofmaterial available, students do not read it sufficiently, according to the staff.

The evaluation showed that guidance has been organised in different waysin different units. Some HEIs had centralised guidance and advisory units.Others were providing guidance by separately operating units. Some hadintegrated guidance into teaching situations or relied on teacher tutoring.

Whatever the organisation model, the problem for most HEIs was thatguidance had been developed step by step in a reactive mode. This had ledto a situation in which a large number of personnel work with guidance buttheir responsibilities have not been clearly defined. This means that from thestudent’s perspective the guidance and related services, instead of formingone entity, are a combination of disparate operations. As a result, studentsand other users of guidance services have difficulties in understanding thenature of the service system and knowing whom to turn to in their particularproblem.

According to students in particular, there is a great deal of informal guid-ance as well. In this context informal guidance means discussions betweensstudents or discussions with teachers in informal situations. The problemwith informal guidance was seen to be that the information given might beinadequate and not equally available to all students.


The evaluation team noted that HEIs mostly use four methods: individualguidance, group guidance, peer guidance and Internet-based guidance. Theevaluation showed that individual guidance is currently the prevailing form.It is available in questions relating to study planning, careers planning, thesiswork, language or work practice, international exchanges and the use ofsupportive services.

Small groups were used in guidance relating to thesis/end-of-programmeproject and in peer tutoring. Although small group guidance is often moreeconomical and more productive than person-to-person guidance, its use wasrelatively modest.

Student support was mostly given in the form of peer tutoring. Universitiesin particular have long traditions in peer tutoring, which in many cases wasconsidered the most efficient form of guidance.

The Internet was mainly used in guidance to distribute information. MostHEIs publish their study guides on the Internet, and students can commu-nicate with the teachers via e-mail. On the other hand, there were no tools tohelp students plan their studies, and the information provided on the net wasoften disorganised.

In terms of the quality and quantity of guidance the evaluation showedthat a great deal of guidance was offered at the early stage of studies, mainlyduring the first days and weeks of the first autumn term, when the studentservices and faculties arranged orientation weeks for new students. Studenttutors play an important role at the entry phase. According to staff andstudents, information was provided in such large quantities at the entry stagethat students had difficulties assimilating it.

The midpoint in studies, which usually means the second or third year,turned out to be a grey area in terms of guidance. After the hectic entryphase, guidance is not sufficiently available during the mid phase, accordingto students. However, according to the students’ self-evaluation reports, guid-ance during the writing of theses was sufficient as regards both quality andquantity.

One topic, which was especially highlighted in the evaluation, wasguidance relating to the structure of the degree and the design of the indi-vidual learning programme. The individual learning programme is a fairlynew phenomenon in universities, whereas polytechnics have at least someexperience of its use.

The assessment of feedback systems revealed that all the HEIs collectfeedback on teaching. According to students, there are shortcomings in theuse of this information, and the feedback is rarely discussed. Students alsohoped for more feedback on their work than they currently get.


Recommendations

For most of the participating HEIs, the evaluation project was the first timethey had analysed their guidance as a whole. According to the evaluationteam, the self-evaluation reports written by the staff and students at the partic-ipating HEIs offer good materials for further development of guidance. Inorder to support this development, the evaluation team compiled 54 nationalrecommendations and 37 examples of good practices into the evaluationreport. The recommendations can be summarized under eight main headings.

The concept of guidance must be clarified. The evaluation team recommendedthat the different viewpoints regarding guidance must be discussed nationally,in each educational institution and at the faculty/programme level. Guidancecan be divided into educational guidance, vocational guidance, and personalguidance (Watts & Van Esbroeck, 1998). According to the team, the focusin guidance should be students’ needs. The team also stressed the responsi-bility of both parties: the student is responsible for progress in his/her studiesand the university/polytechnic must provide sufficient guidance to help thestudent in constructing a relevant programme which can be completed in theset time.

An institutional plan must be devised for guidance services. According tothe evaluation team, lack of a common guidance strategy may result in thequantity and quality of guidance varying greatly between faculties, depart-ments or degree programmes. The team recommended that HEIs draw up astrategic plan for guidance, defining the aims and content of guidance andother advisory services as well as determining the structure of the institu-tional guidance service system. One crucial element in the guidance plan is todetermine the minimum level of guidance guaranteed to every degree student.

The division of work in guidance must be made more functional. The evalu-ation team recommended that the division of work between those providingguidance either part- or full-time should be clarified to give each personinvolved a clear idea of their own role in the system. The team paid specialattention to the in-service and further training of those responsible forguidance.

The guidance service system must be developed as a whole. The team recom-mended a holistic approach to the development of the guidance system (Watts& Van Esbroeck, 1998). Every unit in the service chain should be able to referstudents to relevant services according to the individual needs. The evaluationteam felt that HEIs should discuss what kind of guidance they expect the


student to get through informal channels and whether some forms of informalguidance, such as student tutoring, should be included in the official guidancesystem.

The methods in guidance must be diversified. According to the evaluationteam, the institutions should invest more in small group guidance. On theother hand, the evaluation team considered that more attention should be paidto defining peer tutors’ tasks and to their training. Peer tutoring should alsobe coordinated more efficiently with other modes of guidance, especially withacademic tutoring.

Guidance must cover all the phases of the student pathway. As for the guid-ance in different phases of the student pathway, the evaluation team madefive sub-recommendations: first the entry stage guidance should be stretchedout so that students get the kind of information and guidance they need andcan assimilate over time; secondly the institutions should arrange guidanceduring the mid phase especially with rspect to the choice of major and minorsubjects and careers planning; thirdly attention must be paid to the definitionof the research/project problem and to the coordination of guidance whenthe thesis supervisor changes in the middle of the writing process; fourth thepreparation of the individual learning programme and the monitoring of itsimplementation provide a good means of promoting goal-oriented studies;and finally, elements relating to working life should be linked with studiesfrom the outset (see: Watts & Van Esbroeck, 1998).

Guidance must be systematically evaluated and developed. The evaluationteam recommended that the HEIs create a structure for systematic feedbackon teaching and learning, as well as procedures for the utilisation of this infor-mation. Special attention should be paid to developing feedback on learning,but it is also important to develop students’ capabilities for self-assessment.

Guidance must be included in the topics discussed in target outcome negoti-ations between the Ministry of Education and the HEIs. The evaluation teamalso put forward proposals for the Ministry of Education. The main proposalwas that the Ministry should take guidance as one of the topics discussedin the target outcome negotiations it conducts with HEIs. In addition, theteam recommended that sufficient resources should be made available forthe training of guidance practitioners, for the development of basic guidanceskills for faculty members, and for research related to guidance.


Follow-up of the evaluation

One year from the publication of the results, in summer 2002, FINHEECcollected views on the evaluation by sending an e-mail enquiry to thecontact persons in the 25 institutions that participated in the evaluation.They were asked to assess how the evaluation had benefited their institution,to describe the measures taken following the evaluation, and to give theiropinion concerning the strengths and weaknesses of the project. Altogether11 institutions replied the enquiry.

On the whole, the participating organizations considered the evaluationvery useful. According to their comments, the evaluation helped to under-stand the whole student guidance process and it revealed the deficienciesin student guidance. The participants particularly liked the method, sinceit contributed to co-operation between staff and students. The evaluationincreased the staff’s willingness to co-operate with students and the use ofstudent unions as partners in oncoming projects. As a by-product, the evalu-ation increased the knowledge of evaluation and self-evaluation. Accordingto HEIs, the project also served other concurrent development projects.

The evaluation project, however, was criticised for being laborious. Someconsidered the self-evaluation process very arduous. Despite the preparatoryseminars, one department would have liked to have more instructions for theself-evaluation.

The institution-specific recommendations were generally seen to be usefuland relevant. As examples of concrete measures, several HEIs had drawnup or intended to draw up a strategic guidance plan. A fair number of HEIshave increased or developed their information on the Internet. Some HEIssaid they would make a greater effort to help students plan their individuallearning programmes. Many HEIs had increased the training provided foracademic and peer tutors. Two institutions had established new posts in guid-ance. According to some HEIs, the leadership now appreciates guidance moreand also puts more emphasis on it.

However, many HEIs thought that implementing all the recommendationswould be a demanding task. It would seem that following the evaluation someHEIs have prioritised those recommended measures that best suit their ownoperational culture or are considered the most urgent.

The national recommendations were also considered positive and gener-ally worth implementing. Most significantly on the national level, theMinistry of Education invited all universities to draw up developmentstrategies for basic degrees and studies in which guidance plays an importantpart. In the target outcome negotiations conducted in spring 2002, guid-ance was one of the themes discussed with all the universities. Accordingto the target outcome contracts in 2003, universities will be allocated special


project funding for the development of guidance services. Guidance has alsobeen discussed in corresponding negotiations with polytechnics. Similarly,a Ministry of Education committee preparing the two-cycle degree systemproposed by the European Union in the Bologna process recommended thatall the HEIs use the individual study plan system by the year 2006.

Since the evaluation, guidance has been discussed at the national levelas well. Two national seminars were held in November 2002: one relatingto the development project for promoting individual learning programmes inpolytechnics (Kokko & Kolehmainen, 2002) and another on Finnish nationalpolicy on guidance and counselling (Vuorinen & Kasurinen, 2002).

Additionally, FINHEEC favours a procedure in which the follow-up eval-uation is organised about three years from the actual evaluation. This is seento produce the most comprehensive information about the long-term impactof the project.

Conclusions

The answers received from HEIs and the processes launched by the Ministryof Education showed that the evaluation project managed to support thedevelopment of guidance services in HEIs. The project revealed a cluster ofquestions, which had implications for the development of national guidancepolicies and practice in higher education in Finland. It seemed also to bea springboard for further and more detailed pilot projects focusing on thequality assurance framework of guidance services in HEIs.

However, the chosen method had some limitations compared with systemwide evaluations such as the OECD activity 2000–2003 (Sweet, 2001) onguidance policies, which aim to demonstrate the effectiveness of the systemand provide an evidence base for policy makers. For example, there were noexplicit quantitative data available within the HEIs to analyse the cost andbenefits of the guidance services. Furthermore, it was not possible to clearlydifferentiate the effect of guidance from that of other contributory factorsor to recognise the appropriate balance between different delivery modes orroles of the different parties and stakeholders in the overall service provision.Additionally, wider linkages between employment and welfare policies couldnot be identified.

In addition, the evaluation project gives substance to some further consid-erations. First, when the national bodies are taking concrete measures inimplementing the lifelong learning strategies, guidance in HEIs should bedescribed as an integral element of the lifelong guidance process both frompolicy and practice perspectives. The HEIs should also ensure, that guidanceinterests are represented in institutional decision-making forums. In addition


to the definitions of the basic concepts for guidance, the policy makers mustidentify what the sites and settings are for guidance in HEIs and what thefunctions and tasks are it has to fulfil. In developing evaluation methodologyand the quality criteria in guidance, the role of the student bodies should beemphasized. There is also a need to create more common understanding andlanguage on the quantitative and qualitative outcomes of guidance.

Furthermore, the results of the evaluation should have implications inthe training of guidance practitioners. In addition to guidance methodologycompetencies and qualifications of guidance practitioners, the training shouldfocus more on the evaluation and strategic planning of the guidance servicesin different settings. In the future, quality guidance seems to be an indicatorof overall quality in higher education.

References

Centre for International Mobility (CIMO) (2002). Educational and vocational guidance inFinland. Helsinki: author.

Guba, E. G., & Lincoln, Y. S. (1989). Fourth generation evaluation. London: Sage.Harris, M. (2001). Developing modern higher education career services. Nottingham: DfEE

Publications.Hawthorn, R. (1995). Firts steps: A quality standards framework for guidance across all

sectors. London: Royal Society of Arts/National Advisory Council for Careers andEducational Guidance.

Higher Education Quality Council (HEQC) (1995). A quality assurance framework forguidance and learner support in higher education; the guidelines. London: Author.

Killeen, J. (1996). Evaluation. In A.G. Watts, B. Law, J. Killeen, J. M. Kidd, & R. Hawthorn(Eds.), Rethinking careers education and guidance theory, policy and practice (pp. 331–348). London: Routledge.

Kokko, P., & Kolehmainen, S. (Eds.). (2002). Yhdessä ammattikorkeakouluopintoja edis-täen. [Promoting individual learning programmes in polytechnics]. Hämeenlinna: Hämeenammattikorkeakoulu.

Lehtinen, E., & Jokinen, T. (1996). Tutor. Itsenäistyvän oppijan ohjaaja. [Tutor – promotingautonomous learning]. Jyväskylä: Atena.

Maguire, M., & Killeen, J. (2003) Outcomes from career information and guidance services. Apaper prepared for and OECD review of policies for information, guidance and counsellingservices. Commissioned jointly by the European Commission and the OECD. RetrievedJune 4, 2003, from http://www.oecd.org/els/education/careerguidance/.

Ministry of Education (1999). Education and research 1999–2004. Development plan.Retrieved June 4, 2003, from http://www.minedu.fi/julkaisut/KESU2004/eng/engKESU.html.

Ministry of Education (1998). Opintojen etenemisen tehostaminen. [Promoting the completionof studies]. Opetusministeriön työryhmien muistioita 15:1998. Retrieved June 4, 2003from http://www.minedu.fi/julkaisut/julkaisusarjat/15trmuistio.html.

Moitus, S., Huttu, K., Isohanni, I., Lerkkanen, J., Mielityinen, I., Talvi, U., Uusi-Rauva,E., & Vuorinen, R. (2001). Opintojen ohjauksen arviointi korkeakouluissa. [Evaluation


of guidance services in higher education institutions]. Korkeakoulun arviointineuvostonjulkaisuja 13:2001. Helsinki: Edita.

Patton, M. Q. (2002). Qualitative research & evaluation methods. (3rd ed.). Thousand Oaks:Sage.

Plant, P. (2002). Quality in creers guidance. A paper prepared for and OECD review ofpolicies for information, guidance and counselling services. Commissioned jointly bythe European Commission and the OECD. Retrieved June 4, 2003, from http://www.oecd.org/els/education/careerguidance/.

Quality Assurance Agency in Higher Education (QAA) (2003). Code of practice for theassurance of academic quality and standards in higher education: Career education,information and guidance. Retrieved June 4, 2003 from http://www.qaa.ac.uk/public/COP/COPcex/contents.htm.

Reisenberger, A. (1994). Managing the delivery of guidance in colleges. Further EducationUnit. Department of Employment. Dorset: Blackmore Press.

Sakari, J-P. (2000). Akateemiset rekrytointipalvelut Suomessa. [Academic career services inFinland]. Oulu: Oulun yliopisto.

Sweet. R. (2001). Career information, guidance and counselling services: Policy perspectives.Australian Journal of Career Development, 10(2), 11–14.

Tarkiainen, A., & Vuorinen, R. (1998, February). Guidance and Counseling in open LearningEnvironments, a national project in developing guidance provision in Finnish polytechnics1996–1997. Paper presented at the 46th Conference of the International Association forEducational and Vocational Guidance, Tampere, Finland.

Vuorinen, R., & Kasurinen, H. (Eds). (2002). Ohjaus Suomessa 2002. [Guidance and coun-selling in Finland 2002]. Jyväskylä: University of Jyväskylä, Institute for EducationalResearch.

Watt, G. (1998), Supporting employability: Guides to good practice in employment coun-selling and guidance. Luxembourg: Office for Official Publications of the EuropeanCommission.

Watts, A.G., & Van Esbroeck, R. (1998). New skills for New Futures. Higher educationguidance and counselling services in the European Union. Brussels: VUBPRESS.

Documents

Evaluation of Guidance Services in Higher Education in Finland