Embed Size (px)
Citation preview
Final Report
Report of External Evaluation and Review EnterpriseMIT Ltd Confident in educational performance
Confident in capability in self-assessment
Date of report: 2 July 2014
2
Contents
Purpose of this Report ................................................................... 3
Introduction ................................................................................... 3
1. TEO in context .......................................................................................... 3
2. Scope of external evaluation and review .................................................. 6
3. Conduct of external evaluation and review ............................................... 7
Summary of Results ...................................................................... 8
Findings ........................................................................................ 9
Recommendations ...................................................................... 20
Appendix ..................................................................................... 21
MoE Number: 7166
NZQA Reference: C12761
Date of EER visit: 24-26 September 2013
3
Purpose of this Report The purpose of this external evaluation and review report is to provide a public statement about the Tertiary Education Organisation’s (TEO) educational performance and capability in self-assessment. It forms part of the accountability process required by Government to inform investors, the public, students, prospective students, communities, employers, and other interested parties. It is also intended to be used by the TEO itself for quality improvement purposes.
Introduction
1. TEO in context
Name of TEO: EnterpriseMIT Ltd (operating as Mahurangi Technical Institute (MTI) and referred to in this report by that operational title)
Type: Private training establishment (PTE)
Location: 11 Glenmore Drive, Warkworth
Delivery sites: 11 Glenmore Drive, Warkworth. On occasion, depending on demand, programmes are delivered in the Bay of Islands and the Chatham Islands, and at corporate venues.
First registered: 16 August 2012
Courses currently delivered:
Certificate in Offshore Mariner – International; Offshore Yacht Master Certificate in NZ Coastal Mariner with International Coastal Yacht Master; Certificate in Inshore Launch Master with International Coastal Yacht Master; Certificate in Marine Engineering Class 3; Certificate in Marine Technology; Certificates in Companion Animal Services; Certificate in Aquatic Studies; Certificate in Finfish Culture; Certificate in Superyacht Crewing; National Certificate in Maritime (Commercial Inshore Vessel Operation); Certificate in Employment Skills
Code of Practice signatory: Yes
Number of students1: 303 EFTS (equivalent full-time students) through
1 Based on data provided by EnterpriseMIT
4
various funding streams in 2012
Pakeha 74.8 per cent, European 11 per cent, Maori 21.4 per cent, Chinese 2 per cent, Pasifika 1 per cent
Male 58 per cent; female 42 per cent
Under 25 years 50 per cent
International: one (at the time of the evaluation)
Number of staff: 23 full-time equivalents
Scope of Consent to Assess:
MTI holds an extensive scope of consent to assess, up to and including level 6. This includes subfields, domains and unit standards in: Agriculture, Forestry and Fisheries, Seafood; Community and Social Services; Service Sector.
Distinctive characteristics: MTI is a division of EnterpriseMIT, a PTE registered in August 2012 with NZQA. It operates relatively autonomously, with a general manager reporting to a board which includes representation from the major shareholder, Manukau Institute of Technology. MTI previously operated as an NZQA-registered training provider for more than 23 years, and over that time developed a specific profile as a provider of aquaculture, maritime and service sector programmes. Through a related partner organisation, MTI has also been involved with research and consultancy nationally and internationally in aquaculture, fisheries technology, fish breeding and associated training.
Recent significant changes: As noted, since 2012 MTI has operated as a division of EnterpriseMIT, a PTE registered in August 2012 with NZQA.
The operations manager, a long-standing staff member, resigned to move overseas shortly before the external evaluation and review. A tutor with extensive experience at MTI and with an industry training organisation (ITO) was employed in an acting operational management role during the evaluation team’s on-site visit.
Previous quality assurance history:
Because of its Maritime programme delivery, MTI is subject to quality assurance by Maritime New Zealand of MTI training vessels, educator compliance audits and radio spectrum audits. The
5
most recent of these was in June 2013, and MTI was found to be compliant with no outstanding issues or concerns. Offshore mariner instructor validation occurs with the Royal Yachting Association (United Kingdom) every five years, and the International Maritime Organization also conducts annual technical audits.
External moderation by industry training organisations and other bodies is occurring with Competenz, New Zealand Industry Training Organisation (NZITO), NZQA, ServiceIQ and the Motor Industry Training Organisation. MTI records and tracks all external moderation activity, and based on 2012 and 2013 results almost all assessor decisions were verified. Adjustments and changes were made as applicable.
As MTI delivers short courses in first aid, it was required to submit trainer eligibility and training and qualification requirements as a first aid trainer to NZQA. These were provided in January 2013, and additional discussions with the first aid tutor occurred during this evaluation. MTI is complying with NZQA first aid training requirements.
Other: MTI staff professional affiliations include, for example: the Maritime Seafood Educators Association Aotearoa (founding member and current executive member); Coastguard New Zealand; Marine Transport Association; Companion Animal Council (chair 2011-2013); Aquaculture Aotearoa (current chair); Competenz (sector advisory group member); NZITO (sector advisory group member seafood).
The general manager of MTI was the recipient of a Winston Churchill Memorial Trust fellowship in 2013, to visit the United Kingdom’s leading freshwater hatchery to learn techniques for the captive production of rare freshwater fish and their preparation for release for conservation projects.
6
2. Scope of external evaluation and review During a scoping visit to the PTE by the lead evaluator, four focus areas were selected:
• Governance, management and strategy
This is a mandatory focus area.
• Certificate in Offshore Mariner (Level 4); Certificate in New Zealand Coastal Mariner (Level 3) (and referred to in this report as the Maritime programmes)
This cluster of associated programmes is funded by the Tertiary Education Commission (TEC) through Student Achievement Component (SAC) funding, but does not qualify for student loans and allowances. The programmes incorporate NZQA unit standards up to level 4, leading to the National Certificate in Maritime (Commercial Inshore Vessel Operations) (Level 3). This is a significant and relatively complex area of delivery, with multiple sectors of interest and prerequisite maritime experience. They also fit closely with MTI’s espoused profile as a specialist maritime PTE. The programmes have also been offered across the country.
• National Certificate in Employment Skills (Level 1)
This programme is TEC funded under the Foundation Focused Training Opportunities (FFTO) stream, and incorporates the 60-credit National Certificate in Employment Skills (Level 1) (NCES). Twelve ‘learner places’ are funded. The programme is delivered on the Warkworth campus, but unlike the other programmes selected almost all of the students reside in the Rodney District. These students are referred to the programme by WINZ. The learning, guidance and support needs of these learners are significantly different to those of learners on the other programmes selected.
• Certificate in Finfish Culture (Level 3)
This 70-credit ‘local module’ programme is also funded by the TEC through SAC, but does not qualify for student loans and allowances. It is a distance delivery programme, and relates to MTI’s claim to be the ‘largest NZ Provider of distance learning for Animal Services and Aquatic qualifications’.2 The majority of the students are employed by companion animal retailers.
International student support was considered as part of key evaluation question five (section 1.5 of this report) as the PTE currently has only one international student.
2 Source: MTI Investment Plan 2012-13.
7
3. Conduct of external evaluation and review All external evaluation and reviews are conducted in accordance with NZQA’s published policies and procedures. The methodology used is described fully in the web document Policy and Guidelines for the Conduct of External Evaluation and Review available at: http://www.nzqa.govt.nz/providers-partners/registration-and-accreditation/external-evaluation-and-review/policy-and-guidelines-eer/introduction. The TEO has an opportunity to comment on the accuracy of this report, and any submissions received are fully considered by NZQA before finalising the report.
Two evaluators conducted the on-site component of the evaluation over three full days. The evaluators met with the general manager, tutors from the programme focus areas, the acting operations manager, the head of department maritime, and student liaison and administration staff. Two groups of students from the National Certificate in Employment Skills programme were interviewed. A range of key industry stakeholders, government agency staff, students, graduates and employers of graduates were also contacted by phone.
Key documents considered by the evaluators during the evaluation included the PTE’s investment plan; the 2014 strategic plan; end-of-course programme reports; staff meeting minutes; moderation reports from NZQA and three ITOs, with related correspondence; individual and aggregated student feedback; learning and assessment materials and miscellaneous stakeholder audit reports and related correspondence. MTI provided a summary report on its self-assessment approach and activities to NZQA before the evaluation.
Because EnterpriseMIT is a newly registered PTE, no TEC 2012 educational performance data was available. MTI provided the TEC 2011 educational performance report, historical and current performance information in the form of end-of-course programme reports and data from the student management system, and this was compared with funding-related correspondence from the TEC.
8
Summary of Results
Statement of confidence on educational performance NZQA is Confident in the educational performance of EnterpriseMIT Ltd.
• Learner achievement at MTI is good (against a national PTE benchmark), but somewhat variable. Graduates derive clear value from the programmes of study.
• MTI’s programmes meet diverse stakeholder needs; employers, students and graduates are largely satisfied with content and outcomes, which often link to wider regulatory frameworks.
• MTI’s programmes are relevant and current, and are informed by interaction with industry, with one weaker case contextualised within this report.
• The quality of delivery is subject to external moderation, audit and review by other agencies, and reporting from these sources is very good.
Statement of confidence on capability in self-assessment NZQA is Confident in the capability in self-assessment of EnterpriseMIT Ltd.
• MTI has a systematic approach to understanding need, developing good educational programmes, monitoring quality and understanding outcomes, but with some evident limitations on the last point.
• Documentation and use of self-assessment tools and processes were apparent, and widely understood, but with limitations on consistent reporting, understanding and use of higher-level educational performance data.
• Numerous cases of worthwhile changes and improvements resulting from self-assessment were identified and described to the evaluators.
• Significant change initiatives have been identified by way of self-assessment, and these are being implemented in a planned way.
• Based on the factors outlined above, NZQA can express confidence in MTI’s educational performance and capability in self-assessment.
9
Findings3
1.1 How well do learners achieve?
The rating for performance in relation to this key evaluation question is Good.
The rating for capability in self-assessment for this key evaluation question is Adequate.
Learner achievement on MTI programmes is generally good, with some variation of overall performance noted. The majority of students in two of the focus areas achieve the qualifications on offer, and both graduates and employers indicated to the evaluators a high degree of satisfaction with the training. The achievement of both the Maritime and Finfish programmes is directly related to industry and student need, and the evaluators found clear evidence of this in the information provided by MTI and in direct communication with graduates, employers and related industry stakeholders. The employment skills programme, while based around meeting a need for regional preparation for employment, is not achieving its contractual requirements and may not be particularly well aligned to learner aspirations.
The Maritime programme is based around the National Certificate in Maritime (Commercial Inshore Vessel Operations), which enables graduates to gain the knowledge and skills to meet Maritime New Zealand and other regulatory or company requirements for the safe operation of vessels. Uniquely, this programme also requires a considerable amount of ‘sea service’ time (validated by Maritime New Zealand) to achieve the possible outcome qualifications, such as the restricted radio licence, the local launch operator qualification (LLO), and the inshore launchmaster qualification (ILM). MTI assists students with the validation process for attaining these qualifications. Students enrolling on this programme have a variety of outcomes in mind, and some enrol in discrete portions to acquire specific skills, and are not pursuing the full national certificate. Nevertheless, according to MTI data, qualification achievement is as follows:
National Certificate in Maritime (Commercial Inshore Vessel Operations) (Level 3)
2011 2012 2013*
Students enrolled 33 49 42
Achieved national certificate 27 (81%) 33 (67%) 32 (76%)
PTE sector average achievement
(76%) (69%) Not yet available
* Projected from current student progress
3 The findings in this report are derived using a standard process and are based on a targeted sample of the organisation’s activities. The tables which appear were constructed using data supplied to the evaluators during the on-site enquiry.
10
MTI believes the above to be a good level of student achievement, comparing favourably with the sector.4 Completion of the programme has led to new employment or business opportunities and/or promotion for numerous graduates, as indicated by MTI, and the evaluators identified a good range of examples of this in conversation with a random selection of graduates contacted by phone.
Performance data provided by MTI showed that for the Certificate in Finfish Culture (Level 3), qualification achievement is as follows:
Certificate in Finfish Culture (Level 3)
2011 2012 2013*
Students enrolled 49 40 35
Achieved certificate 44 (90%) 28 (70%) 29 (82%)
PTE sector average achievement
(76%) (69%) Not yet available
* Projected from current student progress
Again, MTI believes this to be a good level of student achievement, and it compares favourably with sector averages. Many students and graduates of the Finfish programme are currently working in the pet retail sector. Some students have entered this programme after completing STAR companion animal courses through MTI and, anecdotally, some graduates are known to have continued on to higher-level studies in animal welfare or science.
The National Certificate in Employment Skills programme has relatively low rates of qualification achievement, and in 2012 achieved only 50 per cent progression to employment or higher and further education.5 This is below the 60 per cent TEC contracted performance requirement. Despite the efforts of tutors and students, only 10 of 39 students achieved the full NCES qualification during the allotted 26 weeks in 2012. Tutors reported that some students exit the programme early to take up employment should the opportunity arise. A few students were achieving NCEA level 1, a non-contracted but nevertheless valuable outcome. Tutors were aware of these performance issues, and a range of teaching and motivational strategies were being implemented to raise qualification achievement rates. The PTE’s strategic planning6 ‒ which pre-dated the recent ministerial announcement
4 TEC educational performance reporting for level 3 and 4 qualification completions:
• 2011 TEC qualification completion rates: PTE sector average 76 per cent; MTI 90 per cent (all tertiary providers 67 per cent).
• 2012 TEC qualification completion rates: PTE sector average 75 per cent; MTI data recorded a completion rate of 78 per cent (all tertiary providers 69 per cent)
5 Achievement data for the National Certificate in Employment Skills for 2011, as well as average credit achievement per student, was not provided to the evaluators, although requested.
6 EnterpriseMIT Limited Strategic Plan 2014
11
that FFTO funding was being discontinued at the end of 2013 – indicates that this programme may possibly be superseded by a much broader programme aligned with Youth Guarantee funding and regional needs.7
Learner achievement is monitored and relatively well understood at MTI. It is discussed regularly among tutors, heads of department and management. Learner achievement is captured intentionally using reporting templates for the less formal fortnightly staff meetings and in the formal, monthly, programme-focused meetings. MTI’s end-of-course programme reports, for example, are framed against the six key evaluation questions, and are structured to encourage evaluative thinking. It was not particularly evident to the evaluators how effective these reporting processes were in driving educational performance planning and decision-making year-by-year, and there was less evidence of this monitoring and understanding in the National Certificate in Employment Skills programme, where monitoring was more related to tracking individual learner progress week-on-week.
Performance information was evidently being used for monitoring and improvement. Examples of actions and worthwhile improvements were noted, and are cited elsewhere in this report. Moderation across an extensive range of ITOs and NZQA is consistently good, and in the case of the Maritime programme some examinations are moderated by external agencies, including observation of practical tests. This form of moderation has both validated the content and teaching and led to changes in delivery which have improved performance. An example of this was modifications to teaching of a radar course component, with higher exam pass rates resulting.
The evaluators noted inadequate knowledge and available data of aggregated educational performance at the qualification/organisational level. An informal target of ‘80 per cent or higher completion rates’ was stated by the general manager, but there was little evidence that this measure was being used meaningfully across programmes. The PTE described a possible over-reliance on TEC educational performance information reports rather than developing its own model. Evaluation of performance reporting by the new major shareholder has also led to the conclusion that although performance is monitored, MTI requires more structured reporting on educational performance. Both the shareholder and MTI have concluded that the current student management system is outdated and cannot produce the types of reporting now required for monitoring and improving performance.
7 The Ministry of Social Development has recently requested that MTI enter into a new 2014 contract of provision for employment training within the Rodney District. This request is based on MTI's historical achievements within the FFTO-funded system. It also gives MTI the ability to cease the delivery of the National Certificate in Employment Skills and offer a programme better suited to the WINZ client base. (information provided by MTI subsequent to the EER).
12
1.2 What is the value of the outcomes for key stakeholders, including learners?
The rating for performance in relation to this key evaluation question is Excellent.
The rating for capability in self-assessment for this key evaluation question is Good.
The value of the outcomes for MTI key stakeholders ‒ in terms of both educational performance and capability in self-assessment ‒ is good. This judgement is made on the basis that MTI has a clearly identified network of stakeholders (including industry and learners), actively seeks to understand their requirements, and engages in ongoing reflection and review of the achievement of the value of outcomes. These contributing factors were substantiated by records maintained by MTI, and by the evaluators’ discussions with stakeholders.
Skills, knowledge and attitudinal outcomes resulting from the training were clearly described and/or valued by graduates, employers and wider industry stakeholders. The latter included a range of enterprises, from large franchise operators to small family-owned ferry and fishing businesses. The National Certificate in Employment Skills learners have the opportunity to achieve NCES and NCEA qualifications, as well as support for gaining employment, but performance here is not meeting the contracted requirements of the funder, and is not as strong as in the other two focus areas.
The Finfish programme provides valued skills and knowledge in humanely breeding, handling and retailing aquarium fish, or for fish farming more generally. For example, a major national retailer employing many of these graduates estimated ‘a 40 per cent reduction in fish mortality’ as a result of implementation of practices learned through the programme. This represents business savings of many thousands of dollars. This programme also meets the legal requirements for staff qualifications in animal husbandry relating to the Animal Welfare Act (1999).8 This outcome is important to pet shop owners and ornamental fish breeders.
Boat owners and operators spoken to by the evaluators in various locations around New Zealand said that the licensing requirements they are able to meet through the Maritime programme have enabled them to continue operating in the tourism market, as commercial fishing boats operators or as passenger ferry operators. Some made reference to valuable safety outcomes (‘no notifiable incidents’) and ‘cost savings’ as a result of the training. Others described promotion to a ‘skipper’s role’ and new job opportunities as outcomes.
MTI has forged significant national and international relationships which shape the PTE’s maritime/aquaculture focus. This includes providing information sessions on aquaculture to various hapū nationwide, aquaculture programme development
8 Source: National Animal Welfare Advisory Committee (2010). Temporary Housing of Companion Animals.
13
for the trade and investment arm of the Pacific Islands Forum Secretariat, and recent work with NIWA (National Institute of Water and Atmospheric Research) and the Far North District Council. MTI makes contributions to community life and the dissemination of knowledge well beyond the Warkworth campus.
All of the focus area programmes reflect a dominant industry or purchaser need ‒ be it regulatory, industry, societal (for example, animal welfare concerns) or those of a particular government agency purchaser. MTI is knowledgeable about these needs and works effectively to meet them. Programmes maintain close connections with the related organisations and employers, often in formal ways such as through the TRoQ process and national associations. However, there are limitations in MTI’s current self-assessment activities, which are not sufficient to demonstrate clear and comprehensive evidence that self-assessment is leading directly to improved outcomes.
1.3 How well do programmes and activities match the needs of learners and other stakeholders?
The rating for performance in relation to this key evaluation question is Excellent.
The rating for capability in self-assessment for this key evaluation question is Good.
MTI has a clearly identified community of interest relating to each of the MTI programmes considered in this evaluation, and in all cases there is a clear match with stakeholders’ needs. Needs assessment has informed the design of the programmes, and MTI is continuously engaging with external stakeholders to ensure a close match between its current offerings and wider requirements. The programme design ethos guiding the PTE is that, ‘we need to be teaching to industry requirements’. Representatives of industry stakeholders spoken to by the evaluators endorsed the relevance of the programmes, and MTI was described as ‘responsive’.
First offered in 2010, the Finfish programme was specifically designed with industry input to fill a gap in the training environment. It is also congruent with the increasing public awareness around animal welfare; the tutor who led the programme development is a recent chair of the Companion Animal Council. The programme also addresses the Ministry of Primary Industry’s objectives around animal welfare policy and practice in New Zealand. A tutor from this programme is currently involved with the TRoQ process for animal care qualifications, and all tutors have direct contact with the students and workplace supervisors. This informs the content and assessment. For instance, integration of new lighting technologies and more knowledge around tank materials have resulted from this ongoing review activity. Comments provided to tutors indicate that students and graduates are better equipped to understand and service customer requirements on reasonably technical issues as a result of the programme.
14
The Maritime programme has content specifically required for employment, licensing and day-to-day safe operation in a number of maritime environments, and is directly transferable to the workplace. Various examples of the relevance of the programme were referenced by MTI, and others were presented directly by graduates, including supporting whānau fishing business objectives on the Chatham Islands, small business tourism operators on Auckland harbour, and meeting the staff training plan objectives of passenger vessel operators. The programme design is necessarily complex, but functions well. Students and graduates contacted were strongly supportive of the programme as matching their needs and aspirations. This included the mode of delivery as well as the content. MTI offers the programme in various locations to meet the needs of students and stakeholders.
There are inherent weaknesses in the correlation between the design of the 26-week National Certificate in Employment Skills and the aspirations of the people referred to the programme. It is, however, being delivered as contracted by the TEC, and delivers useful content to the diverse group of students referred by WINZ. Based on their own experience, and their own self-assessment, the tutors articulate a reasoned and well-formulated series of goals for the programme as delivered. Students spoken to by the evaluators expressed quite varied goals, ranging from achieving a qualification, to getting a job, to gaining enough credits for NCEA level 1 or 2 and updating their knowledge of ‘what employers are looking for’. Once on the programme, the students’ level of engagement with the content is reasonable, and the tutors provided some good examples of learner achievement of employment and a first qualification, but these examples tended to be anecdotal and not comprehensively tracked or recorded. Within the scope of the embedded NCES, tutors have selected particular unit standards they believe best meet the needs of the learners, but achievement overall remains below contracted requirements. With the loss of FFTO funding, MTI has an opportunity to re-evaluate how best it can respond to this learner group, and the strategic plan for 2014 has clear goals in this regard.
MTI programmes have appropriate content, shaped by ongoing consultation with industry, which relates directly to the skills and knowledge required for employment or commercial activities within the PTE’s scope of training. Delivery modes vary between face-to-face, block course and distance delivery modes, and in most instances learner achievement is good. Self-assessment around matching needs is also good.
15
1.4 How effective is the teaching?
The rating for performance in relation to this key evaluation question is Good.
The rating for capability in self-assessment for this key evaluation question is Good.
Based on MTI’s student survey results, and evaluators’ discussions with students and graduates, there is clear evidence of effective engagement between tutors and students across the programmes. Students described the tutors as knowledgeable, approachable and dedicated. Tutors are appropriately qualified and prepared to teach in their subject areas, and in some cases are recognised by external stakeholders as experts in their fields. Subject knowledge is current, and teaching staff are supported by the organisation to participate in external forums and relevant professional development, which supports this currency.
The ethos around approaches to teaching and learning was distinct to each of the three programme focus areas. Some programmes tended to be more learner-centred, while others had a strong focus on content knowledge for exams and licensing. There was an evident willingness to modify teaching strategies to either aid understanding or enhance opportunities for success. Where students were not satisfied, tutors and heads of department were responsive, and worthwhile improvements were made as required. Monthly and annual tutor and programme reports capture this responsiveness, although in some instances the narratives were more descriptive than evaluative.
Oversight of teaching performance and tutor development could be clearer and more consistently documented, although there was some evidence that peer review is occurring. Formal student evaluation processes are used to inform and improve practice, although this was not as clear for the National Certificate in Employment Skills, and the survey return rate for the Finfish distance programme is quite low.
Examples of responsiveness to learner and/or stakeholder feedback leading to better teaching were provided, and changes to content, teaching methods and assessment are occurring. Of particular note is the use of assessment and recognition of prior learning on the Finfish programme occurring within students’ workplaces. This programme also incorporates a flexible structure so that students can sequence their own learning. This, coupled with additional teaching support provided by phone and email, has been linked to the good qualification completion rate. Significant changes to, for example, teaching of the navigation and radar components of the Maritime programme were provided. Students described the main employment skills tutor as particularly responsive and able to create an appropriate learning environment, despite the fact that some students were reluctantly on the programme and thought that it did not ‘acknowledge their life experience’. Some students also considered the required unit standard assessments somewhat ‘onerous’ and ‘pedantic’.
Good quality teaching materials in booklet formats, which are subject to regular review, are used to support delivery, but there is no current use of e-learning
16
approaches to support teaching and assessment strategies. The online TEC literacy and numeracy assessment tool is being used with the Certificate in Employment Skills students, and is being piloted on the Finfish programme, and is informing tutors’ understanding of individual learner needs. The employment skills tutors have, however, observed little in the way of improvement against the adult literacy and numeracy learning progressions. Tutors described new measures they were now employing to improve both the validity and students’ valuing of this testing. Tutors believed the related Pathways Awarua online learning programme has been of more value to the students, and some of the students interviewed supported this view.
1.5 How well are learners guided and supported?
The rating for performance in relation to this key evaluation question is Good.
The rating for capability in self-assessment for this key evaluation question is Good.
A range of effective processes and resources are providing good guidance and support for MTI students, and self-assessment of these is integrated with general self-assessment protocols and methods. Notable cross-programme resources include a dedicated student support position, provision of free daily transport to the venue, and assistance with affordable accommodation. Apart from the Youth Guarantee and FFTO programmes, most of MTI’s campus-based students normally reside outside the local area, so these services are particularly significant for meeting needs and the achievement of educational outcomes. A small library, a relaxation area and computers for general use are provided. The general welfare and well-being needs of students, including the small number of international students, are being met across the programmes and the campus.9
Some informal and formal learner needs assessment is occurring and being responded to across programmes. Administration staff, who act as first point of contact for most prospective students, are well versed in the requirements of the various programmes and are able to provide relevant guidance. In the case of the Maritime programme, these staff play an important role in assisting with the process of validating ‘sea time’ and obtaining the necessary examination permits from Maritime New Zealand. Guidance is also provided by accurate brochures and website information.
Teaching staff are readily available and invest extra time into student learning as required ‒ campus-based students of the Maritime and Certificate in Employment Skills programmes provided the evaluators with information about tutors being available out of scheduled class time to offer additional learning support. This was
9 MTI had recently completed its annual self-review of the Code of Practice for the Pastoral Care of International Students, and documentation supporting this was submitted to NZQA. This material was sighted by the evaluators.
17
also evident from comments made by graduates. In addition, students and graduates reported that assessment protocols and requirements were clearly communicated and assessment processes were fair.
Opportunities for formative assessment and recognition of prior learning or current competence are effectively integrated, particularly in the Maritime and Finfish programmes. Finfish students are also supported by occasional workplace visits and tutors are contactable by phone or email. These students may potentially benefit by more regular workplace visits or e-learning support to strengthen educational performance. The links between monitoring progress and gathering information for effective, evidence-based self-assessment were inconsistent in the Finfish programme.
As noted above, the TEC literacy and numeracy assessment tool is being used in some programmes, but at this stage not in a particularly integrated or embedded way. It was noted that there are no specific plans for tutor professional development around interpreting and using this resource.
Guidance and support of students is timely and appropriate and linked to supporting learners to achieve their learning goals.
1.6 How effective are governance and management in supporting educational achievement?
The rating for performance in relation to this key evaluation question is Good.
The rating for capability in self-assessment for this key evaluation question is Adequate.
MTI has a clearly stated vision and strategy, as described in the 2014 strategic plan. This plan, and the ‘key shifts and strategies’ described in the investment plan with the TEC, are effectively guiding both governance and management decision-making. There are sound planning and financial control processes in place. Training facilities such as training vessels, a fire-fighting simulator unit and aquaculture plant represent significant ongoing investment by the PTE, and in some cases incur regular third-party audit and compliance costs (for example by the International Maritime Organization and Maritime New Zealand). In addition, the provision of transport for students reflects a positive approach to ‘making it happen’ in Warkworth, and not allowing the geographical location to limit the training that can be offered from there.
Staff are aware of change, and expectations are communicated through regular meetings. Some process changes brought about by the new ownership arrangements are starting to have an impact on the workplace, and the evaluators found the staff to be very well engaged with the organisational change. Staff spoke highly of management and are quietly optimistic that the changes will be beneficial. Staff are provided with support to engage in appropriate professional development,
18
although there is a need to ensure that stated policy objectives around completion of adult education qualifications are followed through in a few cases.
An industry advisory group meets at least annually at Warkworth and provides valued support to the general manager. Based on the minutes, the group serves mainly as a networking and information-sharing forum rather than as a mechanism for providing direction to programmes or as a component of self-assessment activity.
As reported above, there are clearly some inconsistencies and gaps in monitoring educational performance at qualification and benchmarking levels. These have been elaborated throughout this report. MTI is possibly somewhat reliant on external bodies to validate or critique its educational practices, but regardless, this external oversight requires constant readjustment by the PTE to meet external standards in training, and is valid and useful. The evaluators found management and staff to be pragmatic and positively engaged with the often multi-layered audit and compliance aspects which touch upon, in particular, maritime training.
Evaluators noted some risk around the loss of a long-standing operations manager, but this is somewhat mitigated by the recent change of ownership and the potential for establishing new standards and requirements around self-assessment information.
19
Focus Areas
This section reports significant findings in each focus area, not already covered in Part 1.
2.1 Focus area: Governance, management and strategy
The rating in this focus area for educational performance is Good.
The rating for capability in self-assessment for this focus area is Adequate.
2.2 Focus area: Certificate in Offshore Mariner (Level 4); Certificate in New Zealand Coastal Mariner (Level 3) (associated short-courses)
The rating in this focus area for educational performance is Good.
The rating for capability in self-assessment for this focus area is Good.
2.3 Focus area: National Certificate in Employment Skills (Level 1)
The rating in this focus area for educational performance is Adequate.
The rating for capability in self-assessment for this focus area is Adequate.
2.4 Focus area: Certificate in Finfish Culture (Level 3)
The rating in this focus area for educational performance is Good.
The rating for capability in self-assessment for this focus area is Good.
20
Recommendations NZQA recommends that MTI:
• Enhance self-assessment activities and reports to provide readily accessible educational performance and progress reporting available for use across the organisation. This should include demographic-specific information on participation and achievement, and allow for comparisons of programme performance over time.
• Review oversight of teaching performance and tutor professional development to ensure consistency of practice across the diverse programme areas being delivered.
• Consider staff development around the embedding of literacy and numeracy into programmes. The National Certificate in Adult Literacy and Numeracy (Educator) (Level 5) may be a useful target qualification for at least some staff.
• Review the terms of reference and membership of the industry advisory group with a view to refocusing its activity to assist with strengthening educational performance and contributing to self-assessment activities.
21
Appendix Regulatory basis for external evaluation and review
External evaluation and review is conducted according to the External Evaluation and Review (EER) Rules 2013, which are made by NZQA under section 253 of the Education Act 1989 and approved by the NZQA Board and the Minister for Tertiary Education, Skills and Employment.
Self-assessment and participation in external evaluation and review are requirements for maintaining accreditation to provide an approved programme for all TEOs other than universities. The requirements are set through the NZQF Programme Approval and Accreditation Rules 2013, which are also made by NZQA under section 253 of the Education Act 1989 and approved by the NZQA Board and the Minister for Tertiary Education, Skills and Employment.
In addition, the Private Training Establishment Registration Rules 2013 require registered private training establishments to undertake self-assessment and participate in external evaluation and review, in accordance with the External Evaluation and Review Rules (EER) 2013, as a condition of maintaining registration. The Private Training Establishment Registration Rules 2013 are also made by NZQA under section 253 of the Education Act 1989 and approved by the NZQA Board and the Minister for Tertiary Education, Skills and Employment.
NZQA is responsible for ensuring non-university TEOs continue to comply with the rules after the initial granting of approval and accreditation of programmes and/or registration. The New Zealand Vice-Chancellors’ Committee (NZVCC) has statutory responsibility for compliance by universities.
This report reflects the findings and conclusions of the external evaluation and review process, conducted according to the External Evaluation and Review (EER) Rules 2013.
The report identifies strengths and areas for improvement in terms of the organisation’s educational performance and capability in self-assessment.
External evaluation and review reports are one contributing piece of information in determining future funding decisions where the organisation is a funded TEO subject to an investment plan agreed with the Tertiary Education Commission.
External evaluation and review reports are public information and are available from the NZQA website (www.nzqa.govt.nz).
The External Evaluation and Review (EER) Rules 2013 are available at http://www.nzqa.govt.nz/assets/About-us/Our-role/Rules/EER-Rules.pdf, while information about the conduct and methodology for external evaluation and review can be found at http://www.nzqa.govt.nz/providers-partners/external-evaluation-and-review/policy-and-guidelines-eer/introduction/.
