DIT review report of the external review panel150611FINAL Institutional Review... · The Dublin Institute of Technology (DIT) is one of the oldest higher education institutions in

1

Review by the National Qualifications Authority of Ireland of the effectiveness of the quality assurance procedures of the

Dublin Institute of Technology

REPORT OF THE EXTERNAL REVIEW PANEL

June 2011

i

TABLE OF CONTENTS

INTRODUCTION ......................................................................................................................... 1

Section 1: The extent to which the DIT meets the European Standards and Guidelines for internal quality assurance ……………………………………………….……………….…..…………3

1.1 Policy and Procedures for Quality Assurance………………………………….…….…….4

1.2 Approval, monitoring and periodic review of programmes and awards…….10

1.3 Assessment of students…………..…………………………………………………………………17

1.4 Quality assurance of teaching staff…………………………..……………………………… 21

1.5 Learning resources and student support.......................................................24

1.6 Information systems………………………………………..………………………………………..28

1.7 Public Information……………………………………………………………………………………..31

Section 2: Follow‐up to the EUA review of the effectiveness of DIT's quality assurance procedures (2005/06)……………………………………………………………………………..………33

2.1 Context………………………………………………………………………………………………………………….33

2.2 Overview……………………………………………………………………………………………….………..…....33

2.3 Research………………………………………………………………………………………………….……..……..34

2.4 Internationalisation……………………………………………………………………………………….…..….34

Section 3: Overall findings and concluding comments…………………….……………………............36

3.1 Overall findings………………………………………………………………………………………….…………..36

3.2 Concluding comments………………………………………………………………………………….……..…36

APPENDICES………………………………………………………………………………………………………………….…..39

Appendix 1 – Terms of reference for the review…………………………………………………………..…...39

Appendix 2 – Members of the external review panel ……………………………………………….……....45

Appendix 3 – Site Visit Schedule, 29 March – 1 April 2011……………………………….……….….……48

Appendix 4 – List of External Stakeholders met by the external review panel......................59

1

INTRODUCTION

The Dublin Institute of Technology (DIT) is one of the oldest higher education institutions in Ireland and is now one of the largest in terms of staff and student numbers. It is constituted in legislation under the Dublin Institute of Technology Act 1992. In 1998, the DIT was granted degree awarding powers by the Irish state, and makes awards at Levels 6 to 10 on the National Framework of Qualifications (NFQ). As a statutory awarding body and provider of higher education and training, and in line with national and international practice, the primary responsibility for the quality assurance of the educational provision and awards made by DIT resides with the Institute itself.

The DIT is subject to external quality assurance, as set out in the Qualifications (Education and Training) Act, 1999. This requires the National Qualifications Authority of Ireland (NQAI) to review the effectiveness of the Institute’s quality assurance procedures “not more than once in every three years and not less than once in every seven years”. In 2004, the NQAI and the DIT commissioned the European University Association (EUA) to undertake the first such review. This was completed in 2006. Given that it is standard national practice that higher education institutions be reviewed on a quinquennial basis, the NQAI decided, in consultation with DIT, that a review should take place in the academic year 2010-11. The objectives for the review, set out in the Terms of Reference (see Appendix 1), were to evaluate:

• the effectiveness of the procedures operated by DIT for the quality assurance of its education and training programmes, and the awards to which they lead in the National Framework of Qualifications, both nationally and internationally;

• the extent to which DIT, in operating these procedures meets the requisite standards, as set out in Part 1 of the Standards and Guidelines for Quality Assurance in the European Higher Education Area; and

• the extent to which the DIT has implemented the 33 recommendations made by the EUA review team in 2006.1

The review process consists of:

• an institutional self-evaluation report prepared by the DIT;

• a review of the self-evaluation report by an expert panel;

• a site visit by the expert panel to the DIT;

• a report by the expert panel for submission to the NQAI; 1 See report, recommendations and additional information at http://www.nqai.ie/award_dit_rev.html

2

• an institutional response by the DIT to the panel report;

• consideration of the external panel’s report and institutional response by the NQAI; and

• publication by NQAI of the review outcomes.

This is the report of the Expert Panel. The Panel members were: Jon Haakstad (Chair), Gordon Clark, Roger Downer, Andrea Nolan, Anna Pringle and Shane Kelly (see Appendix 2 for details). The NQAI acted as secretariat to the Panel. This report is based on the self-evaluation report of the DIT, the site visit by the Panel to the DIT, 28 March-1 April 2011 and supplementary documentation provided by the DIT.

This report is structured around the major criteria to be addressed in the Terms of Reference for the review. The report is evidence-based and aims to support the DIT in enhancing its quality assurance system. Section 1 addresses the effectiveness of the DIT’s quality assurance procedures against each of the seven standards in Part 1 of the Standards and Guidelines for Quality Assurance in the European Higher Education Area. Each is discussed under the headings of context, issues arising, commendations and recommendations. Section 2 examines the extent to which the DIT has implemented the 33 recommendations made in the EUA report of 2006. As many of the issues arising are addressed in Section 1, Section 2 provides an overview of follow-up and focuses on the themes of research and internationalisation. Section 3 provides the overall findings and concluding comments of the Panel.

The Panel appreciates the openness and frankness with which the DIT entered the review process. Staff and students with whom the Panel met were open and cooperative, and requests for information made throughout the course of the site visit were met readily. We thank the DIT for this - it made our work easy. We thank Jan Cairns and Nicole O’Neill, Quality Assurance Officers, for their work on the review, and in particular for providing documentation and clarifications to the Panel during the site visit. The Panel would also like to acknowledge the warm hospitality with which it was received and how forthcoming and accommodating the DIT has been throughout the review process.

It strikes the Panel that the DIT is a quality-oriented institution, with a strong student-centred approach to its activities. Overall, the Panel found staff to be extremely positive and expressed a genuine care for their students and a loyalty to their institution. At the same time, it is an institution that is aware of, and receptive to, feedback from its societal constituents and collaborative partners. We find that the Institute’s quality assurance procedures and practices, in the round, are effective and are consistent with those articulated in the European Standards and Guidelines for Quality Assurance in the European Higher Education Area.

On behalf of the Panel, I would also like to thank the NQAI secretariat to the review, Anna Murphy and Deirdre Stritch.

Jon Haakstad, Chair

3

Section 1: The extent to which the DIT meets the European Standards and Guidelines for internal quality assurance

Introduction

The review panel assessed the effectiveness of the quality assurance procedures of the DIT against the seven standards contained in Part 1 of the European standards and guidelines for quality assurance in the European Higher Education Area (generally referred to as the European Standards and Guidelines).2 These relate to:

‐ Policies and procedures for quality assurance; ‐ Approval, monitoring and periodic review of programmes and awards; ‐ Assessment of students; ‐ Quality assurance of teaching staff; ‐ Learning resources and student support; ‐ Information Systems; and ‐ Public information.

Each of these is considered in the following sections, and includes, as appropriate to each section, consideration of the implementation of the National Framework of Qualifications and its associated policies on access, transfer and progression, and quality assurance procedures for linked national and international provision of DIT programmes and awards.

2 Part 1 of the guidelines refers to higher education institutions, see: http://www.eqar.eu/fileadmin/documents/e4/050221_ENQA_report.pdf

4

1.1: Policy and procedures for quality assurance

Context The Dublin Institute of Technology has developed its quality assurance system over many years. The first edition of the Handbook for Quality Assurance, 1995, codified the best practices developed across the Institute over the previous twenty years. The latest 2010 edition and related guidelines and regulations3 reflect developments and innovations in the interim period. It has been informed by inputs from members of staff, students, graduates and a range of external stakeholders. The Handbook sets out the full range of DIT’s quality assurance procedures:

‐ procedures for review of the quality of programmes, research, schools, colleges, non-teaching units (such as Library Services) and related activities; and

‐ common templates for proposals for new programmes, the nomination of external members of validation/review panels, reports of validation and review panels, annual monitoring of programmes, module-level student surveys, annual review of modules and the annual survey of students (programme-level).

The Handbook is supplemented by guidelines posted on the Academic Affairs website that provide greater specific detail on issues, such as assessment. The Handbook forms a part of the induction of every new member of staff. At institutional level, the responsibility for academic quality assurance falls within the remit of the Directorate of Academic Affairs and Registrar. Responsibility is primarily exercised through Academic Council (which reports to the Governing Body) and College Boards, and is regulated through the procedures contained in the Handbook. A number of sub-committees of Academic Council are responsible for specific aspects of quality assurance. These are Recruitment and Admissions, Academic Quality Assurance, Student Experience, Graduate Research School Board, Learning Teaching and Assessment Strategy, Apprentice Education and Library. The principal monitoring mechanism for quality assurance is the Academic Quality Assurance sub-committee. Its key relationships are with College Boards (four) which in turn relate to their respective School Executives (twenty seven). The latter, in turn, interact directly with Programme Committees. Central to the DIT’s quality assurance and enhancement system is the annual report of each Programme Committee (referred to as a Q5 report). These reports incorporate feedback from students, external examiners, external stakeholders and professional bodies/partners as appropriate, and include data on student numbers and progression. The reports are first reviewed by the School, followed by the sub-committee on academic quality enhancement of the relevant College Board. These feed into action plans which have associated timelines and 3 DIT (2010) Handbook for Academic Quality Enhancement: http://www.dit.ie/media/documents/academicregistrar/AQEH_june10_collated.pdf

5

every School Executive is required to report on progress to the College Board. A note of all proposed changes emerging from the reports is put to the Academic Quality Enhancement sub-committee of Academic Council. At College level, all recommendations requiring reallocation of resources are prioritised and acted upon through the Institute’s relevant committees. A number of Programme Committees have entered into arrangements with external professional bodies, such as the Institute of Chartered Surveyors and Engineers Ireland, to use the annual Q5 as their reporting mechanism. There is general agreement that engagement with professional bodies has a demonstrably positive impact on the quality of the Q5 reports. Every Programme Committee, College Board, Academic Council sub-committee, Academic Council and Governing Body includes student representation. A key element of the annual monitoring report (Q5 report) is the feedback received from students on modules. It is a requirement of quality assurance that all students are handed out a survey (Q6a) by the relevant lecturer for every module they take. Lecturers submit a summary of this feedback (Q6b) to the Programme Chair. A programme survey (Q6c) is also distributed to students at the end of each academic year by the School. Student representatives can present oral feedback through their membership of Programme Committees. The Institute has reviewed 25 out of 27 of its Schools within the past five years. The remaining two will be reviewed in 2011. Three non-academic units have also been reviewed. These, and the outcomes of programme monitoring, generate action plans at the College level. Their implementation is, in general, monitored on an annual basis (some Colleges have a rolling action plan). The DIT plans to commence the cycle of quinquennial reviews of the re-organised Colleges in 2011, when the first cycle of School Reviews will be complete. Issues Arising

The Panel finds that the DIT is a quality-oriented institution. It is concerned with enhancing itself and its services, and thus takes quality seriously. The Panel finds the quality assurance arrangements of the DIT to be comprehensive. They cover all institutional processes and are sufficiently uniform in their conception. It is also clear that the DIT has a strategy to manage quality assurance. Quality assurance procedures are in place at all levels and, in general, appear to work well. The move to School reviews is applauded. However, in reviewing a range of reports of programme monitoring and School reviews, the Panel finds inconsistencies in the extent to which quality assurance procedures are effectively implemented. It also noted a positive correlation between the effectiveness of those procedures and the extent to which programmes are accredited by professional bodies. A further finding is that informal approaches to quality assurance also apply and, in some cases, are more responsive than the formal processes. These also address weaknesses such as those in the Q6 student feedback

6

process (see Section 1.2) in the formal system and introduce innovative responses. In some instances, actions may be taken to deal with issues before they are considered by the appropriate fora in the formal system. Whilst this has benefits in terms of early intervention, these practices can undermine the formal system. The Panel finds that the formal system needs to become more responsive. The issue of dealing with, and communicating feedback on, progress in the follow-up to surveys, reviews and outcomes of programme monitoring was raised in the self-evaluation report and in the course of the Panel’s site visit. Steps have been taken to improve communication of the follow-up to reviews to staff, and staff are appreciative of this as it promotes ownership. The Panel considers that greater access by staff to College action plans and their implementation could further enhance communication. The Panel finds that annual programme monitoring and School reviews are the lynchpin of the quality assurance system (see Section 1.2). These work very well in some Schools. In addition, the College action plans are critical to oversight and leadership. Both could be used to provide an opportunity for a more discursive approach to quality. They could counter-balance a more instrumentalist or technical approach to quality assurance. Although these action plans emanate from a common process, the Colleges develop them independently. They differ in scope, detail and monitoring of follow-up. They appear to work best where the College has a dedicated Head of Learning Development. It is not clear to the Panel how, in general, decisions and information are tracked through the various stages of quality assurance processes. The Head of Learning Development plays a critical role in the Colleges. The Panel finds strong evidence that they are important drivers for quality assurance. These positions remain unfilled in two Colleges, despite a commitment to introducing them since the EUA report, 2006. There is a significant danger that quality assurance processes and quality enhancement could suffer from the lack of a full-time individual in this position with associated administrative support. In terms of strategic thinking, the Panel considers that there is a potential weakness in the DIT’s overall approach to quality assurance in that it did not see a dynamic, strategic attitude to quality assurance and quality enhancement. The Panel would have liked to see more explicit and stronger references to quality assurance in the DIT’s institutional strategy. On the whole, the central level appears to be more concerned with monitoring the large number of procedures that are in place. As a consequence, it is somewhat reactive and technical in its attitude to quality, rather than being pro-active in formulating its idea of, and priorities for, quality assurance, and more importantly, so as to stimulate a broader discussion of quality issues. This attitude was evidenced by the limited self-reflection in the self-evaluation report and in the meetings with the Panel during the site visit. Some consideration might be given by the DIT to the inter-relationships between College, School, periodic programme reviews and programme monitoring so as to avoid overload and a purely process-driven system, and to create space for regular and embedded reflection.

7

Quality enhancement requires more than an awareness and use of procedures, but an integration of the outcomes of quality processes with pedagogical and research strategies. This link is often difficult to discern in the DIT’s procedures and related documents, which are quite technical (although this is also necessary). The Panel is concerned that teaching and learning are not sufficiently interlinked with quality enhancement. For example, these areas are currently within the remit of two different sub-committees of Academic Council. The Handbook for Quality Enhancement is being reviewed in the absence of an Institutional quality enhancement strategy, or an understanding of how such a strategy would link to other institutional strategies. The Panel finds that there is scope for further thinking and development in this area. It suggests that the central level of the DIT should have a stronger driving role in quality assurance and quality enhancement, while taking care not to disenfranchise the Schools or Colleges. Senior leadership might consider how the system could provoke and encourage a more discursive attitude to quality. These comments are offered as a challenge, not as a criticism.

Commendations

The Panel wishes to commend the DIT on the following:

o The fact that a culture of quality is deeply embedded within the DIT. There is a very consistent understanding and acceptance across all levels and categories of staff of quality assurance procedures. These policies and procedures are clear and comprehensive.

o The comprehensive set of policies and procedures that are in place for quality enhancement. The Panel is impressed with the collaborative and consultative developmental process that has been adapted to updating the Handbook for Quality Enhancement and the commitment to making the Handbook a more user-friendly practical academic and management tool.

o The moves to School and College reviews. The Panel considers this to be central to

the coherence and the overall direction of the Institute’s quality assurance processes. The School reviews are of strategic importance. The Panel notes that the process of programme reviews has ended. Instead, these take place within the context of School reviews. This is commended. However, the success of this approach depends on a well-functioning Q5 (annual programme monitoring) process in all Schools and for all programmes (see Section 1.2). Colleges need to maintain oversight of this.

o The development of College action plans. These are a very positive development;

they provide transparency and clearly assign responsibility for actions.

8

Recommendations

The Panel makes the following recommendations to the DIT:

o The DIT, in its next stage of development, needs to contemplate more deeply the

concept of quality assurance and enhancement. In this regard, it should consider how it can make more coherent what is currently a process-driven approach to quality assurance; how it can ensure that ownership of quality processes is central, as well as dispersed; and how, as an institution, it can be more dynamic in its use of quality assurance to ensure that its nature is more analytical and that it focuses on key issues.

o In terms of strategic oversight of quality assurance, the Academic Council and its sub-

committees, particularly the Academic Quality Assurance sub-committee, should focus on activity at the College level. They should review the effective operation of quality assurance processes at the College level. Individual Colleges, in turn, while operating inside an institutional system of processes and reporting, should be delegated more responsibility for setting priorities for Schools and for programmes. Mechanisms to share good practice should be actively promoted at School and College levels, for example, making available all the elements of School reviews (self-evaluation reports, panel reports and follow-up) to all Schools.

o Colleges should act to ensure agreed levels of consistency in the level of quality,

specificity and transparency required of follow-up to College action plans, including feedback loops to School Executives and Programme Committees. The Panel recommends that all Colleges take ownership of the action plans; they should manage the action plans on a rolling basis (as some currently do) so that they are ‘living documents’. The Panel recommends better use of IT to operate these plans so as to increase the potential for sharing good practice, automation and to help identify key priorities for quality assurance and quality enhancement.

o The Panel strongly urges the DIT to fill the remaining two College posts of Head of Learning Development on a full-time basis as a matter of priority and to provide the Heads with the necessary administrative support.

o The DIT should consider how to improve the tracking and monitoring of decisions, documents and records concerning the multiple quality assurance processes, including the introduction of workflow management systems and a records management system.

9

The Panel endorses the proposed actions set out in the DIT’s Self-Evaluation Report:

Codify and disseminate existing best practice in non-academic quality assurance processes (e.g. timetabling, registrations, examinations) as they relate to academic quality assurance

Senior Leadership Team has agreed that a detailed and benchmarked survey of staff will be conducted to clarify specific actions that may be required to improve staff recruitment and selection of policies and their communication

Review the strategy for continuous enhancement of quality in light of the NQAI Review 2011, the National Strategy for Higher Education to 2030, Learning, Teaching and Assessment Strategy 2010 and the possibility of designation as a Technological University

Ensure genuine value is placed on formal evidence that actions/changes ensue from staff feedback in the QE system

Address concerns regarding semesterisation and calendar implications Review all student handbooks

10

1.2: Approval, monitoring and periodic review of programmes and awards Context The DIT’s general approach to programme design is set out in the Learning Teaching and Assessment Strategy, which is currently being reviewed. All of the DIT’s programmes are modularised within a semesterised calendar. Since 2004, all programmes leading to major awards are included in the National Framework of Qualifications (NFQ). The Handbook for Academic Quality Enhancement (2010) details the validation and approval procedures for new programmes. Approval of new programmes leading to major awards of the Institute commences at School level with a new programme proposal outline brief. This is discussed at College Board and tabled for outline approval by the Senior Leadership Team. A programme development team puts forward detailed documentation, including a draft Programme Document. If approved by College Board, the proposal proceeds to validation. A validation review panel is established. The panel must be satisfied that (i) the proposed programme is fit for purpose, (ii) programme learning outcomes reflect the correct NFQ level and (iii) that there is clear alignment between the programme learning outcomes, module learning outcomes and assessment. Academic Council has recently agreed that a rationale for the NFQ level assigned to the proposed programme should be included in programme documents submitted for validation. Admission requirements and procedures; procedures for the recognition of prior learning; procedures for non-standard applicants; and advanced stage transfer and progression opportunities must be specified as part of the validation process. Partner organisations for the programme must be formally accredited as suitable partners of the Institute, prior to programme validation. Programme validations must be approved by Academic Council and subsequently by Governing Body. The Higher Education Authority must also approve any new programmes. A successful validation results in a formal report published on the Institute website that makes a series of conditions and recommendations that should be incorporated into a revised programme for delivery. Programme monitoring is practiced through Programme Committee meetings, which should take place at least twice a year, as well as through annual programme monitoring reports (Q5). Programme Committees monitor the day-to-day operation of programmes and propose any necessary amendments (e.g. to curriculum and resources). Proposals for minor curriculum changes are considered by College Boards and are sent to Academic Council for formal approval. Any major changes are considered by Programme Review Panels. All reports and recommendations from Programme Committees and Review Panels must be approved by Academic Council. The follow-up actions arising from these reports are collated in the College action plans. The practice of regular programme reviews has ceased with the

11

move to School reviews. The latter are intended to include reviews of all programmes within the relevant School. Issues Arising The Panel focused its attention on programme monitoring and on arrangements for programme review. The process of completing Programme Committee annual reports (the Q5 process), which incorporate student feedback is critical for the effectiveness of quality assurance, particularly given that regular programme reviews have ceased. On the whole, there appears to be satisfaction with the process, particularly on the part of academic staff and management. It was clear to the Panel that there are many examples of well-functioning Q5 processes (i.e. regular minuted meetings, participation of student representatives, incorporation of feedback from students, external partners and external examiners’ reports, and notes on student progression). In such cases, the reports address key information needs in relation to quality assurance and identify what is working well and what is not. The Q5 reports are also effective in relation to DIT’s international provision. They surface issues that need to be addressed at College level e.g. resources, and in some cases, in relation to cohorts of students or themes e.g., first year assessment or student retention. Also, some Programme Committees have developed informal mechanisms to address weaknesses in relation to the student feedback process. In many instances, there is also a strong focus on receiving feedback from external partners and stakeholders. The Panel found that the Q5 process is a vital element of quality assurance. It is clear that some Programme Committees take the process very seriously, and use it to inform their teaching and learning and to continuously improve quality.

The Panel, however, saw evidence that the Q5 process is not applied consistently across all programmes. In a number of the cases that it examined, the Panel found that committee meetings had not been held; student representatives had not attended meetings; and that minutes of meetings were absent. Significantly, these omissions pertained to programmes where students had identified problems to the Panel. It is not clear that these inconsistencies are being either identified or addressed at College Board or Institute level.

Other issues brought to the Panel’s attention include the volume and timing of annual Q5 reports reviewed by College Board, which means that the College Board cannot always pick up the detail. The College Boards receive summaries of the Q5 reports from the relevant College sub-committee. The completion of the Q5 reports in October / November can create difficulties in acting on recommendations from external examiners which are time critical and in making changes to modules for the start of the new academic year. This can lead to some Schools introducing changes in advance of approval by Academic Council. This risks circumventing Academic Council, which may not receive accurate and timely accounts of practice. This risk could be addressed by devolving more responsibility for approving and acting upon the Q5 reports to the appropriate Colleges and Schools. This could also address any potential gaps in the current practice of providing summaries of these reports to

12

Academic Council. The Head of Learning Development, with administrative assistance could also help address problems arising from the volume of Q5s received by College Board annually.

A number of issues arise in relation to student feedback and the current structure and functioning of the Q6 processes. In general, both staff and students expressed dissatisfaction with the current process and have a strong desire for change. It was made clear to the Panel that students have lost faith in the process. The Panel, likewise, does not have full confidence in the Q6 processes. The problems concern:

‐ inconsistencies in implementation; ‐ lack of relevant information; ‐ ownership of student feedback; ‐ validity of information which is filtered through the Q6b form by lecturers; and ‐ the timing of the process.

The DIT urgently requires a new and better system which facilitates students to give timely, constructive feedback and one which is applied consistently. It has allowed the collection of student feedback to become an industrial relations issue and to stagnate, although the Panel heard during the site visit that the IR issue has been resolved. Nonetheless, the following issues continue:

o the Q6 process is not implemented in a consistent way within Schools, notwithstanding the industrial relations difficulties. The scope and depth of implementation varies. It is not clear to the Panel that Schools and Colleges have full oversight of the extent of implementation (e.g. this does not appear to be reported on to Schools);

o the relevance and utility of information sought in the Q6 process was questioned by students and staff. A number of staff indicated that a redesign of the Q6 form would make it more useful, perhaps with a focus on a combination of generic and tailor-made questions for each module. This may require greater attention to the kind of information that students can give and to the specific information needs of programmes and modules;

o student feedback on modules (Q6a) is collected at the end of the academic semester. This makes it too late for changes to be made for that cohort of students. The proximity of this to exams may also generate both caution and apathy in completing forms. Many staff and students indicated that receiving feedback earlier in the academic semester would be beneficial. Alternative timing would allow this feedback to have a formative function. It would also encourage further integration of teaching and learning (including styles) into quality assurance. In practice, some staff facilitate informal feedback from students through surveys, focus groups and meetings. This is

13

to be encouraged and appears to work well; however, it is instigated by individual lecturers, rather than a systematic approach; and

o the current operation of the Q6 process makes it personalised. The forms are handed out to, and collected by individual lecturers from, their students. Linked to this, given small class sizes, is the question of how truly anonymous this method is. The student body is concerned about this practice. Many academic staff stressed the value of personalised feedback on their individual teaching methods and styles. However, as this feedback remains with the individual lecturer and only a summary of it is presented to the Head of Department (in the Q6b form), there is a risk that quality becomes a personal issue; the teaching of a module is owned by the individual lecturer and does not become institutional in a way that it should. As a consequence, institutional responsibility for quality may be hampered. There is a concern that the summaries of student feedback leads to ‘sanitised’ information going forward to the College Board. The Institution needs good information in order to ensure the quality of its teaching and programmes.

In relation to programme development, the Panel finds evidence of a well-functioning process. There is a strong focus on the needs of the external community – employers, regulators, professional bodies and community. However, the overall quality and availability of information on the first destination of students and feedback from employers appeared dependent on the networking efforts of individual Schools and staff, rather than on systematic analysis.

Good progress has been made in involving students in quality assurance processes, together with a strong focus and concern for students at all levels within DIT. However, the Panel heard that of the 800 positions for class representatives, some 500 are filled. Only 150 have received training provided by the Students Union. The availability of training is welcomed, but needs to be made known to all student representatives. The Student Union needs to be better supported by the DIT in its efforts to appoint and train student representatives. The strength of student representation is important to student engagement in formal and informal quality assurance processes. The DIT could directly encourage greater engagement of students through the Programme Committees and through Student Services.

The Panel finds evidence which suggests that the ownership and oversight of the quality of inter-disciplinary programmes needs to be addressed at School, and possibly, College level. Difficulties arise in relation to overall assessment, programme coherence and the responsibility for modules on interdisciplinary programmes (e.g., some students expressed difficulty in identifying to whom issues arising on such modules should be addressed, and that Schools were unwilling to take ownership. This was particularly problematic in multi-site contexts). The Panel finds that the further development of inter-disciplinary programmes needs to go hand-in-hand with effective quality assurance processes.

14

The NFQ award-type descriptors and the Level indicators set the overall standards for the DIT’s awards. However, the Panel did not find evidence that the DIT is using NFQ award-type descriptors when developing new programmes; it relies on Level indicators alone. This raises the question that while a programme may well be at NFQ Level 9, how does the DIT assure itself that it is a Masters Degree or a Postgraduate Diploma? The Panel also saw limited evidence of the use of the NFQ non-major award-types, for example, minor, supplemental and special purpose awards. All of the awards currently titled “Continuing Professional Development Certificates” are not identifiable as NFQ awards. This means, from a quality assurance perspective that they cannot be readily treated as NFQ awards and that progression opportunities are not immediately clear to learners.

The regularity and use of staff surveys was discussed in a number of sessions with the Panel. There appears to be a strong desire to develop follow-up mechanisms and to communicate these to staff. The utility of operating these on a College or Campus-wide basis was raised for consideration in the site visit. The Panel finds much evidence of excellent relations between the DIT and its external stakeholders and partners. On the whole, it is very responsive to its community and other stakeholders. For example, some 14, 000 people interact with its outreach activities. Many are involved in the different facets of provision – programme design, course delivery and research. There was evidence from annual programme monitoring reports that such feedback is incorporated into quality improvements made by the relevant committees. The accreditation of programmes by professional bodies was found to have a strong positive impact on quality. Employers have positive views of the employability and quality of DIT graduates.

Commendations


o The widespread engagement of students in quality assurance processes and the provision of training for student representatives. The Panel suggests that the DIT builds on this by informing incoming student representatives of the availability of training, and by supplementing training with relevant induction on active engagement with committees and quality assurance structures.

o The breadth of community and stakeholder engagement and of access programmes. The Panel finds that the Institute’s policies and procedures for access, transfer and progression are clear and robust.

o The engagement of DIT students in the Institute’s outreach activities which benefit both the community and the students themselves via learning and credit achievement.

15

o The DIT’s community links programme which is a wonderful programme that makes a significant difference to the lives of its participants.

Recommendations


o Schools and Colleges must ensure that the Q5 process is implemented fully, consistently and to a high standard across the DIT. This is essential to maintaining the integrity of the quality assurance system.

o In view of the problems outlined in connection with the Q6 and Q5 processes, and knowing that a College Board / quality enhancement sub-committee oversees between 50 and 100 programme Q5’s annually, it seems this oversight could become somewhat superficial. The Panel recommends that the DIT considers making the Schools - rather than Colleges – the central node in its quality assurance system and to underpin this with enhanced accountability and transparency. This is the level which is closest to the academic and pedagogical concerns, and deals with much fewer programmes than the College. Colleges, supported by a Head of Learning Development, should oversee effective implementation, set broad priorities, identify key themes and support mainstreaming of best practice across the Schools. Schools should report annually on the operation of the Q5 process to the Colleges and the Academic Council should consider the findings of these reports annually.

o The Panel recommends that minor changes to programmes should not require approval by Academic Council and should instead be the responsibility of the relevant School Executive. This would allow the Q5 process to function as a prompt to action and allow action to be taken in a timely and efficient manner at the appropriate level. College Boards should receive reports of such changes and their implementation within the context of their responsibilities to oversee the effective and consistent operation of quality assurance procedures across Schools.

o The Panel recommends that, given its centrality to a well-functioning quality assurance system, deficiencies in the operation of the Q5 process should trigger a formal programme review by the relevant School, subject to the approval by the relevant College. This would help address the gap caused by the cessation of regular programme reviews and their incorporation into School reviews.

o The Panel acknowledges that industrial relations difficulties have existed in relation to the collection of student feedback through the Q6 forms. The Panel, however, finds it unacceptable, from a quality assurance perspective, that the individual lecturer owns feedback from the student. The Panel urges the DIT to address this issue as a matter of urgency as it can undermine the effectiveness of the Q5 process. The DIT needs to

16

involve students in finding a workable solution. The Panel strongly encourages academic staff to collect feedback either formally or informally on modules at an early stage and that individuals should not own the feedback. A possible model would entail feedback being collated on a programme rather than on a module basis, thus depersonalising the process. The DIT should also consider online approaches to collecting student feedback, and steps to allow for the collective assessment of feedback by staff in relation to programmes and modules.

o All student representatives should receive formal induction in the operation of quality assurance processes and relevant committees of which they are members. The DIT should, perhaps through Programme Chairs or Student Services, proactively support the recruitment of student representatives and ensure that they receive appropriate training. Consideration should be given to making their presence a requirement for Programme Committee meetings to be held. Student representatives could nominate delegates to facilitate this and take responsibility for ensuring that they are present for such meetings.

o The Panel strongly urges the DIT to utilise the NFQ award-type descriptors for all of

its programmes and awards, both major and non-major. It should also consistently use NFQ titling and credit conventions.

The Panel endorses the proposed actions set out in the DIT’s Self-Evaluation Report:

Further develop monitoring procedures for partnerships with other HEIs and with professional accreditation and recognition bodies and to maintain a single repository for all partnership agreements

Undertake a comprehensive review of feedback processes and procedures from the perspective of staff, students and external examiners

Undertake a comprehensive review of the QE handbook to achieve the right balance between programme validations and reviews, on the one hand, and reviews of Colleges and Schools on the other

Continue the development of structured PhD programmes Complete all School and College Reviews.

17

1.3: Assessment of students Context DIT’s policy on assessment is contained in the Teaching and Learning Assessment Strategy. The Institute encourages a diversity of methods and a shift away from dependence on final written examinations. It also encourages formative forms of assessment. An Assessment Handbook, produced by the Learning, Teaching and Technology Centre, provides comprehensive guidance on assessment. The regulation of assessment is detailed in two subsets of the Handbook for Academic Quality Enhancement: the General Assessment Regulations and the Regulations for Postgraduate Study by Research. The latter are administered by a single Graduate Research School, whose board is a sub-committee of Academic Council and are currently being revised. The criteria and procedures for the assessment and examination of programmes at NFQ Levels 9 and 10 are detailed in the Regulations for Postgraduate Study by Research. There are formal procedures for postgraduate research students who wish to transfer between registers, for example from the MPhil to the PhD register. In each case of transfer, an external examiner (who must be independent of the project, the supervisor(s) and the postgraduate student) is appointed. The student undertakes an oral examination which is attended by the external examiner, the supervisor(s) and the College Director who acts as chairperson. The external examiner assesses the quality and progress of the research work and the quality and feasibility of the proposed programme of work for the award sought. All PhD students are assessed in a similar manner mid-way through their programmes. The Regulations for Postgraduate Study by Research require that, before a student commences, proposed research projects are evaluated and approved by the School and the Research Ethics Committee. To maintain satisfactory progress, each student is required to undergo a rigorous annual evaluation. All Doctoral students undergo a confirmation examination including a viva voce with an external examiner, at least one year after the commencement of study. The final thesis is examined including a viva voce by both an internal and an external examiner. Each College appoints a chairperson for each viva voce examination. The DIT plans to introduce a “licence to chair” training programme in 2011. The Student Handbook, issued at the beginning of the year to each student for every programme outlines the assessment methods and weightings for each module in line with the Student Charter. The DIT undertakes to provide the results of assessment to students within four weeks. However, as acknowledged in the self-evaluation report, this deadline is not being met for a significant minority of students. All programmes are now fully modularised and assessed on a modular basis. Module information is captured online in CourseWise, DIT’s online catalogue of programmes and modules, which is now also beginning to capture assessment information.

18

A variety of assessment criteria and methods are used to ensure that the intended learning outcomes may be achieved and demonstrated by students. Lecturers are conscious to ensure that learning outcomes are measurable. Validation panels are responsible for ensuring that programme Level outcomes are at the right NFQ level and that module outcomes are aligned with these. The DIT operates policies on the recognition of prior learning (RPL) and has issued implementation guidelines. Where a record of prior learning and work-related information is required from applicants, the Europass CV model is beginning to be used as a general template, augmented by local information requirements.

Issues Arising

The shift to learning outcomes, on which there has been considerable progress, gives students a clear sense of the skills they should acquire and creates an in-built pressure for the alignment of learning outcomes with assessment. Following the launch of the National Framework of Qualifications (NFQ) in 2003, the Learning, Teaching and Technology Centre ran workshops on writing learning outcomes. It continues to provide support and feedback on their use, resulting in a continuous improvement. In general, the introduction of learning outcomes is understood and approached through modularisation. Modularisation has generated change in delivery and more codification of practice. In some cases, assessment practice has changed to avoid unnecessary duplication and over-assessment.

The introduction of modularisation has implications for assessment. Issues arising include managing the academic calendar, avoiding over-assessment and providing timely feedback to students. The introduction of a learning outcomes approach within the context of the National Framework of Qualifications requires the alignment of learning methods and assessment and the alignment between programme and module learning outcomes. There is widespread support amongst staff and students for the shift to learning outcomes and alignment of same with assessment. The Learning, Teaching and Technology Centre’s Assessment Handbook provides excellent guidance on this. What is less clear to the Panel is how alignment is monitored and implemented, in particular, how the relationship between assessment at the module level relates to the achievement of programme learning outcomes. The demonstration of alignment is not apparent from the relevant quality assurance processes such as annual programme monitoring reports and external examiners reports. Following requests during the site visit, one instance of such alignment was provided. This was excellent and concerned validation of a programme by a professional body. The Panel had difficulty obtaining such evidence for other programmes.

Students referred to differences in the accessibility and availability of detailed information on assessment, in particular through the Student Handbooks (which are designed for each programme). The degree of detail provided is variable. The publication of assessment criteria

19

for modules and programmes appears to be patchy. CourseWise, the DIT’s recently introduced online catalogue of modules and programmes, is very well regarded by students. A key weakness of it, however, is that it is not used by all staff and, in relation to assessment, that the depth of information varies. Students also reported inconsistencies in the provision of feedback on assessment.

There is evidence of the responsiveness of lecturing staff and tutors to re-scheduling assessments and the delivery of modules to better accommodate student needs and the building of knowledge and skill sets. This is achieved through formal and informal channels.

The critical role of external examiners in ensuring consistency and the achievement of standards was noted by many staff. However, in looking at a number of Q5 reports, the Panel found little evidence in the attached reports of External Examiners that they explicitly consider the alignment of assessment and learning outcomes in their work. It found inconsistencies in the availability of these reports in the Q5 process, and in the depth and quality of these reports. Conversely, it finds good practice of meetings between external examiners and students.

The Panel is impressed with the Institute’s RPL policies, guidance and supports; however it found that staff was not aware of how widely RPL is used. This raises a question of how RPL practice informs policy and quality assurance.

Commendations


o The Panel is extremely impressed with both the transfer examination process for research students and the confirmation exam for students on PhD programmes, and in particular, the incorporation of external examiners into these processes.

o The broad implementation of modularisation, which appears to be both successful and well-received by staff and students.

o The broad implementation of a learning outcomes approach.

o The breadth and quality of DIT’s RPL policies and supports

o The DIT’s continuing efforts to provide timely feedback on assessment to students (and to ensure consistency across programmes).

20

Recommendations


o The DIT is encouraged to continue to improve the quality and consistency of information on assessment in Student Handbooks and on CourseWise.

o The DIT must ensure the consistent alignment of assessment practice with learning outcomes. It could support this through sharing practice, and by devising templates and tools to assist staff. The significance of this is that the DIT must be able to know and demonstrate that programme learning outcomes have been achieved.

o The DIT should monitor and track the implementation of its RPL policies – including tracking students’ use of RPL, their progression and making data easily available to staff and to the quality assurance system.

o The DIT must address inconsistencies in the quality, depth and scope of External

Examiners’ reports and their oversight of the alignment between learning outcomes and assessment.

The Panel endorses the actions proposed in the DIT’s self-evaluation report:

Further develop diversified methods of assessment, taking account of

concerns around plagiarism, lack of resources, non-exam assessment, meaningful timely feedback, bottlenecks in submission dates and individual staff workloads

Consider Institute-wide implementation of assessment methods that support increased independent learning

Consider additional training and support required to ensure consistent application of General Assessment Regulations

Bring to an early resolution the review of the academic calendar with a view (in particular) to including appropriate interventions to avoid repeat assessments and/or the carry forward of failures into subsequent years

Improve Institute’s implementation of the strategy to provide appropriate, timely, and effective feedback for students in the area of assessment

Complete the review of the Regulations for Postgraduate Study by Research.

21

1.4: Quality assurance of teaching staff Context The DIT promotes the quality of its teaching staff through the Learning, Teaching and Technology Centre at a central level, and through the Heads of Learning Development, at College level. It offers a Postgraduate Diploma in Third-level Learning and Teaching which, since 2006, is mandatory for every new member of staff. It is increasingly undertaken by existing staff. In the last ten years, about 120 staff from DIT has undertaken the course, as well as a number of staff from other institutions. The DIT also provides induction for new academic staff, workshops for Programme Committees, dedicated courses for Heads of Schools and other specialist areas, as well as support for continuing professional development. In addition, the Performance Management and Development System (PMDS) is being rolled out across the Institution. Staff are actively encouraged to engage in development opportunities, including professional memberships, networking and achievement of postgraduate qualifications. Lecturers are made aware of workshops and opportunities to upskill. Colleges identify needs and work with the Learning, Teaching and Technology Centre to address them. The DIT supports annual Teaching Fellowships, introduced in 2009, for educational research projects on an individual or team basis. These are funded through the Dublin Higher Education Regional Alliance (DHREA) initiative, funded under the Strategic Innovation Fund. Excellence in teaching is encouraged through teaching awards, in place since 2002, and annual showcases of teaching, learning and assessment innovation. The latter provide for the dissemination of good practice.

Issues Arising

The Panel finds that there is a strong institutional commitment to teaching and learning. There is widespread support and respect for the work of the Learning, Teaching and Technology Centre amongst academic staff. The work of the Centre is having a strong positive impact on practice. The Centre has shifted over time from externally provided training to the use of local knowledge, expertise, peer learning and sharing practice and it also provides significant support for administrative and executive staff. This support means that learning from these areas, as in the case of study skills first developed for Library Services, can inform programme development. The management forum provides an important mechanism for shared learning. The Teaching Fellows are generally held to have a very positive impact on teaching and learning across the Institute.

The Panel finds good practice in relation to the supervision of postgraduate students. It recognises the DIT’s effort to ensure that good levels of supervision are attained, also in instances where staff does not have the relevant experience or qualifications. It welcomes the

22

proposal to introduce a ‘licence to supervise’ as of September 2011 for all research supervisors. The Panel, however, heard instances of excessive delays in the appointment of supervisors. Schools need to monitor practice and ensure consistency in timely appointment of supervisors.

The Panel finds that the Institute’s recruitment strategy is consistent with its strategic plan. Pedagogy and research are also recognised as being key criteria for appointments. Academic staff has a strong commitment to research-informed teaching. The Panel finds that this is restricted by the overall contract of employment obligations across the Institute of Technology sector, which require academic staff to have 18 hours contact time with students (to be increased to 20 hours under the public service [Croke Park] agreement 2010-1014).4 The Panel notes that this volume of contact time and the apparent lack of flexibility in implementing it are an obstacle to staff, particular new staff on assistant lecturer contracts, engaging in research, often at a critical juncture in their research careers. It may also hamper recruitment by DIT which competes with universities who are not subject to such conditions. The implementation of the Employment Control Framework (since 2008) is also impacting on overall staffing levels and recruitment/promotions. The introduction of the PMDS is an important means to enhance the quality of teaching and better matching resources to teaching needs. The Panel notes that this is being rolled out across the Institute.

Commendations


o The Panel recognises the strong institutional commitment to teaching and learning, as shown in its recruitment strategy, support for pedagogy and research, the work of the Learning, Teaching and Technology Centre, student feedback mechanisms and institutional recognition/awards for teaching and learning.

o The Learning, Teaching and Technology Centre is highly respected and has an extremely positive impact on the teaching and administrative environment. It is also highly responsive to the outcomes of quality assurance processes and encourages innovative practice.

o The mandatory requirement that all new members of teaching staff acquire the Postgraduate Diploma in Third-level Learning and Teaching.

4 The Public Service or "Croke Park" Agreement is a commitment by public servants and their managers to work together to change the way in which the Public Service does its business so that both its cost and the number of people working in the Public Service can fall significantly, while continuing to meet the need for services and improve the experience of service users . See http://implementationbody.gov.ie/

23

Recommendations


o There is a need for a co-ordinated strategy to address the teaching, learning and assessment needs of staff, particularly those who have not undertaken induction or completed the Postgraduate Diploma in Third-level Learning and Teaching. This is particularly important in the context of long tenure arrangements for staff.

o In a university-like institution such as the DIT, research activity and research competence is vitally important to the quality of academic staff and the education of students. The contact hours of staff are very high (also when seen from the perspective of student needs). The Panel recognises that it is neither practicable nor desirable for all academic staff to engage heavily in research. However, it is a major challenge to the DIT to ensure, under the current employment obligations, that all or the majority of academic staff relate actively to relevant scholarship and/or research that must inform their teaching. While the current contractual obligations are not the DIT’s responsibility, the Panel urges that they be modified by the relevant authorities to allow the Institution to strengthen its research capacity. The Panel encourages the DIT, meanwhile, to continue to work to find smart solutions to this challenge: for example, the rotation of academic tasks, smaller course-related development work, ‘meta-research’ on relevant research by staff in their own disciplines - and to encourage and support faculty with strong research potential to engage in research.

o The DIT should proceed with the introduction of the proposed “License to Supervise” qualification with rigorous criteria for admission to this status

o The roll-out of the PMDS to all teaching staff should be advanced as a matter of priority.

The Panel endorses the proposed actions set out in the DIT’s self-evaluation report:

Review quality assurance measures inherent to policies and procedures for staff selection and progression

Ensure that PMDS supports training and academic development that continuously improves the quality of teaching and learning

Target academic support for new lecturers and improve overall participation in Learning, Teaching and Technology Centre activities

Create additional measures to encourage excellence in teaching.

24

1.5: Learning resources and student support Context The DIT has undertaken a significant re-organisation of Student Services since 2006, in response to the EUA review, international best practice, national guidelines and the continuous review of internal services. A wide range of services are provided to students in general, as well as targeted services to particular student cohorts and induction for students for every year of their programmes. The Students Union works closely with student services across the Institute. In 2010, DIT established a Directorate of Student Services, which pulls a range of student-related services together in a more coherent way. Campus Life, which includes registration, induction and the DIT’s move to Grangegorman, is housed within Student Services. A quality review of Campus Life and the Library Services has been completed. A major outcome of the Campus Life review was the establishment of a sub-committee of Academic Council to look at the student experience. This committee involves staff from Student Services and the Learning, Teaching and Technology Centre. A second initiative is the ‘Excellence in Administration’ project which aims to overhaul student registration and academic information systems. The International Office provides an induction service for international students. They are also prioritised in terms of student accommodation. The Chaplaincy organises special events for international students and has engaged in multi-cultural awareness-raising for staff.

Issues Arising On the whole, the Panel finds the DIT provides strong learning resources and student supports. It has a strong student-centred focus. Overall, the Panel heard that the student experience in DIT is exceptionally good. There is an excellent relationship between students and staff, and a friendly and welcoming atmosphere. Based on feedback from employers and professional bodies, the Panel heard that DIT’s mission statement to produce rounded graduates is being met. The Students Union is also very effective. The DIT has taken steps to address the basic literacy and maths needs of incoming students through the Maths Centre located in the libraries.

There appears to be low awareness on the part of staff and students within the DIT of the multi-cultural and international make-up of the existing student body and any issues arising. Whilst this was not pursued by the Panel throughout the visit, there may be a need for the DIT to consider this further.

The Panel recognises the enormous physical constraints under which the DIT operates - 38 locations, six libraries and 120 computer labs. This involves huge logistical and cost issues,

25

including the lack of inter-operability between libraries and duplication of provision of key services. The provision of quality library services with diminishing resources is a challenge to the DIT. An advantage of operating six libraries is that students on the different campuses are close to a library facility and get to know staff very well. A disadvantage is that more staff is required to provide the same level of services. The Panel finds that the limited library opening hours is unacceptable, especially given the large number of part-time students in the DIT. The limited inter-operability of libraries creates difficulties, for example, in borrowing and returning materials, especially for students who are based in multiple sites.

The exclusion of the DIT from the IReL (Irish Research eLibrary initiative)5, largely funded by the HEA, but only fully open to the universities in Ireland, was brought to the attention of the Panel. The Panel considers that this exclusion is unfair and that access by the DIT to this is key to enhancing the quality of its research activity.

The appointment of a Director of Student Services and the integration of key functions within this Directorate are welcomed by the Panel as a positive development. There is widespread support amongst students for the Institute’s Counselling Services. In particular, the provision of medical services to students in DIT is highly commended. The mentoring system for first-year students is also well regarded. The DIT has made progress in developing one-stop shops (a recommendation from the 2006 EUA review) and the move to Grangegorman will have a significant impact on this. Overall, the Panel is impressed by the planning process in place for the move to Grangegorman, which is very advanced and thoughtful in its design. The building of quality enhancement into the overall design and approach is to be commended. It is clear that the DIT has thought through the student experience, links with the community and access in its planning and design for Grangegorman.

Within Student Services, enhancing the first-year student experience appears to be a priority. This arises from inter alia findings from quality assurance reviews and data on retention. A number of pilot initiatives are underway, including peer mentoring, and projects to enhance soft skills, as well as core skills within disciplines. Peer mentors (second or third year students trained by the Chaplaincy) have been identified as an excellent way of identifying students ‘adrift’ in first year. They also direct students to support services.

The Panel finds many examples of excellent work to address the first-year student experience in particular. In addition to the work undertaken by the Learning, Teaching and Technology Centre, the Panel heard that some course co-ordinators screen the learning styles of students at the beginning of first year and amend lecturing styles accordingly. The Panel were highly impressed with “Get Smart!” a programme for all first year undergraduates in the School of Hospitality Management and Tourism. It focuses on personal and professional development, and is designed to build student confidence, improve team skills, and increase students’ levels of engagement within the programme and with industry partners.

5 http://www.irelibrary.ie

26

The Panel finds that the issues identified in the EUA review, 2006, in relation to work placement remain unresolved. These include unpaid work placemen and the scheduling of unpaid work placement outside the academic calendar year. The DIT needs to improve institution-level monitoring of the operation of work placement and of its assessment to ensure consistency of approach and quality for learners. Whilst recognising the challenges inherent in managing work placement, they need to be addressed given the extent to which work-placement is embedded in programmes across the DIT.

Commendations


o It recognises the challenges posed to the DIT by its diverse student body and commends its commitment to students and on improving student supports, particularly to first-years and mature students.

o The work of the Counseling Services, and the Institute’s provision of medical services to students.

o The appointment of a Director for Student Services, and the integration of key

functions within the Directorate.

o The moves to create one-stop-shops for student services (and planning for this in relation to Grangegorman).

o The manner in which quality enhancement and the student experience has been embedded in the design, planning and approach to the new campus at Grangegorman.

o The on-going work on the first year experience. In particular, it commends initiatives, such as “Get Smart” and encourages the DIT to consider the role-out of similar initiatives across all Schools.

o The on-going work of some staff to screen student learning styles and amend lecturing styles accordingly. The Panel suggests that the Learning, Teaching and Technology Centre considers the possibility of providing tools and supports to students and staff in this regard.

27

Recommendations


o The Panel fully recognises the enormous logistical and cost constraints under which DIT operates. However, it needs to extend library opening hours and increase inter-operability across libraries.

o The Panel urges the DIT to implement the EUA report recommendations (2006) to introduce both a structured approach to managing work placements and a consistent approach to the assessment of learning associated with work placements.

o The Panel, whilst recognising that this is a matter primarily for the HEA, encourages the DIT to seek access to IREL and to use every instrument to address this.


Continue to take measures necessary to ensure learning spaces are fit for purpose, used efficiently and maintenance is improved where necessary.

Continue detailed planning for the new campus at Grangegorman Leverage on-line resources to enable flexible provision of programmes,

services, information and communication between staff and students Continue implementation of the widening participation strategy Improve student attendances where necessary Provide Diploma Supplements automatically Ensure alignment with international best practice for the recognition of

international qualifications.

28

1.6: Information systems Context

A revised management and governance framework for Information and Communications Technology (ICT) systems was introduced in 2010. An ICT strategy is due to be made available for internal discussion in June 2011. This is driven by Information Systems, Academic Affairs and Library Services staff. Eight major information systems operate across the DIT. The aim is to streamline and integrate them and to develop overall standards for them. The DIT recognises the need to integrate CourseWise (the catalogue of modules) and the Banner system (the student information system, common to all Institutes of Technology).

Issues Arising

The Panel recognises that the DIT has a very complex ICT network that is very expensive to maintain. It welcomes the Institution’s focus on integrating these systems and that the student lifecycle is at the centre of this, alongside standards development and streamlining services. The Panel recognises that there is some way to go to achieving these aims, but was impressed by the approach to developing an ICT systems strategy. The major challenges are to integrate the key information systems in DIT whilst at the same time managing them across multiple sites, and for the transition to Grangegorman. The multi-campus environment places a large burden of costs and technical support (including replacement, recovery and security strategies).

A key issue arising for quality assurance is the question of how the ICT systems underpin the quality assurance system. They are critical to tracking progress in the operation of quality assurance; specifically the management and communication of College action plans, the widespread population and use of CourseWise, and the accessibility of student data. A significant weakness of the current system is that certain data on student progression, programmes and cross-School/College data can only be retrieved manually and are not easily accessible to the key decision-making instances. The Panel considers that the ICT systems must serve core institutional needs in relation to student progression and retention; the employment of graduates, student and staff surveys and the provision of data to external bodies. The Panel notes that the current developmental work on the ICT strategy is not clearly informed by the needs of the DIT’s quality assurance system. Some staff indicated that the Electronic Grading System involves a higher degree of aggregation of results data than pre-existing manual systems. As this limits the scope of information on student performance, some academic staff also use manual grading sheets. The duplication of systems is inefficient and suggests that consideration should be given to enhancing the range of relevant data captured in the Electronic Grading System.

29

Commendations


o The continued management and operation of ICT systems in a very challenging

environment.

o The on-going development of an ICT strategy and its focus on the student lifecycle.

o The regular operation of surveys of students, staff and graduate employment.

Recommendations


o The new ICT strategy should take account of how information systems can support quality assurance.

o Whilst acknowledging the challenges to integrate ICT systems, the Panel recommends

that work be advanced to achieve compatibility between CourseWise and Banner.

o The DIT is encouraged to facilitate the greater automation of quality assurance processes (from input to retrieval) as a means of improving efficiency, managing workflows and records and in ensuring timely access to data and a fully informed profile of activity. This needs to be accessible to Programme Committees, Schools, Colleges, Academic Affairs and the Registrar, dedicated Quality Assurance personnel and Student Services.

o The DIT is also encouraged to develop the capacity of its ICT systems to provide

more automated student cohort analysis, for example, to measure and track student progression from entry to achievement of an award and make this easily available to the quality assurance system (in place of the manual extrapolation of data).

o The DIT is encouraged to consider enhancing the range of relevant data captured in the Electronic Grading System to meet academic needs.


Develop an integrated knowledge management strategy focussed on providing

information needed to inform decision making Integrate all information systems to enable relevant and on demand reports to

30

be generated by end users Student information system to be reconfigured to generate award

classifications as an automatic report Determine and provide additional functionality required to support leveraging

modularisation Develop and implement data definitions, policies, procedures and training

that ensure consistent accurate entry, management and integrity of data, both internally and from external bodies such as the Central Applications Office

Achieve ISO/IEC 20000 standard certification for Information Services.

31

1.7: Public information Context

Factual information regarding academic programmes is updated regularly and made available to the public in print and electronic formats. This information is aimed primarily at current and prospective students, and their advisers; partner institutions; funding bodies; policy makers, potential and existing employers of graduates; and other stakeholders. The quality and effectiveness of the information provided is under continuous review.

At the start of the academic year each student is provided with a Student Handbook, containing information on learning and teaching, content and assessment. CourseWise, the online catalogue of modules includes information on learning outcomes, content, assessment, credits and reading lists for each module. CourseWise is available to the public, with additional information available to registered students and staff. The DIT prospectus, published annually online and in hard copy, displays information on all validated full-time undergraduate and postgraduate programmes.

Quantitative reports on the admission, transfer and progression of students are returned annually to the Higher Education Authority. This information is made publicly available through various HEA reports and publications. Annual audited accounts and reports from external evaluations/reviews are publically available. Full text publications of scholarly research, including PhD theses are also available.

The DIT website underwent a major review in 2008. This addressed concerns regarding its user-friendliness. A new-look website was subsequently developed and training on its use was provided to staff.

Issues arising

The Panel finds that CourseWise is widely valued by students as a key information source. It is recognised by staff as a valuable tool in designing programmes. However, it needs to be fully populated. The DIT’s website has undergone improvement and work in this regard continues. Also, work is underway to develop an information system on research.

The Panel finds that information on the key aspects of the National Framework of Qualifications is not provided consistently in the public information on DIT programmes, qualifications and learning outcomes. In general, information on the relevant award-type and award-title, particularly in relation to continuing professional development and non-major awards, is either lacking or inconsistent with NFQ conventions. Such information is critical to informed student choice.

32

Commendations


o The availability of a number of School review reports on the DIT website or on the

intranet. The Panel encourages this practice as a particularly useful means of sharing good practice across Schools and Colleges, aiding transparency and enhancing quality.

Recommendations


o The Panel acknowledges that substantial work has been undertaken on the updating of the Institute website in recent years. However, some important omissions remain, in particular, information on programme learning outcomes is not regularly included as part of programme information. Summaries of external programme accreditations would also be a useful addition. The DIT may also wish to utilise the internal skills and resources at its disposal in the Schools of Marketing and Computing to enhance the website as a communications instrument.

o CourseWise, the module catalogue, has enormous potential, and needs to be fully populated. The Panel recommends that the DIT continue to develop CourseWise as the single repository for programme information, module descriptors, learning outcomes, assessment criteria and exam papers and to make it fully operational for all programmes and modules. The Panel also heard evidence that training for staff and students on inputting to, and navigating, CourseWise would be beneficial.


Continue to develop Coursewise as the single repository for programme

information, module descriptors and exam papers Archive superseded programme and module information Continue to review the website to ensure that site navigation and presentation

are designed form the users’ perspective, and provide enhanced information including links to quantitative reports on admission, transfer and progression of students

Develop and implement a single integrated research information system Continue to ensure that access and progression routes are communicated

clearly to potential applicants including arrangements for recognition of prior learning.

33

Section 2:

Followup to the EUA review of the effectiveness of DIT's quality assurance procedures (2005/06)

2.1 Context

The European University Association (EUA) undertook the first external review of the DIT’s quality assurance in 2005/06. This was conducted as an institutional review. In its report, the EUA team made 33 recommendations for follow-up.6 The current review includes an examination of the extent to which the DIT has implemented these recommendations. These were grouped under seven main headings: quality assurance and enhancement; administration; teaching and learning; student involvement and services; human resources; research and internationalisation. The Panel has examined these, where possible, in Section 1, as part of the DIT’s compliance with the ESG standards. Given that this review is of the effectiveness of the DIT’s quality assurance procedures, rather than an institutional review, the Panel focuses on those recommendations directly related to quality assurance. This Section first provides an overview of progress and second, addresses research and internationalisation which did not (readily) fall within the ESG standards in Section 1.

2.2 Overview

Many of the key recommendations concerning the effectiveness of quality assurance procedures have been acted upon and continue to be progressed, whilst others remain to be fully implemented. The latter include filling two College level positions of Heads of Learning Development, with adequate administrative support. These positions make a valuable contribution to quality enhancement and need to be acted upon as a matter of urgency. Additional recommendations that the Panel considers need to be progressed are continued work to develop the concept of one-stop-shops for student services, the operation of the Directorate for Student Services and the development of management and administrative capacity (notably College managers). The particular quality assurance dimensions of modularisation, including developing CourseWise, were addressed in Section 1 above. The Panel recognises that Schools and Colleges have a responsibility to exercise leadership in relation to the effective operation and enhancement of programme monitoring and of student feedback. It notes that little progress has been made in relation to developing structured approaches to work placement and to ensuring consistency in operating and assessing these. It strongly urges that the EUA recommendation be implemented as a matter of urgency (see Section 1.5). Further action is required to roll-out the PMDS to all staff. The contribution of the Learning, Teaching and Technology Centre, noted in Section 1.4, to enhancing quality is highly commended by the Panel.

6 Please see: http://www.nqai.ie/docs/awarding_bodies/dit/teamreport.pdf

34

2.3 Research

The Panel considered the theme of research from its view that, while not all staff needs to be research active, all teaching should be research informed. The Panel found the current teaching contract for staff in the Institutes of Technology to be problematic (see discussion in Section 1.4). The Institute’s goal to pursue a research-driven agenda is laudable. The Panel recognises that there are legacy issues in relation to achieving its research goals. They cannot be achieved when approximately one-third of academic staff is research active. It is not clear how the other two-thirds of staff are engaging in scholarship. Undertaking research in these circumstances is particularly difficult for young entrants. Additional barriers are posed by the differential treatment of research activity in the HEA’s funding models for the DIT and the universities (for example, in the teaching load required of full-time research students).

In relation to research students, the Panel is very impressed with the DIT’s overall arrangements for the supervision of PhD students; all supervisors must have a PhD or professional equivalent, while first-time supervisors are mentored, and many PhD students have more than one supervisor. PhDs must be aligned to institutional research priorities. The Panel is also impressed with Gradcam.ie, DIT’s framework for structured PhD programmes approved by Academic Council and which is informed by the National Framework of Qualifications and the EUA declaration on generic skills.

The Panel commends the support for Teaching Fellows and the role that they play in relation to pedagogy. It also commends DIT’s unique approach to Intellectual Property (IP) which gives researchers the right to own their IP where the research is not externally funded.

The Panel recommends that the DIT should identify and support key individuals in their research activity. It also needs to sharpen its research strategy and to put in place the necessary infrastructure to deliver this. For example, there needs to be consistency and greater transparency in relation to the allocation and management of research budgets; ‘time-off’ for PhD supervision, the recognition of research in promotions and the development and use of metrics to promote and reward research.

The Panel endorses the recommendations in the DIT’s self-evaluation report to: consolidate and collaborate to enhance research capability, capacity and quality in thematic areas; continue to develop modules for doctoral students as part of the implementation of structured PhD programmes, and complete the revision and implementation of postgraduate research regulations that will include a ‘Licence to Supervise’ and a ‘Licence to Chair’ viva voce examinations’.

2.4 Internationalisation

The Panel finds that the DIT’s development and oversight of international partnerships to deliver programmes and awards is embedded in its quality assurance system. International linkages enrich academic institutions and the DIT’s existing partnerships are well monitored

35

from a quality assurance perspective. The self-evaluation report proposes action to ensure that compliance requirements are fully communicated and met in partnership agreements. However, the Panel heard evidence that the partnerships with Harbin and Tanzania may not be sustainable (based on current staff/student ratios and fee income). Whilst recognising the DIT’s ambitions and the long-term potential of internationalisation, the DIT may wish to consider the cost-effectiveness of its current international provision. The Panel also notes the DIT’s objective to rapidly expand the number of international students in the DIT. The Panel encourages the DIT to be mindful of the impact of this on student services, English language provision and pastoral care. Furthermore, as mentioned earlier (Section 1.5), there is a need for the DIT to focus on the growing cultural diversity of the existing student body.

36

Section 3:

Overall findings and concluding comments

3.1 Overall findings

(i) The Panel finds that the quality assurance procedures of the DIT are effective and comply with the relevant Standards and Guidelines for Quality Assurance in the European Higher Education Area;

(ii) The Panel makes a number of commendations and recommendations to the DIT in relation to each of the seven European standards with a view to supporting the DIT in enhancing its quality assurance system in a consistent manner across the Institute. It also supports the main actions proposed by the DIT in its self-evaluation report to enhance quality; and

(iii) The Panel finds that many of the recommendations in the European University Association report, 2006, have been acted upon and continue to be progressed. However, some urgency is needed to implement fully key recommendations in support of quality enhancement: namely, the appointment of two Heads of Learning Development and developing a structured approach to work- placement and its assessment. The Panel makes recommendations and suggestions to the DIT in relation to research and internationalisation

3.2 Concluding Comments

The Panel commends the DIT as a quality-oriented Institution. It has a comprehensive set of procedures, a dedicated leadership and a strong commitment to maintaining and improving quality in a very challenging environment. It meets the Part 1 of the European Standards and Guidelines. Its quality assurance system is also informed by national guidelines and good practice. It has implemented most of the recommendations in the 2006 EUA institutional review. In addition, the Institute is clearly student-centred in its mission, practice and ethos. It is open and highly responsive to its external environment – the community, the labour market, employers and professional bodies. The DIT fulfils a valuable function and occupies a major position on the higher education spectrum in Ireland.

The Panel is aware of a number of key factors which, whilst falling outside the scope of this review of the effectiveness of its quality assurance procedures, are relevant to its quality assurance system. The ‘in-between’ status of the DIT as a university or institute of technology in the Irish higher education system means that it falls under the Department of Education and Skills and the Higher Education Authority’s funding, employment and

37

teaching controls in relation to Institutes of Technology. This appears to exclude the DIT from valuable collaborative research services provided to the universities. These factors impact on staff engagement in research and scholarship.

The DIT currently operates within tight budgetary constraints, strong external funding and employment controls. It is distinguished from the universities and the Institutes of Technology through specific legislation, the Dublin Institute of Technology Act 1992, which constituted the DIT. The complex physical environment of thirty-eight sites also poses extra challenges to the delivery of services and the operation of IT systems. Whilst these matters fall outside the scope of this review, the Panel considers that these particular features of the DIT impact on the Institute’s capacity to deliver and ensure high quality services in teaching and research.

The Panel notes the views of some external constituents (met by the Panel in the site visit) that the DIT has an image challenge The Panel feels that perhaps there is not a full appreciation of the multi-dimensional role that the DIT plays in the Irish higher education system and the constraints under which it operates under. This can have negative consequences for its attractiveness to students, and in particular it may impair its capacity to attract research funding and philanthropy.

Significant external constraints impinge on the development of the DIT’s research capacity and research-led teaching and learning. The overarching regulations governing contracts in the Institute of Technology sector impose a high teaching obligation on academic staff. In the Panel’s view, this, combined with the inflexible operation of the contract, is a significant barrier to the engagement by staff in research and scholarship. It is particularly acute for new and younger staff. While these constraints are imposed from outside, the Institute itself can do several things to alleviate them: it can develop a bolder research strategy which articulates the vision and goals for the research direction of the DIT, and prioritise disciplinary areas to be aggressively pursued. At the same time it may critically examine its teaching portfolio to see if there are redundancies and if more productive use could be made of student – teacher contact time. A creative move would be to encourage students to undertake more research and development projects as part of their study and contact hours with staff.

The Panel strongly encourages the DIT’s Academic Council to strengthen its role in providing overall direction and strategic leadership in relation to quality assurance and quality enhancement. Central leadership is crucial to sustaining an institution-level commitment to quality. The Panel also feels that the DIT faces a challenge in ensuring consistency in the implementation of its quality assurance policies and procedures across the Institute. It could benefit from a simpler, coherent analytical approach to quality. In particular it encourages the DIT to consider a stronger role for the Colleges and Schools, who are closer to programme delivery and to staff, in promoting a more analytical approach to quality and a discourse on quality. This delegation of authority would be underpinned by enhanced transparency and accountability.

38

Finally, the Panel stresses the importance of continued programme monitoring in terms of the implementation of the National Framework of Qualifications, the use and assessment of learning outcomes across modules and programme coherence.

Notwithstanding these challenges, the Panel concludes that the DIT is a quality-oriented institution, with a robust quality assurance system. It has a strong student-centred approach to its activities. Overall, the Panel finds staff to be extremely positive, expressing a genuine care for their students and a loyalty to their institution. At the same time, it is an institution that is aware of, and receptive to, feedback from its community and external partners. It prepares itself for change and adaptation and its commitment to enhance the quality of teaching and learning is widely embedded across the Institute.

39

APPENDICES

Appendix 1: Terms of Reference for the Review

Review by the National Qualifications Authority of Ireland (NQAI) of the effectiveness of the quality assurance procedures of the Dublin Institute of

Technology (DIT)

Terms of Reference (20 September 2010)

1. Background

The Dublin Institute of Technology was established as an autonomous institution under the DIT Act in 1992. In 1998, it was granted degree awarding powers by the Irish state, and makes major awards at Higher Certificate, Ordinary Bachelor Degree, Honours Bachelor Degree, Masters and PhD levels (Levels 6 to 10 in the National Framework of Qualifications), as well as a range of minor, special purpose and supplemental awards. The Institute also has a major involvement in and commitment to apprenticeship education and training. As a statutory awarding body and provider of higher education and training, and in line with national and international practice, the primary responsibility for the quality assurance of the educational provision and awards made by DIT resides with the Institute itself.

Side by side with the internal quality assurance procedures which the Institute operates, and again in line with national and international practice, DIT is also subject to external quality assurance, as set out in the Qualifications (Education and Training) Act, 1999. Section 39 (3) of the act provides that the NQAI shall consider the findings arising out of the application of the DIT's quality assurance procedures, as agreed with the Authority under section 39(1) of the Act, and may make recommendations to the Institute which the Institute shall implement. Under section 39 (4), the Institute is required to review the effectiveness of the Institute’s quality assurance procedures ‘not more than once in every three years and not less than once in every seven years’. Section 39 (5) also provides that the Authority shall publish the results of the review of effectiveness.

The first review of effectiveness of the DIT's quality assurance procedures was completed in 2006. The review was carried out by the European University Association on behalf of the NQAI, under the EUA's Institutional Evaluation Programme. The published outcomes of the review are available at: http://www.nqai.ie/award_dit_rev.html. This review process is the second review of effectiveness of the DIT's quality procedures, and is being instituted in fulfilment of the statutory obligations of the DIT and NQAI under section 39 (4).

40

It is of note that plans are advancing for the amalgamation of the NQAI with the Higher Education and Training Awards Council (HETAC), the Further Education and Training Awards Council (FETAC) and the Irish Universities Quality Board (IUQB). The new qualifications and quality assurance body will assume the NQAI's statutory external quality assurance role in relation to the DIT. It is possible that the new body may be established before this review process is completed, and that the review outcomes may be considered and published by the new body. In this connection, the NQAI will build into the review process a formal consultation, which will involve all of the quality assurance agencies involved in the amalgamation, namely, HETAC, FETAC and the Irish Universities Quality Board (IUQB).

2. Review Objectives

The objectives of the review will be to evaluate:

• the effectiveness of the procedures operated by DIT for the quality assurance of its education and training programmes, and the awards to which they lead in the National Framework of Qualifications, both nationally and internationally

• the extent to which DIT, in operating these procedures meets the requisite standards, as set out in Part 1 of the Standards and Guidelines for Quality Assurance in the European Higher Education Area

3. Basis of Review Criteria and Procedures

To meet the review objectives, the DIT will be evaluated and assessed against criteria and procedures that are informed by institutional review practice within the Irish higher education system, and by the standards and guidelines for quality assurance agreed by the Ministers of the Bologna signatory states. Specifically, the criteria and procedures are informed by the following documents:

• Higher Education and Training Awards Council (HETAC): Policy on Institutional Review of Providers of Higher Education and Training (2007); Supplemental Guidelines for Institutional Review (2008); Supplementary Guidelines for the Review of Effectiveness of Quality Assurance Procedures (2008)7

7 http://www.hetac.ie/publications.cfm?sID=44

41

• Irish Universities Quality Board (IUQB): Institutional Review of Universities (IRIU) Handbook (2009); Good Practice in the Organisation of PhD Programmes in Irish Higher Education (2009) 8

• Irish Higher Education Quality Network: Principles of Good Practice in Quality Assurance/Quality Improvement for Higher Education and Training (2005); Principles for Reviewing the Effectiveness of Quality Assurance Procedures in Irish Higher Education and Training (2007); Provision of Education to International Students: Code of Practice and Guidelines for Irish Higher Education Institutions (2009) 9

• European Association for Quality Assurance in Higher Education (ENQA): Standards and Guidelines for Quality Assurance in the European Higher Education Area (3rd edn., 2009) 10

• The UNESCO/OECD Guidelines for Quality Provision in Cross-border Higher Education11

4. Review Criteria

The following are the criteria and procedures for the review of effectiveness of the quality assurance procedures of the DIT. These criteria will apply, where appropriate, to the Institute’s educational provision both in Ireland and overseas.

4.1 Follow-up to the 2006 review of effectiveness of DIT's quality assurance procedures

The criterion is intended to assist the examination of DIT's internal quality culture. Specifically, the review will evaluate the extent to which the DIT has implemented the 33 strategically linked recommendations made by the EUA review team in 2006. Particular attention will be given to the five recommendations made by the EUA team on quality assurance and enhancement, and to the quality issues which the NQAI commented upon in its consideration of the review outcomes, namely:

• the need to implement more proactively and to review the annual programme reporting processes in order to encourage full participation by both students

8 http://www.iuqb.ie/info/iriu.aspx; http://www.iuqb.ie/info/good_practice_guides.aspx

9 http://www.iheqn.ie/publications/default.asp?NCID=154

10 http://www.enqa.eu/files/ESG_3edition%20(2).pdf

11 http://www.oecd.org/dataoecd/27/51/35779480.pdf

42

and staff, and in order to ensure greater transparency in the student feedback process

• the need to accelerate the development of the emerging systems for school, faculty and research review.

4.2 Education and training programmes offered by the DIT

The criteria set out here are intended to assist in the examination of DIT's processes for the quality assurance of the programmes of education and training, including research programmes, which it provides. In line with practice in the Irish higher education sector generally, and Ireland’s commitment to the Bologna Process, the criteria used here are the standards from Part 1 of the European standards and guidelines for internal quality assurance within higher education institutions. These criteria should be considered in conjunction with the accompanying guidelines as set out in Standards and Guidelines for Quality Assurance in the European Higher Education Area (3rd edn. 2009), pp. 16-19.12 The guidelines provide additional information about good practice and in some cases explain in more detail the meaning and importance of the standards.

• Policy and procedures for quality assurance

Institutions should have a policy and associated procedures for the assurance of the quality and standards of their programmes and awards. They should also commit themselves explicitly to the development of a culture which recognises the importance of quality, and quality assurance, in their work. To achieve this, institutions should develop and implement a strategy for the continuous enhancement of quality. The strategy, policy and procedures should have a formal status and be publicly available. They should also include a role for students and other stakeholders. • Approval, monitoring and periodic review of programmes and awards

Institutions should have formal mechanisms for the approval, periodic review and monitoring of their programmes and awards. In relation to this criterion, an assessment will also be undertaken with regard to how these formal mechanisms incorporate the implementation of the National Framework of Qualifications and the relevant parts of the NQAI's policies and procedures on access, transfer and progression.13 Areas to be examined here will include the processes used by the Institute to ensure the accurate levelling of awards, the extent and the manner in which NFQ award-types (major, minor, special purpose and supplemental) are utilised, and the manner in which credit arrangements and transfer and progression routes are embedded in the programme development processes.

12 http://www.enqa.eu/files/ESG_3edition%20(2).pdf

13 http://www.nqai.ie/publication_oct2003a.html

43

• Assessment of students

Students should be assessed using published criteria, regulations and procedures which are applied consistently.

• Quality assurance of teaching staff

Institutions should have ways of satisfying themselves that staff involved with the teaching of students are qualified and competent to do so. They should be available to those undertaking external reviews, and commented upon in reports. • Learning resources and student support

Institutions should ensure that the resources available for the support of student learning are adequate and appropriate for each programme offered. • Information systems

Institutions should ensure that they collect, analyse and use relevant information for the effective management of their programmes of study and other activities. In this connection, consideration should also be given to the Institute's processes for monitoring and analysing grade distribution and profile patterns across its education and training programmes.

• Public information

Institutions should regularly publish up-to-date, impartial and objective information, both quantitative and qualitative, about the programmes and awards they are offering including the award-type to which the programme leads, the level at which it is included in the NFQ and the credit volume associated with the programme. In this connection, consideration should also be given to the information provision requirements on providers as set out in the NQAI’s policies and procedures on access, transfer and progression.

5. The Review Process

In line with best national and international practice, the review process will consist of the following elements:

• an institutional self-assessment report to be prepared by the DIT addressing the agreed criteria and terms of reference

• an expert review panel will be appointed comprising national and international representation to conduct the review process

44

• a review of the DIT self-assessment report by the expert panel and consideration by the panel of any other information they might consider relevant

• visit by expert panel appointed by the NQAI

• preparation of a review report by expert panel for submission to the NQAI, which will include findings and a recommendations in relation to the effectiveness of the DIT's quality procedures, and the extent of its compliance to Part 1 of the European Standards and Guidelines

• preparation of an institutional response, including a quality improvement plan, by the DIT

• consideration of review report and institutional response by the NQAI

• publication by NQAI of the review outcomes.

It is of note, and in consideration of the impending amalgamation of the qualifications and quality assurance agencies, that at all appropriate stages of the effectiveness review the NQAI may consult and seek advice from an advisory group comprising representatives of FETAC, HETAC, and the IUQB. The executive of the NQAI will provide secretarial support to the expert panel, including drafting of the expert panel’s report.

6. Indicative Timetable for Review

DIT commences self-evaluation report against agreed criteria September 2010

Appointment of expert review panel by NQAI Autumn 2010

Preparation of site visit schedule and indicative timetable by NQAI

Autumn 2010

Briefing for expert review panel Spring 2011

DIT self-evaluation report completed and submitted to NQAI for circulation to expert review panel

February 2011

Expert review panel site visit March 2011

Expert review panel report completed April 2011

DIT submits response to expert review panel report to NQAI/new qualifications and quality assurance body

May 2011

NQAI/new qualifications and quality assurance body considers and publishes review outcomes

Summer 2011

45

Appendix 2: Members of the External Review Panel

Professor Jon Haakstad – Chair of Review Panel

Senior adviser to the Norwegian Agency for Quality Assurance in Education (NOKUT)

Jon Haakstaad is senior adviser to the Norwegian Agency for Quality Assurance in Education (NOKUT). He has extensive national and international experience in the area of quality assurance of both agencies and institutions. Within NOKUT, Jon was Director of Institutional Audits (2003-2008) and Director of Analysis and Development (2008-2010).

Jon has chaired three ENQA-directed evaluations of the Austrian Fachhochschulrat (2007), the Institutional Evaluation Programme of the European Universities Association (2009)) and the Finnish Evaluation Agency (FINHEEC, 2010). He has also been a chair/member of evaluation panels, and member of commissions and advisory councils for quality assurance in the field of higher education [Switzerland, Ireland (Waterford Institute of Technology, 2003), Sweden, Austria, Denmark, Nordic Network of Quality Assurance Agencies]. Jon was a member of expert team for the self-certification of the Danish Qualifications Framework for Higher Education (2009). He has extensive experience of quality assurance of universities in Norway and has several national and international publications on quality assurance issues in higher education.

Prior to working in NOKUT, Jon was an Associate Professor of English, Tromsø College of Teacher Education (1974-93), Rector of Tromsø College of Teacher Education (1993-94), Faculty Director, Tromsø University College (1994-96); Office Manager, National Council of Teacher Education (1996-98) and Senior Adviser, Network Norway Council (1998-2002).

Mr. Gordon Clark Former Head of DG Education and Culture at the European Commission Gordon Clark was, between 1999 and October 2010, responsible in the DG Education and Culture of the European Commission for the EU’s lifelong learning policy and strategic framework for cooperation supporting national reforms in education and training. This included the development and management of mutual learning between countries/stakeholders, common European approaches, guidelines and reference instruments. In particular, he was responsible for the development of the European Qualifications Framework for lifelong learning, which promotes mutual trust and transparency and is implemented through national qualification frameworks, being developed in all EU countries based on learning outcomes and quality assurance. He also developed and managed, until 2005, the Copenhagen Process enhancing European cooperation in and the attractiveness/quality of Vocational Education and Training.

46

Professor Roger Downer

President Emeritus, University of Limerick

Professor Roger G. H. Downer is President Emeritus, University of Limerick and has served as Vice-President, University of Waterloo (1989-96), President, Asian Institute of Technology (996-98) and President, University of Limerick (1998-2006). He has been elected to Fellowship of the Royal Society of Canada and Membership of the Royal Irish Academy. He serves on a number of corporate and private boards and undertakes occasional consulting assignments.

Mr. Shane Kelly

Academic Programme Developer with Dorset College, Dublin

Former President of Union of Students in Ireland (USI)

Shane Kelly is the Academic Programme Developer with Dorset College, Dublin, and has responsibility for developing and implementing robust QA processes. He holds an Undergraduate Degree in Business and a Master’s Degree in Political Communication. A graduate of Waterford Institute of Technology, he is a former president of the WIT Students’ Union and the Union of Students in Ireland. Shane is also a former board member of the Higher Education Authority (HEA), the National Qualifications Authority of Ireland (NQAI) and former member of the board of directors of the Irish University Quality Board (IUQB). Shane was also a member of the National Strategy for Higher Education established by the then Minister for Education and Science, Batt O’ Keeffe, and reported to cabinet in September 2010. Shane has been a member of many review panels, but in Ireland and across Europe.

Professor Andrea Nolan

Senior Vice-Principal and Deputy Vice-Chancellor, University of Glasgow

Andrea Nolan graduated as a veterinary surgeon from Trinity College Dublin and after a short time in veterinary practice, embarked on an academic career in the Universities of Cambridge, Bristol and the Technical University, Munich, leading to her appointment as a Lecturer at the University of Glasgow in 1989. She was appointed Professor of Veterinary Pharmacology in 1998 and took up the post of Dean of the Faculty of Veterinary Medicine a year later. She was appointed to the post of Vice-Principal for Learning & Teaching in the

47

University of Glasgow in 2004, and in 2007 incorporated internationalisation into her portfolio. Most recently she was appointed Senior Vice-Principal maintaining responsibility for internationalisation. Her academic career has been developed in the discipline of veterinary pharmacology, with a particular interest in the assessment and management of pain in animals. She is a Fellow of the Royal Society of Edinburgh.

Ms. Anna Pringle

Group Strategic HR Director, NTR Plc

Anna joined NTR plc in June 2008. Prior to joining NTR, Anna was Head of People & Organisation Capability for Microsoft International and HR Director for Microsoft Ireland. Anna has also held senior HR and leadership development roles with Bank of America in the U.S., most recently as head of HR for the Consumer Products Division, based in Charlotte, NC. Anna began her career with a New York City HR consulting firm, specialising in organisation development and leadership. She holds a B.A. in Communications from Dublin City University and an M.Sc. in Human Resource Management from the Centre for Labour Market Studies at University of Leicester

48

Appendix 3: Schedule for the site visit to the DIT, 29 March – 1 April 2011

Review by the National Qualifications Authority of Ireland (NQAI) of the effectiveness of the quality assurance procedures

of the Dublin Institute of Technology (DIT): Site Visit

Tuesday 29th March to Friday 1st April 2011 FINAL SCHEDULE (prepared by DIT)

49

Day 1: Tuesday 29th March 2011 Time Attendees Purpose Venue Name 8.30am

Review Team Meet in Staff Room DIT Aungier Street

Staff Room DIT Aungier Street

9.00am

Review Team arrive at DIT

Private time for review team

Boardroom DIT Aungier Street

9.15am

Review Team and Director of Academic Affairs & Registrar and Head of QA & Academic Programme Records

Introductory Meeting with Director of Academic Affairs & Registrar discussion of logistical issues and arrangements for the review visit


Dr Michael Mulvey Mr Bill Grimson

9.30am Review Team & President

Meeting with President DIT


Professor Brian Norton

10.00am Review Team Coffee Break: Private time for review team


10.15am

Parallel Session 1 Review Team & members of a Programme Committee

DT001 BSc Product Design


Mr Michael Ring Mr Colm O’Kane Dr Ray Ekins Mr Robert Tully Mr Kieran Corcoran Dr Daire Hooper Ms Rachael Gibbons Mr Willie Bergin Ms Laura Cuddihy

Time Attendees Purpose Venue Name 10.15am

Parallel Session 2 Review Team & members of a Programme Committee

DT341 BSc Marketing

Management House DIT Aungier Street

Ms Kate Ui Ghallachoir Dr Brendan O’Rourke Mr Donnacha Ryan Ms Mary Lawlor Mr Gerry Mortimer Ms Avril Murphy

11.30am Review Team Private Time for Boardroom DIT Aungier

50

Review Team Street

11.45am Review Team & Head of Library Services, Modularisation Co‐ordinator, Head of Campus Life, Head of Quality Assurance and Academic Programme Records and RPL Officer

Developments to support programme delivery


Dr Philip Cohen Dr Kevin O’Rourke Mr Brian Gormley Mr Bill Grimson Dr Anne Murphy Dr Brian Bowe

12.15pm Review Team and members of the Steering Committee

Discussion of the Institutional approach to self assessment – the process employed and the inclusive nature of the exercise, including a discussion of the Institute response to the first EUA report


Ms Maev Maguire Dr Mary McNamara Ms Phil Kenna Mr Stephen Best Mr Paul O’Sullivan Ms Clare Cullen Ms Jan Cairns Ms Margaret Kilmartin

1.00pm Review Team LUNCH


Time Attendees Purpose Venue Name 2.00pm Review Team

and members of the Academic Quality Assurance Committee

Discussion on the Institute’s arrangements for ensuring quality assurance activities operate in line with national, European and International best practice and Discussion on how findings and


Mr Bill Grimson Ms Nicole O’Neill Ms Kate Ui Ghallachoir Ms Clare Cullen Mr Paul Bourke Mr Mark McGrath Ms Maria Jose Gonzalez

51

recommendations arising from quality assurance are addressed

2.45pm Review Team and members of Review Group for the Quality Enhancement Handbook

Discussion on the Review of the Institutes Handbook for Academic Quality Enhancement


Dr Brian Bowe Mr Bill Grimson Ms Jan Cairns Dr Kevin Griffin Mr Serge Basini Mr Frank Prendergast Ms Sharon Feeney

3.30pm Review Team, Head of Campus Life, Research and Projects Officer, Retention Officer, and Head of Admissions

Discussion on surveys


Mr Brian Gormley Dr Rachel O’Connor Mr Mark Russell Mr Frank Costello Dr Dominic Dillane Dr Pamela Sharkey Scott

4.15pm Review Team Coffee Break: Private time for review team


4.30pm Parallel

Session 1 Review team and Undergraduate Students, Student Reps and Student Union Officials

Meeting with undergraduates to discuss the effectiveness of the mechanisms used for engaging students in decision‐making and quality assurance and enhancement procedures


Clare Cullen Mark Denihan Simon Donnelly Shauna Switzer Ross Boyce David Rice Glenn Fitzpatrick Jackie Molloy Niamh Moriarty Michael Bolger Avril Murphy Paul Breen

4.30pm Parallel Session 2 Review team and Postgraduate Taught Students and CPD Students,

Meeting with postgraduate taught students to discuss the effectiveness of the mechanisms used for engaging

Management House Aungier Street

Ciaran Nevin Eoin O’Broin David Lacken Frances O’Kelly John Mack Peter Clarke Brian Lanagan Louise Carpenter

52

Student Reps and Student Unions Official

students in decision‐making and quality assurance and enhancement procedures

5.30pm Review Team Private discussion to exchange first impressions


6.00pm Review Team and President and Chair of Governing Body

Discussion of the mechanisms employed by Governing Body for quality assurance and enhancement


Professor Brian Norton Mr Sean Dorgan

6.45pm Review Team Private Meeting of Panel to close out Day 1


Day 2: Wednesday 30th March 2011 Time Attendees Purpose Venue Name 8.15am Review Team Meet in Staff Room

DIT Aungier Street Staff Room DIT Aungier Street

8.30am Review Team Private Discussion Boardroom DIT Aungier Street

9.00am Review Team Taxis to DIT Grangegorman

Travel to DIT Grangegorman

Mr Bill Grimson

9.30am Chair & DIT Co‐ordinator

Brief meeting with the DIT co‐ordinator to clarify and pick up any issues arising from Day 1 that might impact on the schedule

Room 9 Ground Floor Grangegorman

Mr Bill Grimson

9.40am Review Team & Learning, Teaching and Technology Team, Staff Development Officer, Heads of Learning Development

Meeting to discuss the effectiveness of quality enhancement activities in learning, teaching and assessment in light of international best practice


Dr Jen Harvey Ms Phil Kenna Dr Brian Bowe Mr Paul Bourke Dr Kevin O’Rourke Ms Martina Crehan Ms Sharon Feeney

10.30am Review Team Tea & Coffee & Room 9 Learning and Teaching

53

Learning and Teaching Fellows

Poster Presentations from recent Learning and Teaching Showcase

Ground Floor Grangegorman

fellows

11.00am Review Team, Buildings Office/Property & Facilities, Campus Planning

Virtual Tour of DIT Facilities and plans for Grangegorman


Dr Noel O’Connor Mr Peter Heaslip Dr Paul Horan

Time Attendees Purpose Venue Name 11.50am Review Team Taxi to DIT

Aungier Street

12.20pm Review Team Lunch Staff Room DIT Aungier Street

1.20pm Review Team, College Directors and Heads of School

Discussion of the effectiveness of quality enhancement activities


Ms Brid Grant Mr Paul O’Sullivan Professor Mick Devereux Dr Mike Murphy Dr Deirdre Lillis Mr John Lawlor Dr Sheila Flanagan Dr Katrina Lawlor

2.20pm Review Team and Programme Chairs

Discussion of the effectiveness of quality enhancement activities in Schools


Dr Helen Lambkin Dr Christine O’Connor Mr Cormac Allen Ms Lejla Rovcanin Dr Carmel Gallagher Ms Anna Cruickshank Dr Eddie Rohan Mr Declan Doran Ms Marie O’Flynn

3.20pm Review Team & Teaching Staff Unions and Academic Staff

Meeting to discuss the effectiveness of quality enhancement activities and discussion on PMDS and student feedback


Mr Domhnall Sheridan Mr Michael Carr Dr Fergus Ryan Mr Anselm Griffin Dr Claire McDonnell Mr James Wrynn Ms Sharon Feeney

4.10pm Review Team Coffee Break: Private time for review


54

team

4.20pm Review Team & College Administration Staff

Meeting to discuss the effectiveness of quality enhancement activites


Ms Andrea Marcelin Ms Lisa Dooley Ms Elizabeth Heffernan Ms June Phelan Ms Nicola Shortt Mr Darren Myler Ms Jennifer Farrell Ms Teresa Ward

5.00pm Parallel Session 1 – External Stakeholders

Meeting with representatives from the FAS, EI and Professional Bodies to discuss the relevance of DIT offerings and how DIT is meeting its national, regional and social commitments


Tony Power John Williams Martin Hynes Dipti Pandya Martin Lyes Tony Lenehan Fergal Costello Ronan O’Loughlin Mike McDonnell

5.00pm Parallel Session 2 – External Stakeholders and Head of Community Links

Meeting with external industry representatives to discuss the relevance of DIT offerings and how DIT is meeting its national, regional and social commitments


Professor Tommy Cooke Jim Dunne Peppe Santoro Paul Carty John Graby Gerard Murphy Michael Mulryan Natasha Kinsella Jim Gahan Martin McEvoy Eileen Banks Aileen O’Donohoe Tony Devlin Gerry Doyle

6.00pm Review Team Private Meeting of the Panel


55

Day 3: Thursday 31st March 2011

Time Attendees Purpose Venue Name

8.30am Review Team Breakfast: Meeting of the team to prepare for day 3 of the visit


9.00am Chair & DIT Co‐ordinator

Brief meeting with the DIT co‐ordinator to clarify and pick up any issues arising from Day 2 that might impact on the schedule

Telematics Room DIT Aungier Street

Mr Bill Grimson

9.15am Review Team Heads of International Partner Colleges and Head of International Affairs

Discussion with those responsible for the delivering of DIT accreditated programmes in overseas Institutions.


Dr Robert Flood Mr Gerry Creaner Professor Wang Ling

10.00am Review Team Private time for review team


10.10am Review Team, Director of Student Services, Campus Life Staff, Library Staff, Director of HR & Staff, Director of Finance and Resources,

Discussion with those responsible for quality enhancement activities in administrative and service areas together with administrative support in areas that have recently been reviewed


Dr Noel O’Connor Mr David Cagney Mr Paul Flynn Ms Mary Malone Library Staff Ms Anne Ambrose Mr Brian Gillespie Dr Philip Cohen Campus Life Staff Mr Dave Kilmartin Dr Susan Lindsay

10.50am Review Team Coffee Break:

Private time Staff Room DIT Aungier Street

56

for review team

11.00am Review team & Director of Research and Enterprise, staff from Directorate of Research and Enterprise, Heads of College Research

Discussion of the effectiveness of quality enhancement procedures for research


Professor Ellen Hazelkorn Dr John Donovan Ms Jean Cahill Mr Tom Flanagan Dr Mary McNamara Dr Marek Rebow Dr Brian O’Neill Mr Paul O’Reilly

12.00pm Review Team, Institute Researchers and Supervisors



Dr Sarah Jane Delany Dr Mary Lennon Professor Vincent Toal Dr Nissreen Abu‐GhannamDr Brian MacNamee Dr Amr Arisha Dr Joseph Coughlan Mr Max Ammann Dr Aidan Duffy

1.00pm Review team Lunch Private Discussion

Staff Room

2.00pm Review Team, Postgraduate and Doctoral Students



Esther Tippman Amr Mahfouz Viacheslav Filonenko Jackie Bourke Yupeng Liu Zlata Tofzikovskaya Mark Wylie Colm Carey

3.00pm Review Team Private time for review team


3.15pm Review Team & Director of Student Services, Director of Finance and Resources, Chief Information Officer, Manager of IS

Discussion on how effectiveness of information systems to inform programme management and other quality enhancement activities


Dr Noel O’Connor Mr Paul Flynn Mr David Scott Mr Paul Reardon Ms Gillian Donagher Ms Margaret Kilmartin

4.10pm Review Team Coffee Break: Private Time for Review Team


57

4.20pm Parallel Session 1 – Review Team, College Board Arts & Tourism

Discussion of the effectiveness of quality enhancement procedures and the implementation of the recommendations from recent School reviews


Ms Gráinne O’Rourke Mr John O’Connor Dr Clíona Doris Dr Brian O’Neill Mr PádhraicÓ Cuinneagáin Ms Ann Wrigley

4.20pm Parallel Session 2 – Review Team College Board Sciences & Health

Discussion of the effectiveness of quality enhancement procedures and the implementation of the recommendations from School reviews


Professor Mick Devereux Mr Paul Bourke Dr Siobhan Daly Ms Yvonne Donlan Dr Denis O’Shea Ms Mary Scally

5.00pm Review Team Private time for review team


5.10pm Parallel Session 1 – Review Team College Board Business



Mr John Jameson Mr Serge Basini Ms Teresa Hurley Dr Eddie Rohan Dr Mary Prendergast Dr Henry McClave

5.10pm Parallel Session 2 – Review Team College Board Engineering and the Built Environment



Dr Brian Bowe Dr Audrey Martin Ms Anna Reid Mr Tom Dunne Dr Michael Conlon Dr Aidan O’Dwyer

5.50pm Review Panel Private Meeting of the Panel to discuss


58

Day 4: Friday, 1st April 2011 Time Attendees Purpose Venue Names 8.30am Review Team Breakfast

meeting of the team to prepare for the final day of the visit


9.00am Chair & DIT Visit co‐ordinator

Brief meeting with the co‐ordinator


Mr Bill Grimson

9.15am Review Team Meeting to draft the report of the Panel


12.00pm Review Team and Senior Leadership Team and NQAI

Close out Session


Professor Brian Norton Dr Michael Mulvey Dr Noel O’Connor Mr David Cagney Professor Ellen Hazelkorn Mr Paul Flynn Ms Brid Grant Mr Paul O’Sullivan Dr Mike Murphy Professor Mick Devereux Mr Ciaran Nevin Dr. Padraig Walsh (NQAI) Dr. Jim Murray (NQAI) Ms. Trish O’Brien (NQAI)

preliminary findings

59

Appendix 4: External stakeholders met by the review panel as part of the site visit (submitted by DIT, 29th March 2011)

Tony Power , Director of Apprentices, FAS

Martin Hynes, CEO, IRCSET

Dr Dipti Pandya, CEO, IRCHSS

Dr Martin Lyes, Divisional Manager, Enterprise Ireland

Dr Tony Lenehan, Head of Industry Standards & Quality, Failte Ireland

John Williams, Director, Academy of Medical Laboratory Sciences

Fergal Costello, Head of IoT Funding, HEA

Ronan O’Loughlin, Director of Education & Training, Chartered Accountants Ireland

Mike McDonnell , Director, Chartered Institute of Personnel & Development (CIPD)

Professor Tommy Cooke, Head of Community Links, Student Services

Jim Dunne, Chair, Design 21st Century

Peppe Santoro, Partner, Eversheds O’Donnell Sweeney

Paul Carty, Managing Director, Guinness Storehouse

John Graby, Director, Royal Institute of the Architects of Ireland

Michael Mulryan, Director, Brendan Merry & Partners

Dr Gerard Murphy, Director Quality Control, Elan Pharmaceuticals

Natasha Kinsella, CEO, Irish Hospitality Institute

Jim Gahan, Chair (Education Board), Society of Chartered Surveyors

Martin McEvoy, Director, Signia International Dublin

Tony Devlin, Competence & Capability Dev Mgr., Ericsson, Europe

Gerry Doyle, Training Prog Director, ICT Irl (IBEC)

Eileen Banks, Press, PR & Communications Officer, Enterprise Ireland

Aileen O’Donohoe, Director of Strategy, Policy & Comm., Irish Stock Exchange

60

Documents

DIT review report of the external review panel150611FINAL Institutional Review... · The Dublin Institute of Technology (DIT) is one of the oldest higher education institutions in