Embed Size (px)
Citation preview
Peer Review College (PRC) Member Guidance1. Background to the Peer Review College
Thank you for joining the Peer Review College. technē is committed to competitive application assessment by process of peer review. Peer Review College (PRC) members form a key part of this system, which is designed to ensure that assessment is conducted with the utmost attention to fairness and transparency and to the requirements of academic rigour.
2. The Review ProcessFor technē doctoral studentships the review process runs as follows: Applications are considered at departmental and/or institutional level so that the best
applications go forward to the technē competition. The Peer Review College reviews applications. Subject Group panels meet, and moderate the reviews in order to award the applications a final
score, and produce a ranking. Subject Group rankings are considered by the Management Group and awards are made.
Your primary role is to provide technē with informed, expert academic reviews of applications.
PRC reviewers submit reviews which are used by moderating Subject Groups to rank the proposals. Three reviews are required for each application.
PRC reviewers are allocated to applications on the basis of the research classifications and key words listed in the student’s application and those the reviewer listed when enrolling in the PRC. As far as possible, applications are matched to reviewers with expertise in the subject area, though an exact match may not always possible and PRC members may be asked to review applications in areas that they would not consider themselves an expert.
Applications must be reviewed and graded on their merits, in accordance with the guidance in Appendix A: Assessment Criteria.
3. Completing your Review(s) If you are asked to review one or more applications, you will be sent an invitation via email. The email will contain a link to the technē portal. When you have logged in, the page will display the relevant application(s) you are being asked to review. You will be able to view, print or download the application form and associated references and degree transcripts. You will need to complete your review(s) online as described in Appendix B: Using the technē Portal to complete a Peer Review.
If you consider that you have been inappropriately matched to an application or are unable or unwilling to complete the review, please record the reason in Question 01 of the review ‘Do you have a conflict of interest or another reason why you cannot review this application?’ This will alert technē to the need to assign an alternative reviewer. Please do this as soon as possible.
Your review should be completed using the core assessment criteria at Appendix A:• Quality of the student • Quality of the proposal • Feasibility of the proposal
Under each of the three headings you will be asked to write up to 300 words and to give each criterion a score out of six. Please do not put anything in the “weighting” for the score, it is not used by technē.
You should refer to the assessment criteria for the key elements of each criterion to inform your score. The assessment criteria lists which questions on the technē application relate to each criterion.
Example of good practiceA good review…
Considers all the material in the application, including references Limits itself to material in the application (including any permitted links) Is clear and concise Helps the Subject Group by ‘evidencing’ its discussion with material from the application
rather than just giving generic references to criteria such as ‘impact potential is good’. Makes direct links between assessment criteria and evidence from the application. This
includes when assigning very high or very low scores, all scores should be evidenced - very good candidates can be disadvantaged because reviewers neglect to spell out what they think is strong.
For example, assessing “Quality of the student”:
a) Past academic achievement
This student could be considered outstanding. As their reference 1 makes clear, in addition to a first class honours degree, they gradated top of their year from their Masters in Art History, with a distinction in 2015. They won a university prize for their dissertation, whose topic is directly related to their PhD thesis. During their time as a Masters student they also attended a disciplinary conference and later co-presented their Masters dissertation as a paper with their supervisor at the same conference.
b) Contribution of academic and professional experience to preparedness for doctoral study
This student is exceptionally well prepared for doctoral study. In addition to the MA, which included a research dissertation noted above, since graduating they have worked as an assistant curator at the Tate Britain, and have spent time doing exhibition research in the archive directly contribution to exhibitions within the gallery space.
They also have experience running a range of public education events linked to the exhibitions and during this time have published further work related to these exhibitions in two articles in the Tate Papers journal. This professional experience has directly informed the development of the PhD topic and gives a strong foundation for its more experimental methodologies, it also offers access to networks of curators and gallery researchers important for its interview pool. The published writing, alone and as a co-author indicates a high standard of written work.
c) Degree to which doctoral study will contribute to long term career aims
The student is clear they would like to pursue an academic career, but one that enabled them to continue to engage with art galleries through co-curation and event programming. The combination of past track record and this PhD would seem to place the student in a very strong position to achieve this.
Score: 6 (Outstanding)
Example of poor practiceA bad review … Is overly brief: e.g. ‘this all looks very doable to me’, ‘deserves funding’ and ‘best application I
have ever seen’ Only repeats basic evidence, e.g. ‘first class degree’. Fails to use evidence to substantiate points Cut and pastes large chunks of the application form. Does not draw direct links back to terms of grading criteria.
For example, assessing “Feasibility of the Proposal”:
a) Cohere nce of research plan
Fine.
b) Likelihood of research being completed in the timeframe
Seems doable, perhaps stage 2 fieldwork would be tight.
c) Degree to which it will advance work in the current field
Will advance field to some degree.
d) Potential impact [within and beyond the academy]
Some possible impact, if right connections are made.
e) Expertise of potential supervisors
Fine.
f) Feasibility of necessary training, fieldwork or study trips
Maybe ok, concerns about the big fieldtrip and support for work.
Appendix A: Assessment Criteria Quality of student: 33% Quality of Proposal: 33% Feasibility of Proposal: 33%
Assessment criteria: Past academic achievement Contribution of academic and
professional experience to preparedness for doctoral study
How doctoral study will contribute to long term career aims
As evidenced in 06 Professional Experience,07 – 09 Higher Education15 Personal Statement18 Reference 1 (or at the end)19 Reference 2 (or at the end)27 Other Supervision Points
Assessment criteria: Ideas underpinning the proposal Concept and design of research Fit of the project with the technē
philosophy
As evidenced in 11 Research Project Description13 Resources required15 Personal Statement27 Other Supervision Points
Assessment criteria : Coherence and quality of research plan Feasibility of research being completed in
the timeframe Advancement of work in the current field Potential impact Expertise of potential supervisors Feasibility of necessary training, fieldwork or
study tripsAs evidenced in
11 Research Project Description13 Resources required14 Additional Training Needs 20 - 24 Supervisory Team25 Recommended Further Training26 Ethical Issues27 Other Supervision Points
Score6 A student of outstanding quality, who is
outstandingly well prepared to undertake the proposed postgraduate study.
An outstanding proposal in all of the following: studentship, originality, quality, significance and openness to diverse approaches. It provides full and consistent evidence and justification for the proposal in terms of concept and design. The proposal merits the very highest priority for funding.
The research plan is coherent, clear and convincing and the project has strong potential for impact. The project will significantly advance work in the current field and is undoubtedly capable of timely completion. There is an excellent fit between the project and the expertise of the supervisors. Resourcing of training or fieldwork is unproblematic.
5 A student of excellent quality, who is exceptionally well prepared to undertake the proposed postgraduate study.
An excellent proposal in all of the following: studentship, originality, quality, significance and openness to diverse approaches. It provides full and consistent evidence and justification for the proposal. The proposal should be funded as a matter of priority, but does not merit the very highest priority rating.
The research plan is coherent, clear and convincing and the project has strong potential for impact. The project will advance work in the current field and is capable of timely completion. There is a good fit between the project and the expertise of the supervisors. . Resourcing of training or fieldwork is unproblematic
4 A student of good quality, who is well prepared to undertake the proposed postgraduate study.
A very good proposal in all of the following: studentship, originality, quality, significance and openness to diverse approaches. It provides very good evidence and justification for the proposal. It is worthy of consideration for funding.
The research plan is coherent, clear and convincing and the project has some potential for impact. The project will advance work in the current field and is capable of timely completion. There is an adequate fit between the project and the expertise of the supervisors. Resourcing of training or fieldwork is unproblematic.
3 A student of satisfactory quality, who is prepared to undertake the proposed postgraduate study.
A satisfactory proposal in terms of the overall standard of studentship and quality but which is more limited in terms of originality, significance, its contribution to the research field or openness to diverse approaches. In a competitive context the proposal is not considered of a sufficient quality to recommend for funding.
The research plan is adequate. The project may advance work in the current field and it may be possible to complete it within the timeframe. There is an adequate fit between the project and the expertise of the supervisors. Resourcing of training or fieldwork may present some problems.
2 The quality of the student is inconsistent. The student may be of insufficient quality or may not be well prepared to undertake and complete the proposed postgraduate study.
A proposal of inconsistent quality which has some strengths, innovative ideas and/or good components or dimensions but also has significant weaknesses or flaws in one or more of the following: conceptualisation, design, methodology. As a result of the flaws or weaknesses identified, the proposal is not considered to be of fundable quality.
There are significant weaknesses or flaws in the management of the project and it would be unlikely to be completed within the timeframe or to advance work in the field. The fit between the project and supervisors is inadequate. Resourcing of training or fieldwork is likely to be problematic.
1 A student of an unsatisfactory quality who is not well prepared to undertake and complete the proposed postgraduate study.
A proposal of an unsatisfactory quality which: Has unsatisfactory levels of
originality, quality and/or significance
Contains insufficient evidence and justification for the proposal
Displays limited potential to advance the research field
It is not suitable for funding.
The project is unconvincing in terms of its management or capacity to deliver the proposed outcomes or its contribution to the field. The fit between the project and supervisors is inadequate. Resourcing of training or fieldwork is likely to be problematic.
Appendix B: Using the technē Portal to complete a Peer ReviewYou will receive an email to request you to review an application form, which will be in the format below. Click on the link in the email, which will take you to the relevant part of the portal, via the logon screen.
From: TECHNE <[email protected]>Date: 7 December 2018 at 08:41:13 -00:00Subject: Request for techne Peer Review of application formTo: [email protected]
Dear John,
As part of the techne application assessment process, you have been selected to provide a review for the following application(s):
Application reference Applicant Organisation Date your score is due
TS2\100162 Jane Doe Royal Holloway, University of London
Please provide your review(s) on our online portal by clicking on the following link. Review application(s).
To login you will need your email address (the one this email has been sent to) and your password that you set when you enrolled. If you do not remember the password, use the "Forgot password?" option on the screen to request a new password. Please do not hesitate to contact us if you have any queries about this email or the review process in general.
Best wishes,
The techne [email protected]
Once you have entered your email and password it will take you to the screen below. It is recommended that you use the button “Download all as PDF” on this screen, so that you can later have both the PDF of the application(s) and your review form open at the same time.
Then, click on the Reference link for the application you will review.
Open the application form first (either from the button “Open application form” or from the PDF previously downloaded), in preparation to answer the first question about whether there is any reason you cannot perform the review:
If you select “Yes”, you will be asked to record the reason (for example, if you do not have sufficient expertise in the area of this application) on this screen:
If you select “No” because there is nothing that prevents you from performing the review, then you will be shown the following pages:
Your completed review is stored on the portal and you can view it at any time. If you require a copy, you can download either before or after you press the “Submit” button by clicking at the bottom right on “Download completed review form” as shown below.
