Upload
harmony-edmunds
View
229
Download
0
Tags:
Embed Size (px)
Citation preview
ASSESSING THE NATIONAL ENTRIES
Brian [email protected]
Objective of the assessment The Assessment Process The procedure to be adopted Timescales Accessing the entries Examples and discussion
Outline
To review the entries and “validate” the self assessments to enable a short list to be
selected.
Objective
To provide an independent view of the entrant’s maturity level to assist them in
improving their operation
Primary Objective
Secondary Objective
Process
The companies Self-Assess their maturity level against a series of statements and provide evidence to support this level
Assessors check the companies assessment against the evidence they provide
Assessors’ role is to check that the maturity level selected looks reasonable, i.e. not obviously wrong
The Assessors do not rescore the entry from scratch based on the evidence provided
Assessment Aims
The aim is to select a short list of companies for each category
We need confidence that the best entry in that category will be among the companies short-listed
Companies differ and the score alone, while an indicator, may not provide the complete picture
Comments
Comments, both on individual questions and overall will be very helpful To explain any scoring changesTo assist with shortlistingTo focus on appropriate issues during the
review visitTo provide feedback to the company
What information will Assessors have?
Companies will indicate their assessment by selecting one of 5 statements showing a progressively higher maturity level
They will also provide evidence to support this selection by ticking relevant statement(s) in relation to the core questions and by providing free text evidence
The amount of evidence that companies can provide is limited, hence full proof of their own assessment is not possible or expected
Company Entry Screen – Core Questions
Click most appropriate
maturity level
Completion by Company
Click evidence pre-set that
applies
Completion by Company
Add free text evidence
Completion by Company
Navigation
Moving between
statements in Section
Navigation
Tabs to sections
Assessment Procedure 1
Read the general information first to get a feel for the company and its operations
Check each question, noting the maturity level selected
Check whether the evidence is reasonably supportive of the maturity level selected
Assessment Procedure 2
The lowest two maturity levels require no written evidence to support them
Higher maturity levels should be supported by evidence that is relevant and indicates that the maturity level has been achieved
The highest level needs more specific justification rather than general claims
Note the maturity level statements are generic and may not match every company. Sensible parallels need to be drawn in some cases.
Assessment Procedure 3
Where the evidence generally supports the maturity level selected – no action required
For higher maturity levels some evidence is needed but don’t look for complete confidence
Reduce score by one level where evidence is clearly inadequate – add comment to explain
In exceptional cases reduce further but only where evidence clearly indicates a lower level – again comments should explain reasons
Companies do score themselves too low so be prepared to raise scores when appropriate – comment should explain reason
Assessment Procedure 4
“Not applicable” (N/A) is an option for all questions
Entrants need to explain why the statement is “Not Applicable” if selecting this
Not Applicable means that it is not relevant to the type of business and its operations
It is not a valid response where the company has no system when a similar “best practice” company would have
As previously in MX if N/A is accepted the question is ignored in calculating percentages
Assessors need to check that the reason given is valid and N/A is a reasonable response
Not Applicable responses
Assessment Procedure 5
Two or three Assessors will be asked to assess each entry independently
A Lead Assessor will compare the assessments and adjudicate where necessary to produce an assessed report and score
The MX Board will review the top scoring entries, including any comments from Assessors to determine a short-list of companies to be visited.
Timescales
The Awards have now been launchedThe final date for entries is 23 May 2014Entries will sent for assessment as receivedFew expected much before closing dateAssessors need to complete assessment by
13 June 2014, earlier if possibleThe Short-list will be announced by 20 June
2014Vital that if Assessor is unable to undertake
an allocated Assessment they notify staff immediately
System Log -in Screen
Screen following Log-in
Screen following Log-in
Click to access entries to be
assessed
The companies you are to assess listed here
Assessor 1 & 2 here when assigned. Click to select
company
Assessor’s Entry Sheet
Assessor’s Entry Sheet
Click to access
statementsClick to see
summary report
Initial Screen for Company
Takes you to the questions
Details about company
Initial Screen for Company
Do not click back from this
page or questions
Initial Screen for Company
Click Return to Administration
to go back
Example showing Maturity Level, Pre-set Evidence and Free text Evidence
Entry Options
As with previous MX companies can enter for overall or individual categories
Category entry consists of some Core questions plus some additional questions
To be eligible for overall winner, company must complete all Core questions plus some categories
If entering only Categories then only the relevant questions need to be answered
Entrants Screen
Core questionsMX Category
Questions
Colour coded
Core questions attempted
Category attempted
Drop down to select
What have companies entered?
Entry Screen for extra question for MX Category
Suggestions for evidence – no pre-sets
2000 characters for evidence
instead of 1000
Tabs go to contributing
questions from Core set
Core questions retain pre-set
evidence
Also allow 2000 character evidence
Assessor’s view of Statement Page
Category notes apply to whole section so comment must indicate statement it applies to: “e.g. SM 1 No financial benefits evidenced, 5 reduced to 4”
Assessor’s comments go
here
Assessor changes
maturity level if reqd.
Key Points
Change higher maturity levels only if clearly not supported by the evidence
Be prepared to increase score where justified
But don’t rescore entry just on the basis of the evidence
Comments are invaluable whether maturity level changed or not
Make comments brief, specific and constructive
Any questions?
Admin details from staff
Thank you