APPLICATION EVALUATION FRAMEWORK REPORT

APPLICATION EVALUATION FRAMEWORK REPORT

Application 2

Friday, 10 August 2012

Perspective

A report generated by RMDSTEM software.

Perspective v3.0

Page 2

AEF Report (Application 2)

EXECUTIVE SUMMARY

Application Information

Title of Application

Funding Program Western Australian Fellowships Program

Number of Research Programs

1

Research Priority Area

Total State Government Funding Request ($K)

Impact of Application

Overall Benefit to Cost Ratio

�

Leverage Ratio (Cash) �

Leverage Ratio (Cash & In-kind)

�

Environmental Benefit Score

0 (0)

Social Benefit Score 0 (0)

Knowledge Non-monetary Benefit Score

0 (0)

Research Programs (RPs)

Inputs Total Cash $K Total In-kind $K Total Contribution $K

Total Contribution

(PV) $K

RP 1: 0 0 0 0

Delivery Risk Profile Probability of Research Success

Probability of Adoption

Probability of Impact

RP 1: 50 80 90

Usage Costs Risk Free $K Risk Adjusted (PV) $K

RP 1: 0 0

Monetary Benefits Risk Free $K Attributed Risk Free $K

Attributed Risk Adjusted (PV) $K

RP 1: 0 0 0

Non-monetary Benefits Environmental Score Social Score Knowledge Score

RP 1: 0 (0) 0 (0) 0 (0)

Page 3


GUIDE TO THE EXECUTIVE SUMMARY The Executive Summary summarises key application information, including the impacts (benefits) for each research program in the application and for the application as a whole. The evaluation tool has automatically calculated the impacts (monetary values and non-monetary scores) based on applicant input. See Appendix D for further information on calculation methods.

Impact of Application

Overall Benefit to Cost Ratio See Appendix D for information on calculation of this ratio

Leverage Ratio (Cash) State Government funding request : applicant’s total committed cash from all other sources

Leverage Ratio (Cash & In-kind)

State Government funding request : applicant’s total committed input (cash and in-kind) from all other sources

Environmental Benefit Score1 The sum of the environmental benefit scores for all research programs

Social Benefit Score1 The sum of the social benefit scores for all research programs

Knowledge Non-monetary

Benefit Score1

The sum of the knowledge economy non-monetary benefit scores for all research programs

1The number in brackets after the score is the total number of benefits contributing to the score.

Research Programs (RPs)

Inputs Total Cash $K Total In-kind $K

Total Contribution $K

Total Contribution

(PV) $K

Research Program X

State Government funding request + total committed

cash from all other sources

Total committed in-kind from all

sources

Total contribution (total cash + total

in-kind)

The present value (PV) of the total

contribution

Delivery Risk Profile2 Probability of

Research Success Probability of

Adoption Probability of Impact

Research Program X

Calculated from the applicant’s input for

Technical Risk

Calculated from the applicant’s input for

Adoption Risk

Calculated from the applicant’s input for Impact

Realisation Risk 2See Delivery Risk Profile (section 4) for a summary of how the risk profile was derived from the applicant’s input.

Usage Costs Risk Free $K Risk Adjusted (PV) $K

Research Program X

Derived from the applicant’s input for each usage

Usage cost adjusted for risk and discounted to determine the present value

Monetary Benefits Risk Free $K Attributed Risk Free $K

Attributed Risk Adjusted (PV) $K

Research Program X

Risk free value claimed by the applicant

Risk free value attributed to the proposal (calculated from the attribution level claimed

by the applicant)

The attributed value after adjusting for risk and

discounting to determine the present value

Non-monetary Benefits

Environmental Score1 Social Score

1 Knowledge Score

1

Research Program X

The sum of the scores for the program’s

environmental benefits identified by the applicant

The sum of the scores for the program’s social

benefits identified by the applicant

The sum of the scores for the program’s knowledge economy

non-monetary benefits identified by the applicant

Page 4


Table of Contents

EXECUTIVE SUMMARY ............................................................................................2

GUIDE TO THE EXECUTIVE SUMMARY ..................................................................3

1 INTRODUCTION....................................................................................................5

1.1 Application Information..................................................................................5

1.2 Funding Program ..........................................................................................5

2 STRATEGIC ALIGNMENT.....................................................................................6

2.1 Key Priority Area ...........................................................................................6

2.2 Contribution to Western Australia’s Performance ..........................................6

2.3 Contribution to Innovation Systems Performance..........................................7

3 APPLICATION CHARACTERISATION ..................................................................8

3.1 Characterisation of R&D Activities.................................................................8

3.2 Characterisation of Non-R&D Activities .........................................................8

3.3 Beneficiaries .................................................................................................8

3.4 Collaboration.................................................................................................9

3.5 Technology Objective....................................................................................9

3.6 Technical Leadership ....................................................................................9

3.7 Degree of Innovation.....................................................................................9

3.8 Significance of Market Failure .......................................................................9

4 RESEARCH PROGRAMS....................................................................................10

4.1 Research Program 1: ..................................................................................10

APPENDICES...........................................................................................................17

Appendix A – Framework Overview ...................................................................18

Appendix B – Strategic Alignment Dimension Choices.......................................19

Appendix C – Application Characterisation Dimensions .....................................25

Appendix D – Executive Summary Monetary Calculations and Non-

monetary Scoring......................................................................................29

Appendix E – Attribution Level Anchored Scale for Economic/Knowledge

Economy Monetary Benefits .....................................................................30

Appendix F – Impact Level Anchored Scales for Non-monetary Benefits ...........31

Appendix G – Risk Scoring System....................................................................32

1 INTRODUCTION This report is an automated summary of information the applicant entered into the Department of Commerce’s evaluation tool for the application “”. The department uses the report to assist with evaluating the application for funding. The evaluation tool is a software implementation of the science and innovation investment evaluation framework developed by RMDSTEM Limited on behalf of the department. The framework comprises five pillars, namely application information, strategic alignment, application characterisation, impact evaluation and delivery risk profile. For more information about the framework, please see Appendix A.

1.1 Application Information

Title of Application Lead Applicant Organisation

Lead Applicant Research Office Contact Details

Proposal Description

1.2 Funding Program

Funding Scheme Fellowships Scheme

Funding Program Western Australian Fellowships Program

Does this application meet all the eligibility criteria for this program?

Page 6


2 STRATEGIC ALIGNMENT This section maps the alignment of the proposal with Western Australian strategic priorities. The data from this section, aggregated over a number of funding rounds and across different funding programs, will enable the Department of Commerce to evaluate the contribution of its investment to the strategic areas. The choices available for each of these dimensions are listed in Appendix B.

2.1 Key Priority Area

Key priority area:

Comment:

2.2 Contribution to Western Australia’s Performance

The purpose of this dimension is to evaluate the contribution of the proposal to Western Australia’s economic, environmental and social performance. Note that there is benchmark data available for these dimensions which permits comparison of Western Australia’s performance against other jurisdictions.

2.2.1 Economic Performance

Contribution Areas Description

2.2.2 Environmental Performance

Contribution to Western Australia's environmental performance is divided into four main areas as listed below.

Area Contribution Areas Description

Sustainable Resource Management

Air Quality and Adaptation to Climate Change

Sustainable Management of Oceans and Estuaries

Biodiversity

2.2.3 Social Performance

Contribution to Western Australia's social performance is divided into three main areas as listed below.

Area Contribution Areas Description

Health and Lifestyle

Education and Training

Page 7


Crime and Security

2.3 Contribution to Innovation Systems Performance

The purpose of this dimension is to evaluate the contribution of the proposal to key dimensions of Western Australia’s innovation systems performance.

2.3.1 Research Capacity and Skill Base


2.3.2 Business Innovation


2.3.3 Links and Collaboration


Page 8


3 APPLICATION CHARACTERISATION This section summarises the application characterisation of the proposed R&D and non-R&D activities. The set of characterisation criteria can be found in Appendix C. The data from this section will be aggregated over a number of funding rounds and across different funding programs to provide the Department of Commerce (and potentially external review panels) with a good understanding of the balance of the department’s investment portfolio and the mix of funded projects.

3.1 Characterisation of R&D Activities

3.1.1 Classification of R&D Activities

R&D Field Comment

3.1.2 Type of R&D Activities

R&D Type Comment

3.1.3 Spread of R&D Activities

R&D Spread Comment

3.2 Characterisation of Non-R&D Activities

Non-R&D Activity Examples

3.3 Beneficiaries

3.3.1 Scope of Beneficiaries

Scope of Beneficiaries Comment

3.3.2 Type of Beneficiaries

Type of Beneficiaries Examples

Page 9


3.4 Collaboration

3.4.1 Scope of Collaboration

Scope of Collaboration Comment

3.4.2 Collaboration Partners

Collaboration Partners Examples

3.5 Technology Objective

Technology Objective:

Examples:

3.6 Technical Leadership

Technical Leadership:

Examples:

3.7 Degree of Innovation

Degree of Innovation:

Examples:

3.8 Significance of Market Failure

Significance of Market Failure:

Comment:

Page 10


4 RESEARCH PROGRAMS This section evaluates the potential benefits of the proposal at program level. This evaluation is broadly based on the input-to-impact framework adopted by the Department of Innovation, Industry, Science and Research (DIISR) in the Cooperative Research Centres funding program.

4.1 Research Program 1:

4.1.1 Inputs

Total Inputs ($K): 0 Total cash contribution ($K): 0 Total in-kind contribution ($K): 0

Cash Contribution

Year 1 Year 2 Year 3 Year 4 Year 5 Total Present Value

Requested WA Government funding ($K)

0 0

Comment: Committed business cash contribution ($K)

Comment: Committed research service provider cash contribution ($K)

Comment: Committed government R&D funds ($K)

Comment: Committed other cash contribution ($K)

Comment: In-kind Contribution

Year 1 Year 2 Year 3 Year 4 Year 5 Total Present Value

Committed business in-kind contribution ($K)

Comment: Committed research service provider in-kind contribution ($K)

Comment:

Page 11


Committed other in-kind contribution ($K)

Comment:

Page 12


4.1.2 Activities

Type of Activity Description

4.1.3 Outputs

Below are the lists of outputs of the proposed research program that are expected to be delivered. Output 1 Type: Sub-type: Description: Key milestones: End of Year 1 End of Year 2 End of Year 3 End of Year 4 End of Year 5

4.1.4 Usages (Uptake of Outputs)

Below are the lists of usages of outputs delivered by this research program and, if applicable, others specified in the application. Usage 1 Description: Underpinning output(s): Usage cost ($/unit): Number of units (#): Usage start year: Year Time to reach full adoption (from usage start year):

years

Key milestones: End of Year 1 End of Year 2 End of Year 3 End of Year 4 End of Year 5 End of Year 6 End of Year 7

Page 13


4.1.5 Impacts

Below are the lists of impacts created from the uptake of outputs of this research program and, if applicable, others specified in the application. Appendix E describes the anchored scale for attribution of the economic/knowledge economy monetary impacts. Appendix F describes the anchored scales for the level of impact of the non-monetary impacts.

4.1.5.1 Economic Impacts

Economic impact 1 Benefit name: Area of benefit: Sub-area of benefit: Description: Underpinning usage(s): Potential annual benefit ($Kpa):

Benefit start year: Year Time to reach full impact (from benefit start year):

years

Benefit duration: years Attribution: Key milestones: End of Year 1 End of Year 2 End of Year 3 End of Year 4 End of Year 5 End of Year 6 End of Year 7 End of Year 8 End of Year 9 End of Year 10

Comment

4.1.5.2 Environmental Impacts

Environmental impact 1 Benefit name: Area of benefit: Sub-area of benefit: Description: Underpinning usage(s): Level of impact: Key milestones: End of Year 1 End of Year 2 End of Year 3 End of Year 4 End of Year 5 End of Year 6

Page 14


End of Year 7 End of Year 8 End of Year 9 End of Year 10

Comment

4.1.5.3 Social Impacts

Social impact 1 Benefit name: Area of benefit: Sub-area of benefit: Description: Underpinning usage(s): Level of impact: Key milestones: End of Year 1 End of Year 2 End of Year 3 End of Year 4 End of Year 5 End of Year 6 End of Year 7 End of Year 8 End of Year 9 End of Year 10

Comment

4.1.5.4 Knowledge Economy Monetary Impacts

Monetary Impact 1 Benefit name: Area of benefit: Description: Underpinning usage(s): Potential annual benefit ($Kpa):

Benefit start year: Year Benefit duration: years Attribution: Key milestones: End of Year 1 End of Year 2 End of Year 3 End of Year 4 End of Year 5 End of Year 6 End of Year 7

Page 15


End of Year 8 End of Year 9 End of Year 10

Comment

4.1.5.5 Knowledge Economy Non-monetary Impacts

Knowledge Economy Non-monetary Impacts Benefit name: Area of benefit: Description: Underpinning usage(s): Level of impact: Key milestones: End of Year 1 End of Year 2 End of Year 3 End of Year 4 End of Year 5 End of Year 6 End of Year 7 End of Year 8 End of Year 9 End of Year 10

Comment

4.1.6 Delivery Risk Profile

To calculate the final impact of proposed outputs, the risk profile along the path to impact has been evaluated. Specifically, for each research program, the applicant indicated the magnitude of several barriers to success in each of three risk categories: • Technical risk – the applicant indicated how ‘barriers to innovation’ (measures of

technical risk) would affect output delivery, which provided a value for the probability of research success;

• Adoption risk – the applicant indicated how ‘barriers to adoption’ (measures of adoption

risk) would affect output adoption/utilisation by end-users, which provided a value for the probability of adoption; and

• Impact realisation risk – the applicant indicated how ‘barriers to impact’ (measures of impact risk) would affect realisation of the expected benefits from adopted outputs, which provided a value for the probability of impact.

For each barrier to success, the applicant chose one of six qualitative levels – negligible, low, minor, moderate, major and significant. Appendix G shows the probability of success assigned to each of these six levels.

Page 16


For each of the three path to impact stages – output delivery, adoption and impact – the total probability of success was determined by multiplying the probability of success of all barriers in that stage. The probabilities of success for the three path to impact stages for this research program are shown in the table below. To determine the program’s potential risk adjusted monetary benefit, the program’s attributed monetary benefit was multiplied by these three probabilities.

Probability of Research Success 50%

Probability of Adoption 80%

Probability of Impact 90%

To avoid being over-optimistic, the maximum probability of success for each path to impact stage is capped using values based on best-practice and consultation. The caps are 50% for the output delivery stage, 80% for the adoption stage and 90% for the impact stage, even if all barriers have been identified as ‘negligible’. The probability of success for a single stage will be 100% only when the research program has successfully completed that stage.

4.1.6.1 .Technical Risk

Technical Risk Level Comment

Technical complexity & degree of innovation required to deliver outputs

Skills and competence for output development

Sufficient resources for outputs development

Sufficient time allocated for outputs development

4.1.6.2 Adoption Risk

Adoption Risk Level Comment

End-user awareness of the application outputs

Cost of adoption/implementation

Perceived cost/consequence of failure

Competition from superior alternative solutions

Public perception, regulation and government approval

4.1.6.3 Impact Realisation Risk

Impact Realisation Risk Level Comment

Sensitivity of the impact realisation to adoption/implementation conditions

End-user skills and competence for output adoption/implementation

Ability to integrate the outputs into existing operating processes

Acceptance of new outputs by end-users’ staff members

Page 17


APPENDICES

Page 18


Appendix A – Framework Overview

The evaluation framework consists of five pillars which enable the Department of Commerce to select and manage investments. Pillar A identifies the application by the nature of funding program and applicants. Pillars B and C enable the Department of Commerce to evaluate the alignment of the innovation investment portfolio, over an extended period of time and across different funding programs, with key strategic dimensions. Pillars D and E enable the department to evaluate and rank the impact of individual applications within each funding program.

The five pillars enable the Department of Commerce to perform all evaluation framework functions. Pillar A permits capture of basic application (birth certificate) information such as the funding program applied for and the applicant’s name and contact details. Pillar B permits evaluation of the alignment of the investment portfolio with the Department’s three key strategic dimensions, namely the research priority area, the contribution to Western Australia’s (economic, environmental and social) performance and the innovation systems performance in the state. Pillar C permits a deeper analysis of the investment portfolio against a set of selected dimensions such as degree of innovation of different funded projects, level and extent of collaboration created by the different funding programs. Pillar D permits evaluation of the potential benefits of each application, assuming success in delivering all proposed outputs. This evaluation is broadly based on the input-to-impact framework adopted by the Department of Innovation, Industry, Science and Research (DIISR) in the Cooperative Research Centres funding program. Pillar E permits evaluation of the risk profile of each application, focusing on risks associated with delivering the proposed outputs, adoption/utilisation of the outputs by potential end-users and the risk of not realising the expected benefits even after outputs have been adopted or implemented. Importantly, pillars B and C define the positioning or strategic alignment of an application while pillars D and E relate to the application’s projected impacts.

Page 19


Appendix B – Strategic Alignment Dimension Choices

1. Key Priority Area Minerals (including minerals technologies) Gas (including petroleum technologies) Environment Medical, including preventative health Other

2. Economic Performance

Contribution Area Description

Western Australia’s state

income

The proposal will contribute to the creation of new industry, strengthen

competitiveness of existing industry or enable attraction of

international investment.

Employment The proposal will contribute to employment in Western Australia

through creating new job opportunities, enabling better access to

existing job opportunities or developing skills in the existing workforce.

Labour productivity The proposal will contribute to improving labour productivity through

novel technology solutions, training modules or management

systems.

Other Applicant’s description.

3. Sustainable Resource Management


Sustainable natural land

management

The proposal will contribute to increasing the amount and connectivity of natural vegetation, improving vegetation condition index, supporting maintenance or growth of ecological processes, or increasing resilience of landscape to climate change impacts.

Sustainable forest management

The proposal will contribute to increasing re-forestation, enabling efficient maintenance of forest ecosystem processes or improving forest health condition.

Sustainable water management

The proposal will contribute to improving the management of hydrological balance and wetland ecosystem processes, improving the wetland condition index in Western Australia and strengthening water management in industry to reduce demand on freshwater resources.

Sustainable food production

management

The proposal will contribute to effective management of food production through improving production efficiency at farm level and understanding of the links between farm production and delivery of products into the market.

Sustainable mineral resources management

The proposal will contribute to the sustainable management of natural resources through development of effective decision making tools, improving land use planning and environmental protection strategies.

Sustainable energy

resource management

The proposal will contribute to improving the management of energy resources by improving energy efficiency of existing applications, enabling more energy efficient products/processes or increasing utilisation of renewable energy sources.

Sustainable waste management

The proposal will contribute to improving the management of waste by reducing the volume of waste produced, enabling more effective waste treatment processes, creating more applications/market for waste recycling/reuse products or developing more effective waste storage technologies.

Page 20



4. Air Quality and Adaptation to Climate Change


Greenhouse gas emissions The proposal will contribute to reducing greenhouse gas emissions

through more efficient usage of fossil fuel based energy (per person,

stock or product), wider use of renewable sources of energy or more

effective technology for capturing and storage of greenhouse gases.

Other pollutant emissions The proposal will contribute to reducing the impact of pollutant

emissions (SOx, NOx, volatile organic compounds, dust, ...) through

reducing emissions rate/amount and/or improving treatment/recovery

technologies.

Adaptation to minimise impact of climate change

The proposal will contribute to improving Western Australia’s resilience

to climate change consequences (such as rising sea levels, severe

storms and floods, long periods of drought, …) through better

prediction models, better infrastructure design and/or identifying

alternative water sources.


5. Sustainable Management of Oceans and Estuaries


Management of fish stocks The proposal will contribute to enhanced fish stock management

through regulation to protect endangered fish species and/or

monitoring technologies to study behaviour of fish stocks in different

environments.

Management of estuaries The proposal will contribute to improving the management of

estuaries. This could be achieved through better data collection and

monitoring tools/programs, and more effective response and recovery

strategies.

Management of ocean and sea ecosystems including oil (and gas) spills and other major polluting events

The proposal will contribute to improving the management of oil and

gas spills (or other major polluting events) through better infrastructure

design, more accurate testing and modelling of oil spill events and

more effective responses to these events.

Management of aquaculture The proposal will contribute to sustainable aquaculture management through enabling effective conservation, management and development of living aquatic resources, supporting economic viability and environmental and social sustainability of fisheries and aquaculture and protecting the interests of the wider community.


Page 21


6. Biodiversity Contribution Area Description

Management of marine ecosystems

The proposal will contribute to improving the management of

marine ecosystems. This could be achieved through better data

collection and monitoring tools/programs, and more effective

response and recovery strategies.

Management of terrestrial ecosystems

The proposal will contribute to improving the management of

terrestrial ecosystems. This could be achieved through better data

collection and monitoring tools/programs, and more effective

response and recovery strategies.

Native flora and fauna The proposal will contribute to reducing the impact of climate

change and/or other threats on native flora and fauna through

better data collection and monitoring tools/programs, and more

effective response and recovery strategies.

Impact of climate change on marine and terrestrial environments

The proposal will contribute to minimising the impact of climate

change on marine and terrestrial environments through better

prediction models, better data collection and monitoring

tools/programs, and more effective response and recovery

strategies.


7. Health and Lifestyle


Life expectancy The proposal will contribute to improving life expectancy in

Western Australia through better maternity, infant and early

childhood care, better access to health services and/or more

effective community education programs.

Quality of life The proposal will contribute to improving quality of life in Western

Australia by reducing the incidence of chronic lifestyle related

disease (such as obesity, smoking, alcohol consumption) and/or

reducing the burden of diseases such as mental illness or age-

related health issues.

Improve effectiveness of health sector

The proposal will contribute to introducing more effective planning,

monitoring and management systems/tools to enable successful

prevention, treatment or management initiatives/strategies.

Health impact of industrial development on local communities

The proposal will contribute to reducing the impact of industrial

development on local communities by lowering their social and

environmental footprint or enabling more effective monitoring,

modelling or management strategies.

Indigenous health The proposal will have a special focus on Indigenous health.


Page 22


8. Education and Training Contribution Area Description

Number of people with non-school qualifications (25-64 years)

The proposal will contribute to increasing the number of people

with non-school qualifications* in Western Australia by enabling

wider access to education services.

Effectiveness of education system

The proposal will contribute to improving literacy and numeracy

through higher participation, attendance retention and graduation

rates.

Training and industry participation in workforce development

The proposal will contribute to increasing participation in training

and industry training and professional development through

smarter, more flexible and more accessible training modules/tools.

Indigenous education The proposal will have a special focus on Indigenous education.


9. Crime and Security


Number of victims of crime The proposal will contribute to reducing the impact of personal and

household crime on Western Australia’s society. This could be

achieved through enabling more effective security procedures and

public campaigns.

Standard of community security The proposal will contribute to improving the standard of

community security through better protection against new threats

such as cross-border or cyber threats, or business related crimes.

Standard of corrective services The proposal will contribute to improving the standard of corrective

services.

Indigenous communities The proposal will have a special focus on security in Indigenous

communities.


Page 23


10. Research Capacity and Skill Base Contribution Area Description

Publications by researchers

participating in the proposal

The proposal will contribute to improving the number of peer reviewed publications (both journal and conference publications) by researchers participating in the proposal.

Number of fields (at 2-digit

level), published by research

partners, with higher than world

average citation rate

The proposal will contribute to improving the quality of publications as indicated by the number of fields (at 2-digit level, ANZSRC 2008 classification) with higher than world average citation rate. Applicants will be asked to state the average citation rate in the field of their publication as a benchmark value.

Number of students completing

higher degrees by research in

Western Australia

The proposal will contribute to increasing the number of Masters and PhD students, both Australian and international, completing higher degrees by research in Western Australia.

Number of students completing

higher degrees (course work) in

Western Australia

The proposal will contribute to increasing the number of Masters students, both Australian and international, completing course work Masters degrees in Western Australia.

Number of Honours students

participating in technology

development projects in

Western Australia

The proposal will contribute to increasing the number of Honours students, both Australian and international, completing a research project as part of their undergraduate degree in Western Australia.

Higher education expenditure

on R&D

The proposal will contribute to increasing higher education expenditure on R&D in Western Australia.

Public expenditure on tertiary

education in Western Australia

The proposal will contribute to increasing public expenditure on tertiary education.

R&D infrastructure in Western

Australia

The proposal will contribute to improving the standard of R&D infrastructure in Western Australia.

11. Business Innovation


Business participation in the innovation system of Western Australia

The proposal will contribute to improving business participation in

the Western Australian innovation system through attracting new

business partners or improving the level of participation of existing

partners.

Enabling industry partners to register for the R&D tax concession

The proposal will enable existing and potential industry partners to

register for the R&D tax concession. Applicants should refer to the

eligibility criteria for the R&D tax concession before selecting this

dimension.

Business/industry partners R&D expenditure

The proposal will contribute to increasing business/industry

partners’ R&D expenditure.

Patent applications by participating research and/or industry partners

The proposal is likely to result in outputs that could be patented by

research and/or industry partners.

Investment in early stage venture capital in Western Australia

The proposal will contribute to increased investment in early stage

venture capital in Western Australia.

Business participation in non- The proposal will contribute to increasing the level of participation

Page 24


technological innovation in primary industry sector

in non-technological innovation in the primary industry sector. Non-

technological innovation refers to organisational and marketing

innovation. Organisational innovation refers to the implementation

of new organisational methods not used in the firm before.

Marketing innovation refers to the implementation of new

marketing methods.

Business participation in non-technological innovation in the manufacturing sector

The proposal will contribute to increasing the level of participation

in non-technological innovation (as defined above) in the

manufacturing sector.

Business participation in non-technological innovation in the service sector

The proposal will contribute to increasing the level of participation

in non-technological innovation (as defined above) in the service

sector.

12. Links and Collaboration


Collaboration between industry and publicly funded research agencies

The proposal will contribute to improving collaboration between industry and publicly funded research agencies (PFRA).*

Collaboration between industry and universities participating in the application

The proposal will contribute to improving collaboration between industry and universities.

Formal agreements on academic/research collaboration between Australian universities and overseas institutions

The proposal will contribute to improving the level of collaboration between Australian universities and overseas institutions as indicated by the number of formal agreements on academic/research collaboration.

Higher education expenditure in R&D financed from abroad

The proposal will contribute to increasing higher education expenditure in R&D financed from abroad.

Proportion of Australian science and engineering publications co-authored with overseas researchers

The proposal will contribute to improving collaboration between Australian and international researchers as indicated by the proportion of Australian science and engineering publications (including agriculture and medical publications) co-authored with overseas researchers.

SME collaboration with Western Australian higher education institutions

The proposal will contribute to improving collaboration between SMEs (broadly defined as firms with fewer than 200 employees) and Western Australian higher education institutions.

Large firm collaboration with Western Australian higher education institutions

The proposal will contribute to improving collaboration between large firms and Western Australian higher education institutions.

Page 25


Appendix C – Application Characterisation Dimensions

1. Classification of R&D Activities R&D field Description

Science—General (210000, 230000-270000)

For the purpose of this evaluation, applicants will be asked to consider this field as inclusive of Mathematical, Physical, Chemical, Earth and Biological Sciences.

Social Sciences, Humanities and Arts—General (220000, 330000-440000)

For the purpose of this evaluation, applicants will be asked to consider this field as inclusive of Education, Economics, Commerce, Management, Tourism and Services, Policy and Political Science, Studies in Human Society, Behavioural and Cognitive Sciences, Law, Justice and Law Enforcement, Journalism, Librarianship and Curatorial Studies, The Arts, Language and Culture, History and Archaeology and Philosophy and Religion.

Information, Computing and Communication (280000)

For the purpose of this evaluation, applicants will be asked to consider this field as inclusive of Information, Computing and Communication.

Engineering and Technology (290000)

For the purpose of this evaluation, applicants will be asked to consider this field as inclusive of Industrial Biotechnology and Food Sciences, Aerospace, Manufacturing, Automotive, Mechanical and Industrial, Chemical, Resources, Civil, Electrical and Electronic, Geomatic, Environmental, Maritime, Metallurgy, Materials, Biomedical, Computer Hardware, Communications Technologies, Engineering Interdisciplinary Studies and Other Engineering and Technology.

Agricultural, Veterinary and Environmental Sciences (300000)

For the purpose of this evaluation, applicants will be asked to consider this field as inclusive of Agricultural, Veterinary and Environmental Sciences.

Architecture, Urban Environment and Building (310000)

For the purpose of this evaluation, applicants will be asked to consider this field as inclusive of Architecture, Urban Environment and Building.

Medical and Health Sciences (320000)

For the purpose of this evaluation, applicants will be asked to consider this field as inclusive of Medical and Health Sciences.

2. Type of R&D Activities

R&D type Description

Pure basic research Experimental and theoretical work undertaken to acquire new knowledge without looking for long-term benefits other than the advancement of knowledge.

Strategic basic research Experimental and theoretical work undertaken to acquire new knowledge directed into specified broad areas in the expectation of useful discoveries. It provides the broad base of knowledge necessary for the solution of recognised practical problems.

Applied research Original work undertaken primarily to acquire new knowledge with a specific application in view. It is undertaken either to determine possible uses for the findings of basic research or to determine new ways of achieving some specific and predetermined objectives.

Experimental development

Systematic work, using existing knowledge gained from research or practical experience for the purpose of enabling innovation to create new or improved products/processes.

Translational studies Studies aimed at translating research findings into practice and evaluating their impacts/outcomes to stakeholders.

3. Spread of R&D Activities

R&D spread Description

Single-discipline research Single discipline research advances the knowledge base within one field. Multiple-discipline research

Multiple-discipline research involves the use of knowledge and techniques in one field applied to obtain results in another field.

Page 26


Interdisciplinary research Interdisciplinary research involves one or more researchers representing several different fields of study.

4. Characterisation of Non-R&D Activities

Non-R&D activity Description

Education and training of personnel and students

Refers to teaching, training students and public education activities, using established methods and subject knowledge. Postgraduate research, including supervision of the research, should be considered as an R&D activity and captured in dimension C1.

General purpose or routine data collection

Refers to data collection of a general nature that is normally carried out to record natural, biological, economic or social phenomena of general public or government interest. Examples are topographical mapping and routine geographical or environmental surveys.

Maintenance of national standards

Refers to routine testing and analysis of materials, components, products, processes, soils, atmospheres, etc., for standard compliance that are excluded from R&D activities.

Feasibility studies Refers to feasibility studies which involve gathering information about existing conditions, for use in deciding whether or not to implement a project such as studies to determine the viability of a petrochemical complex in a particular location.

Patent and licence work Refers to commercial, administrative and legal work associated with patenting, copyrighting and licensing.

Policy related studies Refers to routine management studies or efficiency studies. Marketing and market studies

Refers to market research and opinion polls.

5. Scope of Beneficiaries

Scope of Beneficiaries Description

International Where a project has the potential to be applied internationally because of its superior results or it is part of internationally cooperative research. In addition to benefiting Australia (including Western Australia), more than 50% of the application benefits will be delivered to international partners outside Australia.

Australian Where a project has the potential to be applied within Australia because it targets domestic conditions or it is part of cooperative research within Australia. In addition to benefiting Western Australia, more than 50% of the application benefits will be delivered to national partners within Australia.

Western Australian Where a project has the potential to be applied in Western Australia because it targets regional conditions or it is part of cooperative research within Western Australia. The majority of the application benefits will be limited to Western Australia.

6. Type of Beneficiaries

Type of Beneficiaries Description

Research organisations The majority of the proposal benefits will be delivered to participating research organisations. Examples are enhanced reputation/ranking or attracting high quality students and researchers.

Industry The majority of the proposal benefits are likely to be realised by the application industry partners. Examples are improved process efficiency or reduction in operating costs.

Community The majority of the proposal benefits are likely to be realised by the different community groups. Examples are better working conditions or reduction in greenhouse gas emissions.

Government The majority of the proposal benefits are likely to be realised by the State Government. Examples are higher effectiveness of health and education investments or better standards, policies or regulations.

Page 27


7. Scope of Collaboration

Scope of Collaboration Description

International The project is co-funded (cash and in-kind support) by international collaborators (public and private), and has access to international expertise.

Australian The project is co-funded by national collaborators (public and private), and has access to national expertise.

Western Australian The project is co-funded by state collaborators (public and private), and has access to expertise in Western Australia only.

8. Collaboration Partners

Collaboration Partners Description

Government (Commonwealth and all states) R&D funds

Federal or State (including Western Australia) departments will host proposal activities and/or will contribute proposal resources (cash and/or in-kind support).

Universities Australian or international universities will host proposal activities and/or will contribute proposal resources (cash and/or in-kind support).

Publicly funded research institutions

Australian publicly funded research institutions will host proposal activities and/or will contribute proposal resources (cash and/or in-kind support).

Private research institution

Private research institutions will host proposal activities and/or will contribute proposal resources (cash and/or in-kind support).

Corporate research groups

Corporate research groups will host proposal activities and/or will contribute proposal resources (cash and/or in-kind support).

Industry Industry partners will contribute proposal resources (cash and/or in-kind support) and/or will host proposal activities.

Non-profit organisations Non-profit organisations will contribute proposal resources (cash and/or in-kind support) and/or will host proposal activities.

9. Technology Objective

Technology Objective Description

Research and development

Where a proposal is aiming at carrying out an R&D activity (as defined in Dimension C1.2) to explore a new research area or develop new technology.

Scanning The primary objective of the proposal is to scan existing technology options developed/pursued by other research groups to identify potential areas of interest to the applicant or its end-users.

Adoption Where a proposal is primarily aiming at enabling trial/adoption of an existing technology developed by other research groups.

Technical support Where a proposed activity is primarily focused on the technical support of an existing end-user operation/process. Typically, this may be the provision of routine technical analysis, laboratory test work or technical consultation.

Page 28


10. Technical Leadership Technical Leadership Description

Market shaping The proposal is likely to result in significant changes to an existing process or develop new markets/applications which are currently not available. Typically, core technologies in this area are ground-breaking and considered cutting edge developments.

Innovative development The proposal is likely to result in significant changes to an existing process to provide a step change in the end-users’ competitive position.

Niche application The proposal is aimed at solving a particular challenge/issue within a sector/market/process/organisation.

Smart application The proposal is aimed at applying known processes/products/technologies but uses enhancements to suit local applications.

Fast follower application The proposal applies relatively proven processes/products/technologies to a new sector/industry.

Competent application The proposal is about provision of expertise to select and operate standard processes/products/technologies.

11. Degree of Innovation

Benefit Horizon Description

Horizon 1 The proposal will develop a better understanding of, or introduce incremental change to, the existing knowledge base.

Horizon 2 The proposal will introduce a step change in the existing knowledge base to expand its application or result in a breakthrough.

Horizon 3 The proposal will introduce a novel technology that is likely to result in a paradigm shift and open new opportunities.

12. Significance of Market Failure

Significance Description

Low The proposal will target areas/issues that private industry is capable of addressing without government support.

Medium The proposal will target areas/issues that the can be addressed only by the most efficient/profitable market segments. A significant proportion of value created is likely to be lost without government support.

High The proposal will target areas/issues that cannot be addressed even by the most capable private company. Only a small proportion of the value created can be realised without government support, in the current market situation.

Page 29


Appendix D – Executive Summary Monetary Calculations and Non-monetary Scoring

Monetary calculations

The Executive Summary of the Application Evaluation Framework Report contains values automatically calculated by the evaluation tool based on applicant input. The calculation methodology for each category of monetary values is shown below:

1. Overall Benefit to Cost Ratio

The overall benefit to cost ratio for the application as a whole is calculated by dividing the overall risk adjusted monetary benefit value by the sum of the overall risk adjusted usage cost and the overall cash and in-kind input (present value).

2. Leverage Ratio (Cash & In-kind)

The leverage ratio is the ratio of the funding requested from the WA State Government to the committed cash and in-kind input attracted from other sources to Western Australia.

3. Risk Adjusted Usage Cost

The overall risk adjusted usage cost for each research program is the sum of the cost for each of the program’s usages, and is a net present value (NPV). Each usage cost was derived by:

• multiplying the implementation cost/unit by the number of units;

• distributing take up of the usage over time, based on the start year and time to reach full impact and assuming an appropriate S-curve adoption profile;

• applying a risk factor based on the level of the program’s technical and adoption risks; and

• discounting the usage cost using a discount rate of 8% to determine the PV.

4. Risk Adjusted Benefit

The overall attributed risk adjusted monetary benefit value for each research program is the sum of the value of the economic benefit(s) and the knowledge economy monetary benefit(s) for the program. Each benefit value was derived by:

• adjusting the attributed benefit value per annum by an appropriate S-curve to reflect the time taken to reach full impact;

• applying a risk factor based on the level of the research program’s technical, adoption and impact realisation risks; and

• discounting the benefit value by using a discount rate of 8% to determine the present value (PV).

Non-monetary scoring

The non-monetary scoring provides a semi-quantitative evaluation of the expected environmental, social and knowledge economy non-monetary benefits for each research program and for the application as a whole (as a cumulative score). NB: for each score, the number in brackets after the score is the total number of benefits contributing to that score. To determine overall scores for each benefit category, the evaluation tool adds the individual scores for each benefit identified by the applicant in that category. The individual scores are based on the level of impact as in the tables in Appendix F.

Page 30


Appendix E – Attribution Level Anchored Scale for Economic/Knowledge Economy Monetary Benefits

Level of Attribution

Description Scale

High

The adoption of the technical solution could not occur without the application. The benefits (being discussed) could not have been derived without the application outputs. It is fair to attribute the great majority of the benefit to the application.

90%

Moderately high

These benefits are mostly attributable to the application outputs. It is unlikely that the benefits would be realised in the same timeframe without the application. It is fair to attribute a significant proportion of the realised benefits to the work of application.

75%

Moderate

These benefits are moderately attributable to the application outputs. It is reasonable to assume that the attribution of benefit is likely to be shared between the application and that of other parties, or It is probable that the expected benefits could be realised in a longer timeframe (>10 years) without the application, or It is likely that other research providers could generate similar benefits to end-users but over a longer time to achieve a similar outcome.

50%

Moderately low

There is a moderately low attribution of benefits from the application outputs. The attribution of benefit is reasonably credited substantially to the work of other parties, or It is probable that the benefit could be generated in a longer timeframe (5-10 years) without the application, or It is likely that other research providers could generate similar benefits to end-users within the next few years.

25%

Low

These benefits are modestly attributable to the application. A fair attribution of the majority of the benefit is to other parties, or It is probable that the benefits could be realised in the same timeframe without the research program.

10%

Page 31


Appendix F – Impact Level Anchored Scales for Non-monetary Benefits

A. Environmental Impact Scale Level of Impact

Description Scale

Minor There is a small environmental benefit in this area, however this is not a reason for applying for Western Australia Government funds. The benefits are local and/or likely to be short term.

1

Moderate The environmental benefits in this area are limited in scope or apply to a limited number of stakeholders. They do not justify the application on their own.

2

Significant The environmental benefits in this area apply to a significant region and/or several stakeholders and are likely to be long term. They are not sufficient to justify the application on their own.

5

High The environmental benefits of the application are significant and widespread and enjoyed by many stakeholder groups and will be long term. They may justify the application on their own.

10

Very High The environmental benefits of the application are highly significant and widespread and enjoyed by many stakeholder groups at a national level. They definitely justify the application on their own.

20

B. Social Impact Scale

Level of Impact

Description Scale

Minor There is a small social benefit in this area, however this is not a reason for applying for Western Australia Government funds. The benefits are local and/or likely to be short term.

1

Moderate The social benefits in this area are limited in scope or apply to a limited number of stakeholders. They do not justify the application on their own.

2

Significant The social benefits in this area apply to a significant region and/or several stakeholders and are likely to be long term. They are not sufficient to justify the application on their own.

5

High The social benefits of the application are significant and widespread and enjoyed by many stakeholder groups and will be long term. They may justify the application on their own.

10

Very High The social benefits of the application are highly significant and widespread and enjoyed by many stakeholder groups at a national level. They definitely justify the application on their own.

20

C. Knowledge Non-Monetary Impact Scale

Level of Impact

Description Scale

Minor Obtaining a knowledge economy benefit in this area is not a reason for applying for Western Australia Government funds but the application will have a small positive impact.

1

Moderate The application will provide a limited contribution to the knowledge economy of participating research organisations but this is insufficient to justify the application in its own right.

2

Significant

The application will provide a significant contribution to the knowledge economy of participating research organisations and the wider research community but this is insufficient to justify the application in its own right.

5

High

The benefit to the knowledge economy in this area will provide a significant benefit to research organisations and other stakeholder groups including business and/or the wider community. It may justify the application in its own right.

10

Very High The benefit to the knowledge economy in this area clearly justifies the application in its own right. It will result in significant benefits to a wide range of stakeholders that otherwise would not be possible.

20

Page 32


Appendix G – Risk Scoring System

The delivery risk profile for each research program is derived from the applicant’s specification of barriers to success in three stages along the path to impact – output delivery, adoption and impact. The table below shows the probability of success assigned to each of six levels the applicant can choose when specifying barriers to success in any of these three stages. Note that the descriptions in this table only apply to the technical complexity barrier of the output delivery risk sub-section. However, applicants chose levels for the other three barriers in this sub-section, and for the barriers in the adoption risk and impact realisation risk sub-sections, based on similarly qualitative descriptions. Magnitude of

Barriers Description

Probability of Success

Negligible This is a negligible risk for the project. 100%

Low Low level of technical complexity in R&D. Simple interacting elements that all research partners deal with regularly.

97%

Minor Low to medium level of technical complexity in R&D. Multiple interacting elements that some research partners have demonstrated on a number of occasions that they can deal with.

92%

Moderate Medium level of technical complexity in R&D. Multiple, complex, interacting elements, most of which some research partners have demonstrated on a few occasions that they can deal with.

88%

Major

Medium to high level of technical complexity in R&D. Multiple, complex, interacting elements, a few of which at least one research partner has demonstrated they can deal with. Most elements, however, are new to the majority of research partners.

73%

Significant High level of technical complexity in R&D. Large number of complex interacting elements outside of research partners’ experience.

65%

Documents

APPLICATION EVALUATION FRAMEWORK REPORT