View
442
Download
3
Category
Preview:
DESCRIPTION
Director of Research Solutions Teresa Clarke shares a presentation on research outcomes for SmartSimple's Community Conference 2014.
Citation preview
Assessing Research Impact
Teresa Clarke PhD
Director of Research Solutions
Serkan Bilgi MBA
Senior Business Analyst
Objectives
• Ways you can use your SmartSimple system to measure the impact of your research projects.
• Learn how to capture research metrics and KPIs to maintain stakeholder engagement.
Agenda
• Motivation of Research Funding Agencies• Gathering Key Performance Indicators (KPIs)
– Classes and Demo
• Utilizing Information– Portfolio analysis for Governance– Streamlining reporting for Researcher– Identifying success for Public
• Conclusions
Motivation of Funding Agencies
Funding Agency
Governance• ROI and
Outcomes• Governance• Accountability
Researcher• Simple
reporting• Award renewal
Public• Media• Education• Entertainment
Differences between RMS360⁰ and GMS360 ⁰
• Research funding agencies award large amounts of funds over multiple years ($1-5M over 4 years).
• Increased frequency of progress reports (annual or more frequently).
• Increased detail of monitoring.
Performance Measurement
• Reporting requirements are included in the award contract.
• Progress reports are required to document grantee accomplishments and compliance with terms of award.– Describe scientific progress– Identify significant changes to project plan– Describe plans for the next budget period– Report on personnel– Number of accomplishments
Measurable Impact
• Driven by the program deliverables.• Short term objectives:
– Job creation– Retaining international caliber talent– Building research infrastructure
• Long term goals:– Public policy and services– Economic development– Developing new therapies
Program Deliverables• Fits within the strategic plan of the agency.• Private: Industry or Charity
– Specific research area or to generate knowledge for the public good
• Public: Government– Reflect national research priorities
Captured in SmartSimple on the Program
Level.
Project Milestones • Relate to the program deliverables.• SMART
– Specific, Measurable, Attainable, Realistic and Timely
Captured in the project application form and progress reports.
Key Performance Indicators (KPIs)
• Job creation• Publications
– High impact journals– Other products to government, communities of interest and the public
• Additional investment or partnerships– Leveraged Funding– Partner funding
• International caliber performance– Awards, collaborations, conferences, committees
• Commercialization• Protection of Intellectual Property (IP)
Captured as individual records on progress report.
Measuring Impact using KPIs
• Live demo of Research Metrics in SmartSimple.
Portfolio AnalyticsType Indicator Comments
Level of Funding Summary Overview containing the amount of funding and distribution across CanadaDistribution across burden of disease
Breakdown of awarded amount of funding by disease area in each provinceSum of funding by provinceBreakdown of number of projects and amount of funding by province only
Competition Summary Number and range of targeted funding competitionsNumber of applications in each competition
Breakdown of number of funded projects by disease area in each competitionSum of funding per competition
Team count per competition
Number of teams funded in each priority area
Breakdown of count of teams in each disease area in each competitionSummary count of number of teams in each competition
Performance Metrics Number of publications, presentations and awards, etc.
Count of performance metrics (number of publications, presentations and awards, spin-offs etc.) per projectSummary count of each performance metric
Publication impact Number of significant research advancements
Count of high impact journal articles per project(list of high impact journals to be provided by Brain Canada)Sum of articles per journal
FTE count Number of supported personnel List of FTEs and HQPs per projectSum of FTE and HQP count per position
Streamlined Reporting
• Information Standards• CASRAI - Consortia Advancing Standards
in Research Administration Information http://casrai.org/
• euroCRIS http://www.eurocris.org/ establish metadata standards (CERIF)
Governance and Audits• Accountability• Return on Investments (ROI) in key areas
– Geographical location– Distribution in disease area
• Identify trends in research• Project future needs
Generate informative reports and dashboards
Dissemination of Information
• Promote scientific breakthroughs• Build collaborative environments
– Networks of researchers
• Knowledge search tools– Researcher Databases– National Surveys and Census
Promote research investments using SmartSimple tools or connectors.
Predicting Research Success
• Most funders use a mixture of quantitative and qualitative criteria during the peer review process.
• A “success” story is qualitative mix of public interest and funder priorities.
• Identify attributes in a research proposal that could contribute to desired outcomes.
Using Janus to connect research outcomes to potential projects.
Future of Research Funding
• Align with strategic plan and priorities of stakeholders.
• Predicting “success” of funded research by anticipating outcomes.– Definition of specific metrics important at beginning of
funding cycle
• Tailor RFA, select projects with criteria indicative of research performance.
• Ensure continued funding to research.
Questions?
Recommended