of 28 /28

2012 Avoca Quality Summit Report

Embed Size (px)

Text of 2012 Avoca Quality Summit Report

Page 1: 2012 Avoca Quality Summit Report



Operationalizing Best Practices in Proactive Quality Management

Page 2: 2012 Avoca Quality Summit Report

“Never underestimate the power of a small group of committed people to change the world. Indeed, it is the only thing that ever has.”

Margaret Mead

The Avoca Quality Summit, held May 1-2, 2012, in New Brunswick, NJ, was the official kick-off of The Avoca Group’s Quality Consortium, a cooperative effort that brings together quality, outsourcing, and operational professionals from member pharma, biotech, and CRO organizations to accelerate the development of a best-practices approach to quality management and CRO oversight. To date, the Avoca Group’s Quality Consortium includes 27 members: 16 from pharma/biotech companies and 11 contract research organizations (CROs) (see Table 1). The corporate sponsors of the Quality Consortium are Eli Lilly & Co. and Pfizer Inc.

PharmaNet/i3 was the corporate sponsor of the Quality Summit.

Page 3: 2012 Avoca Quality Summit Report


CONSORTIUM MEMBERS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .MISSIONS AND GOALS OF THE QUALITY CONSORTIUM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .HISTORY OF THE CONSORTIUM: Opening Windows of Opportunities . . . . . . . . . . . . . . . . . . . . . . . . .

2012 Avoca Consortium on Quality: Quality Delivered by CROs (Sponsor Data) . . . . . . . . . . . . .THE AVOCA QUALITY SUMMIT. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

SUMMIT DAY 1: First Steps Toward Creation of a Quality Agreement Template and Quality Metrics Document . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

SUMMIT DAY 2: Exploring Issues Around Quality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .ROUNDTABLE 1: Documenting Expectations for Quality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Does quality drive price up? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Defining quality and articulating expectations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Empowering individuals to reach for quality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Is having standards practically achievable? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .The elephant in the room . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .The burning platform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Summing it up - Roundtable 1: Issues and suggested actions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

ROUNDTABLE 2: What are the Considerations when Implementing Comprehensive Metrics Programs? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Measuring quality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Quality is hard to define . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Pooling data to achieve a standard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Controlling the “perverse side of metrics” . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Deciding to share information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Summing it up - Roundtable 2: Issues and suggested actions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

ROUNDTABLE 3: Ensuring High-Quality Oversight in Global Trials in Emerging Regions . . . . . .Quality over cost . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Parallels to “Moneyball” . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Summing it up - Roundtable 3: Issues and suggested actions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

ROUNDTABLE 4: What Does the Future Hold for Sponsor/CRO Strategic Partnerships? . . . . . .Will developing more consistent and agreed-upon approaches to quality agreements and metrics

lead to more satisfaction and better performance? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .The implications for CROs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .CROs: Learn to come to the table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Which business models for sourcing are best for quality improvement? . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Summing it up - Roundtable 4: Issues and suggested actions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

CONCLUSION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .NEXT STEPS: The Future of the Avoca Quality Consortium . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .






Page 4: 2012 Avoca Quality Summit Report

The Avoca Group Inaugural Quality Summit: Establishing a Best-Practice Approach to Quality

Management and CRO Oversight. 1


Pharma, Biotech, and Device Companies Clinical Service Providers

• Alexion Pharmaceuticals• Amgen• Astellas Pharma• AstraZeneca• Biogen Idec• Bristol-Myers Squibb• Cerexa• Cubist Pharmaceuticals• Eli Lilly & Co.• GlaxoSmithKline• Grünenthal• ImClone Systems• Janssen Research & Development• Pfizer• Purdue Pharma• Roche

• Chiltern International• Covance• Harrison Clinical Research• INC Research• ICON• PAREXEL International• PharmaNet/i3• PRA International• Quintiles• RPS• Theorem Clinical Research


Consortium members have a common mission: (1) to optimize their individual companies’ approaches to quality management within a short time frame, through collaboration with other Consortium members and The Avoca Group, and (2) to effect change overall in the industry by creating a new paradigm and standard for managing the quality of outsourced trials and programs.

The Quality Consortium’s short-term goal is to create standards for Quality Management, with a focus on Quality Agreements and Quality Metrics. The group’s long-term goal is to develop a new paradigm in the approach to Quality Management and partnering with CROs to ensure high quality and mitigate risk while introducing efficiencies and potential cost savings. Members of the Consortium will work together to establish mutually agreed-upon Quality Management standards in order to strengthen partnerships and reliability throughout the industry.

Page 5: 2012 Avoca Quality Summit Report

The Avoca Group Inaugural Quality Summit: Establishing a Best-Practice Approach to Quality

Management and CRO Oversight. 2

HISTORY OF THE CONSORTIUM: Opening Windows of Opportunities

The Avoca Group’s 2011 Report on Quality and CRO Oversight identified a number of thought-provoking issues and “gaps” between sponsors and CROs in the area of quality. The data from 2011 also identified areas of opportunity, particularly in the development of proactive approaches to quality management. Avoca formed the Quality Consortium to address challenging areas head-on and to develop more consistent approaches to the proactive management of the quality of outsourced trials. According to Patricia Leuchten, Avoca’s President and CEO, “We know that sponsor companies are striving to become more efficient in the oversight of CROs and to reduce the duplication of effort while focusing on maintaining very high quality.”

2012 Avoca Consortium on Quality: Quality Delivered by CROs (Sponsor Data)Over the last 3 years, the quality delivered by our CROs has been*:

Eliminating duplication of effort requires collaboration on a higher level, and Avoca sees its role as a catalyst for moving things forward. While there is no “one-size-fits-all” formula for oversight that can be applied to all organizations, Ms. Leuchten believed the time was right for the formation of a consortium to work together to bridge gaps and to develop mutually agreed upon standards for quality.

Avoca is using its consultants, subject matter experts, and partners such as the Metrics Champion Consortium (MCC) to move things forward quickly. “The Consortium and Summit are opportunities to improve outcomes through collaboration in Quality Management and metrics,” Ms. Leuchten says. “It is a forum to share industry best practices and establish new standards, and it represents a significant opportunity - because we have the industry’s leaders together.”

Better than the quality delivered by our in-house teams.

The same as the quality delivered by our in-house (sponsor) teams.

Worse than the quality delivered by our in-house teams.

*Data from the 2012 Avoca Consortium on Quality

N = 167



Page 6: 2012 Avoca Quality Summit Report

The Avoca Group Inaugural Quality Summit: Establishing a Best-Practice Approach to Quality

Management and CRO Oversight. 3


Consortium members attending the Quality Summit engaged in interactive discussions and collaborated on how best to implement robust quality programs in their organizations to improve the quality and outcomes of outsourced clinical trials. The Summit was the first of what will be several opportunities to collaborate on the implementation of new standards. In short, it provided a forum to find proactive, rather than reactive, approaches to the fundamental challenges the industry is facing.

SUMMIT DAY 1: First Steps Toward Creation of a Quality Agreement Template and Quality Metrics Document.

On Day 1 of the Quality Summit, 50 sponsor pharma/biotech members of the Avoca Quality Consortium reviewed new data from Avoca’s 2012 Quality Consortium Assessment Sponsor Survey and discussed the creation of Quality Agreement template/standard that applies to many outsourcing models. A draft template that was created by Avoca based on a compilation of sponsor companies’ individual Quality Agreement templates was reviewed and discussed. (CRO members of the Consortium are now in the process of reviewing and providing comments on the second draft of this template.)

Consortium members working on the Quality Agreement template/standard decided that a more streamlined template was needed to make it easier to move toward an industry standard. It was determined that functional details should be omitted from the standard template since this language typically appears in other types of documentation such as monitoring and data management plans. The language and text specific to functional areas will be preserved in a separate repository of information so that members have access to the specific language if they need it. Thus, there will be references to these details in the Quality Agreement.

Consortium members also worked on the creation of a Quality Metrics document, creating a list of 12-15 essential metrics focused on measuring quality.

In advance of the Quality Summit, sponsor Consortium members were asked to submit their quality metrics to Avoca. These were reviewed and consolidated prior to the Summit. “We were provided with cycle time and performance metrics and some relationship metrics,” noted Lisa McKay, Avoca’s Senior Director of Relationship Management Programs. “We also saw a great variation in what types of metrics people are using, so it was obvious that there is a real need to develop standards and really assess and address how the industry is defining quality metrics.”

Cycle time metrics and performance metrics not directly related to monitoring of quality were taken out of the final mix, leaving 116 metrics that were provided to sponsor members of the Quality Consortium as a starting point for their work.

Page 7: 2012 Avoca Quality Summit Report

The Avoca Group Inaugural Quality Summit: Establishing a Best-Practice Approach to Quality

Management and CRO Oversight. 4

The 116 metrics fell into 3 categories:

• Quality of operations in outsourced trials (or in-sourced trials);• Oversight of investigative sites;• Quality of response to findings.

Using these 116 metrics, each individual was asked to select the top 12-15 that would become essential to measuring quality in clinical trials. “There is still work to be done,” Ms. McKay said, “but we did see a great deal of consensus on some aspects.” Avoca’s next step is to get input from CRO members and then to work closely with the MCC to integrate the quality metrics into MCC’s system of organizing metrics by work streams.

The Quality Metrics document is still a work in progress, but the top 3 metrics that emerged at the end of Day 1 were:

1. Number/percentage of repeat audit findings or issues after corrective and preventive actions (CAPAs) have been implemented;

2. Percentage of personnel passing Good Clinical Practice (GCP) compliance and CRO/sponsor Standard Operating Procedure (SOP) testing prior to being assigned to the project;

3. Percentage of significant quality issues as defined in the Quality Management Plan (QMP), escalated to the sponsor within a defined timeline.

SUMMIT DAY 2: Exploring Issues Around Quality.

Day 2 of the Quality Summit, open to Consortium members and non-members, featured four roundtables focused on 1) documenting expectations for quality, 2) measuring quality, 3) exploring how to ensure high-quality and proper oversight in emerging regions, and 4) strategic alliances and how partnering has an impact on quality.

The message permeating the day’s roundtables and presentations was the need for collaboration and cooperation at the highest possible levels of the industry. “We’ve got a collective problem that impacts our jobs, our abilities to get medicines to patients,” said Jeffrey Kasher, PhD, Vice President and COO of Global Medical R&D at Lilly. “We as individuals and individual organizations can’t solve this problem-it’s bigger than any one of us. But I’m convinced that if we work collectively, we can solve the problem and move the needle forward.” Dr. Kasher echoed what Patricia Leuchten noted earlier about Avoca’s goal in launching the Consortium and Summit: “It starts with a group of people driving change; then the rest of the folks come along. We have to drive change. If we’re not ready to do that, then the cures for Alzheimer’s disease and cancer are going to come from some other new industry, not ours.”

Page 8: 2012 Avoca Quality Summit Report

The Avoca Group Inaugural Quality Summit: Establishing a Best-Practice Approach to Quality

Management and CRO Oversight. 5

As if to underscore that very criticism, in his introductory remarks, Jeffrey Kasher asserted, “As an industry, we like to come together and talk about problems but not act.” He urged everyone to make that statement untrue.

Jeffrey McMullen, President and Chief Executive Officer of PharmaNet/i3, and John Hubbard, PhD, FCP, Senior Vice President and Worldwide Head Development Operations of Pfizer Inc., also spoke forcefully in their introductory remarks about the need to work together to change the industry so that it meets the challenges facing it. “We’re all confronted with increased complexity of clinical trials, increased regulatory scrutiny, increased globalization of trials-real challenges we all need to address successfully,” Mr. McMullen said. “We’re in this together. We’re all professionals. We share a common desire to produce the highest possible quality in the work that we do and in the products that we ultimately bring to market.”

Whatever side of the equation you’re on, pharmaceutical/biotech or CRO, there is incredible pressure. The need for collaboration between sponsors and providers has never been greater. “This industry has been a target for just about everybody,” John Hubbard noted. “We will not be successful if this is just a biotech or just a pharma initiative. It has to be an industry initiative that includes our partners. This has to be a collaboration.”

Jeffrey Kasher used starker terms to explain why the time to bridge differences, look for common ground, and harmonize to get to some common solutions is now: “We have a big problem: the industry as we know it and love it is not viable anymore. We are facing death. That is just a reality. We are too expensive, too slow, and we’re not bringing forward medicines that add enough value to society.” While there’s no magic bullet to turn that around, he said there are several key levers that can be used to retain the industry or transform the industry going forward. Quality is one of those levers because focusing on quality can increase speed, drive down costs, and increase the value of the products/medications companies are working on.

The fundamental importance of developing definitions and standards for quality was underscored in all the roundtable discussions held at the Quality Summit. “Quality is clearly on the agenda and maybe even in the crosshairs of the FDA and other regulatory bodies,” said Jeff McMullen. “It is the responsibility of our management team to identify the objectives around quality, to build employee commitment to it, to train our employees and investigators relative to that commitment, and, most importantly, to employ continuous process improvement initiatives.” His mission is to maintain a “culture of quality”. But defining quality is much easier said than done. “Quality includes the obvious tangibles, such as things that we measure around deliverables,” said Mr. McMullen. “But there are also the intangibles, which I frankly think is what’s behind some of the data Avoca presented with respect to different perceptions about quality reported by CROs versus Pharma/biotech.”

“It’s all part of what I call the Universe of Quality-both tangible and intangible,” Mr. McMullen explained.

Getting to quality as a culture is a journey, Jeff Kasher acknowledged. At Lilly, there is a robust quality system in place: integrated standards, business processes, clear governance, management oversight, and a robust organization, which ensures the maintenance of the quality system. Lilly has also put a big focus

Page 9: 2012 Avoca Quality Summit Report

The Avoca Group Inaugural Quality Summit: Establishing a Best-Practice Approach to Quality

Management and CRO Oversight. 6

on oversight of outsourced work. “We’ve got documentation, quality agreements, clear metrics, leading indicators—and we are trying to work with our partners to ensure the work is in place, we’ve got clear governance, and a means of escalation of how all of that work gets done,” Dr. Kasher said.

“What’s needed now is consistency of quality standards across our organizations and industry,” he said. The only way that is going to come to pass is if work is shared and discussed with CROs. “And we can’t just talk about it. We need to come together and say, ‘This is how we’re going to do things: Quality Agreements are going to be like this across the industry, and here are the metrics we’re going to track as an industry.’ ”

“I like to use the phrase, ‘different is different, better is better; don’t confuse the two’,” said Dr. Kasher. “We have been fond of everybody doing everything a little bit differently. But at the end of the day, is that really better? Most of the time, the answer is no.”

John Hubbard reiterated the need for establishing a Quality Agreement in the very beginning of the study design process. “As much as we try to standardize processes, we’re still dealing with variability across teams,” he said. “We always hear ‘oncology is different,’ or ‘immunology is different.’ Every team feels they’re different. Sitting down with each of those teams and defining that Quality Agreement prospectively with the alliance partner helps to reduce the ‘noise’ and also the feeling that we’re not getting the quality that we want.”

Page 10: 2012 Avoca Quality Summit Report

The Avoca Group Inaugural Quality Summit: Establishing a Best-Practice Approach to Quality

Management and CRO Oversight. 7

ROUNDTABLE 1: Documenting Expectations for Quality

Does quality drive price up?

The tension between cost and quality was addressed by panelists in the first Roundtable. “There’s huge pressure to get products to market quickly, and concern over loss of exclusivity, which creates a tension that unfortunately sometimes favors speed over quality,” said John Hubbard. “I’ve made clear in my organization that quality will not be sacrificed for anything else. That’s a good thing, and the right answer, but it creates constant tension.”

Tension is caused by a lack of understanding about what the real costs are and what it really takes to deliver a project. With the advent of more outsourcing, these costs are finally entering everyone’s consciousness. “A big problem has been the industry’s tendency to commoditize things that are not commodities,” said Dr. Hubbard. “We want to negotiate good rates for activities, but we have to understand how the complexity of a study will drive up costs for certain activities even though the rates may be low.” Frequency of monitoring, for example, should be driven by what’s required for the protocol and the enrollment rate. “Take the conversation out of the cost realm and put it more into the operational delivery realm, which is where I think it belongs,” he concluded. “Then it’s a debate around what’s the best approach. Maybe there’s an approach that will still deliver quality at a lower cost, but it’s not driven by cost, per se.”

2011 Avoca Research: Causes of Issues with QualityDo you believe that there is a correlation between the demand for decreased prices by sponsors

and an overall lowering of quality by CROs?*



Don’t know

*Data from the 2011 Avoca Industry Research: Ensuring Quality in Outsourced Clinical Trials0






CROs Sponsors







93 84N

Page 11: 2012 Avoca Quality Summit Report

The Avoca Group Inaugural Quality Summit: Establishing a Best-Practice Approach to Quality

Management and CRO Oversight. 8

Defining quality and articulating expectations.

Re-work emerged as another explanation for the gap between sponsors’ and CROs’ satisfaction levels. Jorge Guerra, MD, Senior Vice President of Global Clinical Development Operations at Biogen Idec, pointed out, “Re-work has a huge cost; if we could just build in quality up front and do a much better job and eliminate re-work, we could increase quality and satisfaction.” The issue of re-work goes directly to the issue of defining quality and establishing clear expectations. “We’re going into these relationships all agreeing that we want to produce a high-quality product,” noted Jeffrey McMullen; but we don’t define quality. “We have these expectations that are not defined, and somewhere along the way we find out that the CRO is going in a direction that’s not in line with the expectation of the sponsor.” This is where re-work is required and inefficiency results. Standards, definitions, expectations, and metrics used effectively will improve quality and control costs, Mr. McMullen believes.

“Expectations differ from team to team. We can be working with the same CRO and it’s still confusing. If people don’t know what they’re shooting for and what the expectation is, they’re going to miss,” Jeffrey Kasher said. If we’re consistent and do things to a consistent standard in a repeated manner, quality will be higher and the work will be done better and more efficiently, which results in speed, reduced cost, and better quality.

Paula Brown Stafford, President of Clinical Development at Quintiles, agreed that different customers and sponsors have a somewhat different take on quality. That is why it’s necessary to first define what quality means. “If we define it as a standard, it would help everyone,” she said. “Next, we have to measure how to define quality. It’s not enough just to put in an integrated quality plan-we have to measure to that plan on a consistent basis across the teams.”

“A lack of consistent standards adds to the risk of failure,” said Jeff McMullen. If you’re working with 75 different clients and doing things 75 different ways, it’s a challenge to get everyone properly oriented to the processes they need to conduct. “If we can get industry standards across the industry around quality, we’ll minimize the need to address problems with the incredible amounts of training and oversight we all do. The more we can standardize, the more opportunity we have to lower costs because of greater efficiency.”

“We have examples where you have a highly complex therapeutic area, a highly complex protocol, and people are expecting perfection on a bargain-basement budget but aren’t prepared to make the tradeoffs in other areas to assure the quality,” John Hubbard asserted. “Those discussions need to happen very early. In the past they were just assumed, and that’s when you have a misalignment between the sponsor and the CRO.”

“Year after year our data show that the industry as a whole seems to have trouble articulating expectations,” Avoca’s Patricia Leuchten noted. “This is also true internally, within sponsor organizations; there is trouble just getting agreement internally on what the expectations should be. Is there an issue within both sponsors’ and CRO organizations concerning the staff ’s ability to know how to get clarity around expectations, communicate those expectations, and understand that when things change, there has to be another conversation?”

Page 12: 2012 Avoca Quality Summit Report

The Avoca Group Inaugural Quality Summit: Establishing a Best-Practice Approach to Quality

Management and CRO Oversight. 9

It’s human behavior, ventured Jeffrey Kasher. “Everybody’s project is the most important project and every project is slightly different. If you’re on a team, your focus is your molecule, your study, your expectations and demands, and in a large organization you have that happening in every team.” For that reason, it’s important to look at the portfolio level and agree on standards that meet needs across the portfolio. It’s the only way to get the efficiency and overarching speed of movement that we need.”

Problems arise from a mismatch of expectations-–whether that’s within a team or across a portfolio, noted Harris Koffer, President and CEO of RPS, Inc. Expectations should be identical; there should be no difference between partners—internal or external. Expectations must be aligned up front, making sure that the roles and responsibilities and accountabilities are clearly understood between the parties. “While standardizing the Quality Agreement is part of the tool, I think that you have to look at the whole process,” he said. “Quality Agreements define expectations around things that can be measured at an outcomes stage, as opposed to looking at the processes, starting with study design. But many of the challenges the industry faces begin at the scientific study design stage, not in the field.”

Empowering individuals to reach for quality.

How do companies empower individuals to reach for quality? This question was inspired by Jorge Guerra’s question, “Why does a CRO accept running a poorly designed protocol? The best thing is if someone at the CRO says they can’t run this poorly designed protocol, because it makes everyone rethink and start again with building quality into the protocol. The sites that tell us the study can’t be done are the best sites to work with! They know what they’re doing.”

For this scenario to happen, however, there has to be a culture change in the industry, said John Hubbard. CROs have to feel empowered and supported to “hit the red button”. At the end of the day, at the study team level, if the individual feels pressure to go faster and feels like he or she might have to cut corners to hit the date, they will have to believe they are empowered to say there’s a risk of having a quality issue downstream if things aren’t stopped. “That’s the cultural change that has to happen to the individual,” noted Dr. Hubbard. “All of us have quality built into our goals, our systems, but the point of failure is still that one person on a study team who’s being hammered by a project manager or clinicians or someone to hit the date.” Leadership has to be ready to delay a timeline, or a first patient visit, if it’s going to potentially have an impact on quality. Once people see that behavior was rewarded, a culture change can begin toward a focus on quality. It was discussed that CROs have to have the ability to push back on sponsors and not be replaced just because they are not in agreement with a timeline or approach. “We need to do a better job with our rewards systems,” suggested Dr. Guerra, “incentivizing behaviors that promote quality.”

Paula Brown Stafford directed some self-criticism to the CROs for their lack of proactive behavior with respect to Quality Agreements. “We providers may have failed somewhat because we have waited for pharma to ask us for Quality Agreements, and we have built a standard that we have been using. But how often when we’re going into a new relationship do we proactively say, ‘Can we put a Quality Agreement in place?’ CROs can do themselves a favor while the industry is moving toward those standards to at least use their own standards, to proactively ask for that Quality Agreement. It’s too late once you’re in the

Page 13: 2012 Avoca Quality Summit Report

The Avoca Group Inaugural Quality Summit: Establishing a Best-Practice Approach to Quality

Management and CRO Oversight. 10

middle of a program, a trial, or an FSP relationship because then you’re only reacting when you’re already in crisis mode.”

Is having standards practically achievable?

Unfortunately, the history in the industry is that companies are not adept at coming together to reach broad agreement. But several Roundtable participants asked why should each company have a separate set of SOPs? Why not have one uniform set of SOPs for all of the functions?

“The big fallacy in our industry is that this is proprietary,” John Hubbard asserted. While nobody shares SOPs because they’re considered proprietary, in fact most of the work is driven by GCP and ICH, he said. “There may be variations on the themes, but the basics are all the same.” Standardization will make it easier for the CROs to deliver better value because they do not have to recreate things from the differences that we have across the industry. “I think everyone is motivated to do this,” Dr. Hubbard said. “We’re doomed as an industry if we don’t get this fixed. It’s not a sustainable model. Costs are continuing to go up and we have to find a better way to be more efficient and reduce unnecessary complexity.”

Harris Koffer agreed that there’s no reason for SOPs to be proprietary. “Frankly, there’s nothing proprietary about investigator selection either, other than the fact that companies don’t want to share that information,” he said. “Because we don’t share it, it takes longer to start trials and longer to finish trials, and that will be the case until companies get together at the senior level and say there will be one set of SOPs across the industry.”

“We suffer in this industry from our inability to get standards-data standards or quality standards. We will only move forward if we take the time to have the discussions and get industry-wide agreement on standards,” said Jeffrey Kasher. Yes, we have relationships inside companies, but now we’re working in a more networked world, in partnership with many CROs, and we often don’t take the time up front to define what the fundamentals are. It’s the changing rules, constantly changing landscape, that causes problems. “If we can define things and adhere to them, we’ll make progress,” he said.

“This Summit is an opportunity for the industry to get ahead of the regulators and agree on something that is going to become an expectation in the near future from every agency around the world,” said Jorge Guerra. “Everything starts with a small group,” he said, “That’s why we are here today.”

The elephant in the room.

One of the obstacles to achieving a high level of cooperation was brought into focus by Paula Brown Stafford. “The elephant in the room is that sponsors compete on products, but CROs compete on our people, our processes, and our systems. I do believe that we have to standardize at some level to ensure quality, but you have to think about time, cost, and quality being what we compete on. How far do we come toward one set of SOPs for the industry whereby there’s no advantage between us? At the same time, we do want a healthy industry; we want competition between ourselves to become better partners with you.”

Page 14: 2012 Avoca Quality Summit Report

The Avoca Group Inaugural Quality Summit: Establishing a Best-Practice Approach to Quality

Management and CRO Oversight. 11

There’s another way of looking at that competitive advantage, however, said Harris Koffer. “The advantage is not from the SOP; it’s how you implement and apply it around those functions, the experience of your people, how well they’re trained, how well they execute around the SOPs.” The hurdle we have to get over is the failed notion that the nonstrategic aspects of the business-not the innovative aspects of design and how you commercialize the product—but the bread-and-butter execution of these important but nonstrategic elements of the business are still viewed as proprietary.

The concern about compromising individual companies’ competitive advantages by sharing SOPs and other formerly proprietary materials is not an isolated one. “When we started talking to companies about the Quality Consortium, some of the executives from pharmaceutical companies said they’d spent years developing some of these processes and had put enormous time, energy, and resources into them,” said Patricia Leuchten. They asked why they should hand them over to the Avoca Quality Consortium so that others can be better. “There are some hurdles to get over in terms of moving toward industry standards, and it is incumbent on us to choose what are the high priority activities where everyone benefits-sponsors and CROs,” she noted.

There are people in the industry that “don’t get it”, said Jeffrey Kasher. This movement to collaborate and create industry standards hasn’t happened yet because, quite simply, everyone has been able to do things differently and hold their cards to their chests. “Because we could afford to,” he said. “Nowadays, we can’t. I’ve seen a definite change in the last 2-3 years, where it’s still too much talking versus doing, but people are finally coming together and putting their cards on the table.”

An audience member, Dr. Francisco Harrison (founder of Harrison Clinical Research), expressed disappointment that the industry tends to commodify quality. “Quality is a result,” he said. “I totally agree with Jeffrey Kasher: if we don’t change this system, we are going to go down like the Titanic! We have to remember that on the other side of the table is not the sponsor or the CRO. On the other side of the table there is disease, death, and people who are suffering from the consequences of disease. This is why we have to have quality: for the patient. If we understand this message and come to the same side of the table, we will solve the problem.”

The burning platform.

Productivity has declined to one-fifth or one-sixth of what it was 15 years ago, according to Harris Koffer. “That is not a sustainable platform, it’s a burning platform,” he said. The end result of a burning platform is that the patient doesn’t get the drug in a timely manner, but the term “burning platform” also suggests the unsustainability of an industry that has had productivity decline at the rate it has. “When you think of how much training goes on because each company does it a different way and because each company has a different way of operating, and how monitors are trained for sometimes three months, think of how much time, effort, cost, in that one area alone could be saved,” Dr. Koffer concluded.

Responding to “burning platform” imagery, Jeffrey Kasher ventured the thought that the industry is wearing very well-insulated shoes. “For an industry that is pretty close to death—and I don’t think that’s much of an exaggeration-most people aren’t feeling the heat,” he said. “Look at the economics of the business. Are enough people going to realize that?”

Page 15: 2012 Avoca Quality Summit Report

The Avoca Group Inaugural Quality Summit: Establishing a Best-Practice Approach to Quality

Management and CRO Oversight. 12

Summing it up - Roundtable 1: Issues and suggested actions.

In Roundtable 1, many issues were identified as industry-wide problems that require creative solutions:

• The constant tension between quality and cost can favor speed over quality.• There is a lack of understanding about the true costs of high-quality clinical trials.• Too much re-work is required—largely the result of poor communication, a lack of clarity, and failure

to define quality.• There is a lack of consistent standards.• Many communication problems are the result of poor clarity of expectations.• The present industry business model is not sustainable, as is demonstrated by declining productivity.

The business model relies on a failed notion that nonstrategic aspects of the business are proprietary.

An important part of the solution for many of these issues is the governance of quality and ensuring that standards are implemented as part of this governance. Specific suggestions included:

• Define and build in quality from the very beginning—starting with protocol design. • CROs should be involved earlier in the process and given more input to the process.• Greater clarity and specificity about communicating expectations. • Standards, definitions, expectations, and metrics used effectively will improve quality and control costs.• When changes occur, have new conversations.• Avoid re-work by doing it right the first time—a more likely scenario if quality was built into the

protocol from the start.• Look at the portfolio level to realize greater speed and efficiencies.• Build quality into every individuals’ goals and objectives, at every level of the company. Then measure

it and reward it.• Empower individuals to feel they can “hit the red button” when something’s not right—a true culture


Page 16: 2012 Avoca Quality Summit Report

The Avoca Group Inaugural Quality Summit: Establishing a Best-Practice Approach to Quality

Management and CRO Oversight. 13

ROUNDTABLE 2: What are the Considerations when Implementing Comprehensive Metrics Programs?

Measuring quality.

Jonathan Lee, VP of Development Operations at Cerexa, said that because he works with smaller companies it’s typically hard to get traction with senior management in terms of setting up metrics. An important consideration is figuring out what the “maturity level” of a company is and whether they are ready to engage in metrics. Cerexa aims to “breed” metrics into their system by negotiating metrics up front. “There are 14 metrics written into every CRO contract,” he said. “Our CROs sit in protocol design meetings. We pay them separately for feasibility. We only dictate the timeline and number of subjects and ask where in the world we should go to do the study.” Feasibility is codified in the contract and there are bonuses and penalties associated with the work. Each month, those 14 metrics are reviewed.

Hazel Collie, VP of Global Clinical Quality Assurance and Compliance at Grünenthal GmbH, also works with small CROs. “In the past, we’ve done some metrics exercises, and realized it didn’t tell us much. Because we’re small, we have months with no audit reports, no throughput, so that’s a real issue for us. We want to measure, but what does it tell us? Can we compare what we do in-house to what the CROs do? We’re getting there but we’re not there yet.”

At Amgen, however, which completes about 275 studies a year across different therapeutic areas and countries, robust metrics are very important. Dr. Elspeth Carnan, Amgen’s Executive Director of Global Clinical Site Management, explained that because of the scope of their work, 11 different providers are used. “The metrics really have to mean something,” she said. Any effort to create a Quality Metrics industry-wide document will require a tremendous effort to narrow down the 116 metrics Avoca consolidated to a few that mean something. “We have to be able to ensure we can measure them and have appropriate targets set,” said Dr. Carnan. “The metrics have to be easily measured, easily transmitted in our systems, and timely.”

Providing the CROs’ perspective, Angelika Tillmann, Clinical Research VP of Global Clinical Quality Assurance and Compliance at Theorem Clinical Research, said there are different aspects to consider. “Because you will have to have metrics directed toward the study, they will be different for each sponsor,” she said. “We are still struggling to find metrics that mean something.” It’s easy to check certain metrics off the list, Ms. Tillmann acknowledged, when they relate to performance rather than quality.

Quality is hard to define.

Why has the industry struggled to create quality metrics? It’s simply easier to define speed or performance or cost metrics, but quality is hard to define. “That’s why it’s been challenging, but now we’re coming to the point where we are starting to see some consistency across some of the definitions and we’re getting more data around errors and training,” said Dr. Carnan.

Page 17: 2012 Avoca Quality Summit Report

The Avoca Group Inaugural Quality Summit: Establishing a Best-Practice Approach to Quality

Management and CRO Oversight. 14

“We monitor audit reports. We build quality metrics into our score cards (Key Performance Indicators) and our expectations are clearly articulated to our providers,” said Dr. Carnan. “They are told that we measure these metrics and monitor them; we expect action plans if we go off track against these metrics. We dive deep into causality of issues, as far as audits findings, because it’s important to have a deeper understanding of what goes wrong.” Amgen wants to reduce that type of audit finding and they hold people accountable at all levels of the company.

At Theorem, risk-assessment meetings are conducted to examine the risk of the protocol, and to get the sponsor involved early on to make them aware that previous experience in similar studies showed that a particular aspect of the study has certain risks attached. “We all need to think in these terms, of drawing closer together and the boundaries disappear a bit,” said Ms. Tillmann.

Quality metrics are particularly difficult to use as leading indicators. Cerexa uses a composite metric that they’ve created; the subject has to be evaluable. There are 6 criteria. “We set it as a key in our service level agreements. For individual subjects it is a lagging indicator, but it becomes a leading one as the subject goes through the study,” Jon Lee explained.

“Amgen uses audit reports as a lagging indicator. Audit reports reflect after-the-case. If you see trends, it can be used for risk evaluation. You have to be able to gather enough data to know what are the triggers to complex issues that are going to lead to more errors,” Dr. Elspeth Carnan said.

2012 Avoca Consortium on Quality: Use of Quality Metrics (Sponsor Data) In general, do you feel that the quality metrics you use are adequate…













in terms of scope (sufficient number

of quality metrics)?

in design (i.e., provide meaningful reflection of actual quality delivered)?




*Data from the 2012 Avoca Consortium on Quality

183 181N

Page 18: 2012 Avoca Quality Summit Report

The Avoca Group Inaugural Quality Summit: Establishing a Best-Practice Approach to Quality

Management and CRO Oversight. 15

2012 Avoca Consortium on Quality: Use of Quality Metrics (Sponsor Data) Do the teams with which you work use quality metrics to evaluate the performance of clinical service providers

with respect to the quality of services and deliverables?*

Pooling data to achieve a standard.

The idea of pooling data to achieve a standard was raised. Elspeth Carnan noted that Amgen operates close to a “CRO placebo” model because as an FSP model with all their CRO providers operating in their systems and processes, it normalizes how the CROs are working. “We do measure our metrics against each other and we show them back to the providers,” she said. “I think that they find it very interesting and I think it helps them understand where they have deficiencies. And that’s what we’re in this for—to get patients medications quicker.”

0 20 40 60 80 100















































*In this report, selected results are presented for each consortium member company. In the slides containing these “benchmark” comparisons, companies are designated by letter, in alphabetical order based on mean rating for overall satisfaction with the service delivered by their CROs. Thus, respondents from company A were on average most satisfied in this respect, and respondents from company N were on average least satisfied.

No *p<.0001, company effect / p=ns, function and level effects




Page 19: 2012 Avoca Quality Summit Report

The Avoca Group Inaugural Quality Summit: Establishing a Best-Practice Approach to Quality

Management and CRO Oversight. 16

Controlling the “perverse side of metrics”.

John Oidtman, VP of Clinical Trial Support and Compliance and Clinical Operations at Pfizer, asserted that metrics drive behavior. He asked the panel, “How can companies control ‘the perverse side of metrics’ to drive the right behaviors?”

Jon Lee suggested that companies need to measure “soft skills-leadership, judgment, ethics.” At Cerexa, the soft skills are part of the interview process as well as employee performance evaluations. “Employees see that gaming metrics gets you nothing,” Mr. Lee said. “We also have to shift who we are hiring-away from ‘doers’ because it’s a different skill set required for managing. It’s more about how do you form/influence relationships without direct control. So the paradigm has to shift.”

Deciding to share information.

“You have to be careful about what you share, but I do think the sites would perform better and they would welcome feedback on whether they’d performed well,” said Elspeth Carnan. “As an industry, we need to come together and make a decision about that,” agreed Hazel Collie. “In Europe, the inspectors share that information, so why as an industry shouldn’t we be sharing that information?”

“Speaking honestly, a lot of information is de facto shared,” said Jon Lee. “I know what goes on; we all know what goes on. I have my network of people, and we know things through that network. What’s wrong with sharing on a wider basis, with anonymized data? I want to know when I go to a site what things have happened there before.

Summing it up - Roundtable 2: Issues and suggested actions.

In Roundtable 2, the main issues that were identified revolved around the difficulty inherent in measuring quality. In addition, the following issues were addressed:

• There is a lack of agreement on definitions for metrics.• Again, expectations are poorly articulated.• Can companies control the “perverse” side of metrics and drive the right behaviors?

Solutions that were cited addressed the need for the industry to come up with a core set of agreed-upon metrics that can be shared. In addition,

• Metrics must be negotiated into the system up front.• Metrics have to be easily measured and transmitted.• To promote the right behaviors, companies need to also measure for “soft skills” in their employees:

leadership, judgment, ethics.• Dive deep into causality of problems to learn from mistakes.• Hold people at all levels of the company accountable.• Share metrics with providers so they can compare themselves to others and learn from results.• Share and use anonymized data throughout the industry so the wheel is not constantly being


Page 20: 2012 Avoca Quality Summit Report

The Avoca Group Inaugural Quality Summit: Establishing a Best-Practice Approach to Quality

Management and CRO Oversight. 17

ROUNDTABLE 3: Ensuring High-Quality Oversight in Global Trials in Emerging Regions

There are many reasons why emerging regions are important to the pharmaceutical/biotech and CRO industry. Emerging regions provide access to patients, access to therapeutic areas where trials can be conducted, access to markets, and potential cost savings. Emerging markets also provide diversity, a sought-after concept in clinical trials. But diversity has its challenges when it comes to quality because consistency becomes an issue in emerging regions. According to Dr. Imad Alsayed, VP and General Manager - Americas at Covance, these challenges include:• Infrastructure issues• Standard of care differences• Language issues; cultural issues around disclosure and informed consent• Unstable regulatory environments, changeable political environments• Uneven investigator experience and differences in adherence to GCP• Access to talent• Complex start-ups• A greater need for relationship building

To manage and mitigate these challenges, sponsors must have a well-thought-out and methodologic strategy. “It should not be, ‘I have a trial, let me go and place a CRA,’” said Dr. Alsayed. He recommends the development of a 3- to 5-year plan that builds on local leadership. Especially with smaller biotech and pharma, it’s necessary to collaborate with the CRO on the protocol and its operationalization.

Quality over cost.

Quality has to be foundational in the company’s strategy. “If quality is done right, it will not add cost; on the contrary, it will reduce cost,” said Dr. Alsayed. “And it will accelerate the timeline. All the re-work is where we start spending money.”

Quality should be the basis of site selection in emerging regions. At Covance, there is internal tracking on the history of the site’s quality, using feedback from local affiliates. If there are doubts about site quality, the site will not be selected.

In cases where there is not enough infrastructure support, some CROs offer flexible resources, like a study coordinator to come to the site, which is very valuable to ensure quality. GCP training is critical for site staff as well.

Pfizer’s John Oidtman acknowledged Pfizer’s learning curve with respect to quality and running clinical trials in emerging regions. “We had done the low-cost type of strategy where we were trying to drive down operational costs,” he recounted. “It didn’t necessarily drive speed and quality. It didn’t give us the quality we wanted; it gave us low cost.” The company took a step back and, from an organizational perspective, asked what it was trying to achieve and why it cares about emerging markets.

Page 21: 2012 Avoca Quality Summit Report

The Avoca Group Inaugural Quality Summit: Establishing a Best-Practice Approach to Quality

Management and CRO Oversight. 18

“We care because we want to bring our molecule to more patients and penetrate markets we don’t ordinarily penetrate,” said Mr. Oidtman. “If you’re looking at cost, you’re probably looking at the wrong measure.” He recounted how Pfizer saw the net present value of one of its molecules increase by $200 million over the patent expiry region of that drug because of the addition of one country to its market. Saving a couple of hundred thousand dollars pales in comparison with that sales figure, said Mr. Oidtman.

“It’s time to change the perceptions and misconceptions that quality is bad in emerging regions,” said Dr. Alsayed. “In a lot of cases, emerging regions are providing the highest quality, and we can’t generalize that one bad site makes the whole country bad.”

Parallels to “Moneyball”.

Lilly believes it has a successful model in China, where the company’s CRO partner is PAREXEL. The case study of Lilly’s experience in China was presented by Merry Chu, MSc, Director of Medical Quality Regional and Third Party Management.

Ms. Chu credits that success to an integrated approach to working with the CRO, quality oversight, and establishing good relationships. Lilly has an integrated approach to working in the emerging markets. Their 3-pillared goal is to be customer-centric, cost effective, and flexible in terms of realizing all of those deliverables.

The challenges in China include a lack of expertise, the need for a lot of training, and high turnover rates. Lilly’s Quality Agreement, articulated expectations, and clinical management plan, as well as a Quality Control program, audit program, and shared learning are among the elements contributing to success.

“Moneyball is a film about the manager of the Oakland A’s baseball team who utilizes statistics to select his players,” Ms. Chu explained. After she saw the film, she wondered whether Lilly is like the Oakland A’s or the Red Sox, which may not have been the first to find about the statistical model, but they won the World Series soon after adopting it. “We’ve got data, metrics, information. How are we utilizing all that? Who benefits from it? If we use and leverage data appropriately, our patients will benefit.”

Ms. Chu summed up by saying, “The real message is that we need an opportunity for change, to share our knowledge, to share what we thought was proprietary information. The important question is, who is going to make that change—those of us in this room, or will it be a room of lawyers? We need to make that decision about how open we are to sharing that information.”

Summing it up - Roundtable 3: Issues and suggested actions.

In Roundtable 3, the main issue confronting companies working in emerging regions is that diversity, while an excellent goal, has many associated challenges, such as lack of expertise, high turnover, and the need for a great deal of training.

Page 22: 2012 Avoca Quality Summit Report

The Avoca Group Inaugural Quality Summit: Establishing a Best-Practice Approach to Quality

Management and CRO Oversight. 19

The suggested solutions, based on case study analyses, were the following:• To manage and mitigate the challenges associated with conducting trials in emerging regions,

sponsors must have a well-thought-out and methodologic strategy that plans 3-5 years ahead.• Quality must be at the foundation of the strategy.• Expectations must be clearly articulated.• There must be collaboration between sponsor and CRO on protocol and its operationalization.• Remember the big picture—getting the drug to the patient—not solely cost savings.• Seize the opportunity to share what has been treated as proprietary information, and decide which

knowledge should be shared so that patients benefit and the industry continues to succeed.

ROUNDTABLE 4: What Does the Future Hold for Sponsor/CRO Strategic Partnerships?

The top 5 reasons sponsors cited in Avoca’s 2012 data for forming strategic partnerships were reduced costs, improved quality, improved efficiency in use of internal staff, access to operational expertise, and process improvement.

About half of sponsor respondents surveyed in 2012 felt that their partnerships generally met or exceeded expectations with respect to their key objectives, but a fair-sized portion wanted more out of their partnerships than they were getting. In general, clinical service provider respondents reported greater levels of contentment with their strategic partnerships than did sponsors. The top reasons cited by clinical service provider respondents for wanting to form strategic partnerships were business stability, improved efficiency in the use of staff, and increased profit.

Poor quality was the top reason cited by sponsors for discontinuing relationships, which happens in 22% of cases—an unexpectedly large number.

The data also show that in long-term alliances, satisfaction appears to build over time. Most sponsor respondents with strategic partnerships lasting over 3 years said that key benefits were realized within 1-2 years. In general, the data also showed that sponsor respondents with strategic partnerships more satisfied with the quality delivered by their providers than were those without such relationships.

The data on strategic partnerships that Avoca shared at the Consortium show that, in terms of satisfaction and quality, there is generally a benefit to developing long-standing strategic partnerships. This data was borne out in the final Roundtable discussion.

Will developing more consistent and agreed-upon approaches to quality agreements and metrics lead to more satisfaction and better performance?

“Yes, because when you form strategic partnerships, you go through a natural learning curve of working together, developing documentation together, understanding not just the expectations but

Page 23: 2012 Avoca Quality Summit Report

The Avoca Group Inaugural Quality Summit: Establishing a Best-Practice Approach to Quality

Management and CRO Oversight. 20

the organizational factors between organizations,” said Alistair Macdonald, President of Clinical Development Services, INC Research. “There’s a lot baked into the team already.”

For strategic partnerships to continue being successful, standards must be developed between pharma/biotech and CROs, said David Dockhorn, PhD, Vice President, Executive VP of Product Registration, The Americas at PRA. John Graham, Director, Head of Global Strategic Resourcing and Performance Analytics at GlaxoSmithKline, agreed that consistency is very important. “Ultimately we need to standardize how we’re working. We need to define a base level of standard rather than have each individual continue to reinvent things, because then you’re starting from scratch every time.”

While standards and documentation are indeed the first step, beyond that there has to be a sense that there’s one team providing a seamless service, which ends up with a single high-quality program, said Peter Carberry, MD, Senior VP Global Development Operations at Astellas. “We can’t throw our list of expectations over the fence and come back and have a shootout,” he said.

Steven Cutler, Group President, Clinical Research, ICON Plc., noted that CROs ought to challenge themselves more. “Setting expectations up front is critical and having a quality template will help. But, quite frankly, 3 years to start to see the benefits of partnership is too long! The industry has a real crisis right now. We need to be faster and more agile in bringing forward strategic partnerships.”

2012 Avoca Research: Comparison between Clinical Service Providers and SponsorsOverall, how satisfied are you with the work that your company has done during the strategic

partnership/that has been done for your company by the strategic partner?







Suppliers Sponsors








Very satisfied

Generally satisfied

Neither satisfied nor dissatisfied

Generally dissatisfied

Very dissatisfied

67 108N

Page 24: 2012 Avoca Quality Summit Report

The Avoca Group Inaugural Quality Summit: Establishing a Best-Practice Approach to Quality

Management and CRO Oversight. 21

The implications for CROs.

But will sharing and creating standards as an industry spell death for CROs? Will any competitive advances for the CROs end if they do not have individual standards? “What CROs have to offer is better processes and better systems, and there’s a constant push for CROs to keep raising their game to be more competitive,” noted Paula Brown Stafford of Quintiles. “If processes become more standardized, does that push go away?”

John Bell, Corporate VP, Corporate Quality at PAREXEL, agrees that it is a significant concern. “The CRO industry has developed in a very specific, very brutal, evolutionary space,” he says. “We have come up with standards and programs that work in that brutal space, and if we agree with a certain level of documentation and share content with the pharma community, we may find that we are going to become like-and I say this carefully—‘mini pharma’.” In the end, he says, CROs may become less of what sponsors need, so care needs to be taken about what level these standards are set at because it may have a negative effect on strategic outsourcing relationships.

Other industries have embraced more openness and found it increases competition and profits-for example, the software industry. Is this a model for CROs to consider? “Sure, our industry has a lot to learn from them,” Steve Cutler responded. “But I’m not entirely convinced that open source as far as our technology is something that I want to offer to competitors.” The concept of a base level of standards from which people can move up or down raised as part of the discussion and was commented on by Astellas’ Peter Carberry: “I am surprised and intrigued by the association of standards with ‘lowering’. To me, standards means doing your best, being better than average, amongst the best, so you have a chance of quality data. So our definition of standards is necessarily pretty high.” Dr. Carberry asked if the industry needs to have another conversation since we don’t agree about what standards are and to what standards we are operating. “This might explain why we’re not satisfied with each other and why there are gaps,” he hypothesized.

CROs: Learn to come to the table.

“As an industry, we are still struggling with the service delivery aspect—‘Yes sir, no sir, 3 bags full’ versus ‘This is the better way to do it so we’re going to do it this other way’,” said Steve Cutler. “I agree with you 100%,” said Mitchell Katz, PhD, Executive Director, Medical Research Operations, Purdue Pharma. “As service providers, you should come to the table and say this is what you believe should be done.”

“CROs have to change their attitudes and step up and tell sponsors we can add value and have smarter models. Our customers have to change their expectations of what a CRO can provide. The ‘3 bags full’ perception is true, but it’s changing. Let’s help design these trials. Our best practices can provide added value,” said Alistair Macdonald.

John Graham pointed out that sponsors are very risk averse, and strategic partnerships are environments where CROs can challenge sponsors and bring them value. “Innovation is needed to make it easier

Page 25: 2012 Avoca Quality Summit Report

The Avoca Group Inaugural Quality Summit: Establishing a Best-Practice Approach to Quality

Management and CRO Oversight. 22

for investigators to perform and conduct trials because the practice of medicine has changed so much globally; the amount of time that a physician can actually spend and dedicate to the patient-and also dealing with a complicated protocol and then transcribing data into the documentation—is decreasing,” noted Mitchell Katz.

Which business models for sourcing are best for quality improvement?

The panel was asked which CRO model is best in terms of reaping quality improvement. The responses tended to depend on whether the commenter was a CRO executive or a pharma/biotech executive.

CRO Executives

“The strategic partnership model gives the best opportunity—this is a very trust-based kind of relationship and the transparency and predictability of this model allows the two partners to set certain expectations, have great clarity about goals, and for people to live up to it. This is not the case in other models I have experienced; you do not have that very intertwined destiny. So in a standard setting, transparency, durability of the relationship—all of these result ultimately in the kind of relationship that has the biggest positive impact on quality,” said John Bell.

“A full service model within a strategic partnership has the best potential for innovation and driving forward technology. That’s not to say it always happens, but the potential is there more so than with an FSP where you function on one function,” said Steve Cutler.

“Our staff gets more satisfaction from working in our full service strategic models because they feel a bigger part of a bigger piece,” noted Alistair Macdonald. “In our FSP relationships, more often than not, at governance one of the first topics we talk about is where we are with cost savings for the customer.”

Sponsor Executives

“We require both models,” said Peter Carberry. “Because of the nature of drug development, we cannot operate at a single capacity level across our whole portfolio. What is successful is to approach those relationships at a strategic level, where you’re committing beyond a single program or study because the level of investment in the trust and in the standard-and the gains from not having to start all over again—can only be brought into a long-term relationship. We hope to have both models working because we have different needs. We’re going to be opportunistic, but we do want to have relationships, co-investment; we want to get better with each other because we owe it to our company and to our patients.”

“At GSK, we have both models as well: we have some FSP models and some full service,” said John Graham. “Longer term, there is potentially value that you get out of some full service models, and we are using those in areas where we don’t have a lot of experience. We’re trying to tap into the CRO to bring some of that experience in. Rather than our typical knee-jerk reaction, which in the past would have been

Page 26: 2012 Avoca Quality Summit Report

The Avoca Group Inaugural Quality Summit: Establishing a Best-Practice Approach to Quality

Management and CRO Oversight. 23

to ‘hire it’ where we don’t have a lot of experience, we’re leveraging full service. Where we do have a lot of knowledge around certain classes of assets, the FSP is more aligned to our strategy.”

“There isn’t a one-size-fits-all CRO model,” said Mitchell Katz. “We don’t have the infrastructure that a GSK does. A strategic relationship makes good sense but there are always going to be those niche providers I’m going to be dependent on for certain tasks,” he acknowledged. “I’d like to speak for some sponsors and at least for Astellas,” said Peter Carberry. “We’re not outsourcing because we want to reduce costs. When you enter a relationship, you have to ask what the driver is behind the need. For us, it’s not cost, though together perhaps we can become more efficient and it could lead to profitability. We are in these strategic relationships so we can embrace higher quality and continuity.”

Summing it up - Roundtable 4: Issues and suggested actions.

In Roundtable 4, the main issues identified for action concerned finding ways for sponsors and providers to achieve greater satisfaction with their relationships and to realize better performance. Key issues and questions were as follows:

• Will developing more consistent and agreed-upon approaches to Quality Agreements and Metrics lead to greater satisfaction and better performance?

• Will competitive advances for CROs end if they no longer have individual standards?• What CRO model is best in terms of reaping quality improvement?

Perhaps the most important conclusion to draw from this final Roundtable was that companies have entered into these strategic relationships to embrace higher quality and continuity.

• Greater collaboration is needed: CROs have to demonstrate to sponsors that they can add value and be proactive about providing intelligent solutions; sponsors have to change their expectations of what a CRO can provide.

• There is no one-size-fits-all CRO model; sponsors require the flexibility of using different models.

Page 27: 2012 Avoca Quality Summit Report

The Avoca Group Inaugural Quality Summit: Establishing a Best-Practice Approach to Quality

Management and CRO Oversight. 24


The biggest impact of the Avoca Quality Summit was the agreement on the core components of a standard Quality Agreement template and the development of a short list of essential Quality Metrics by the pharma and biotech Consortium participants on the first day of the two-day meeting. This is an important step in moving toward a global industry standard that will be adopted by both sponsors and CROs. The Quality Summit was an industry first and a success, both in the level and type of individuals that came together and in the depth of discussion. Throughout both days of the Quality Summit, it was acknowledged that, while developing standards and more efficient ways for sponsors and CROs to work together is necessary, it has been challenging historically. Given the engagement and commitment expressed by Consortium members from both sponsors and CROs, there is greater potential for success in terms of the ability to quickly move towards standards, particularly in the areas of the Quality Agreements and Quality Metrics.

NEXT STEPS: The Future of the Avoca Quality Consortium

Avoca welcomes greater involvement in the Quality Consortium and new Consortium members. We are committed to using the Consortium as a catalyst for change and action and as a vehicle for bringing together companies interested in positive change within the industry within a short period of time. Avoca believes that the keys to success in developing industry standards for proactive quality management is collaboration and senior executive involvement. We will continue to provide the vehicle and platform for collaboration and engage the executive leadership of our members to ensure that standards developed as part of the Consortium and agreed upon by members are effectively implemented. The new standards, including comments and input from CROs, will be deployed to all members in July 2012. In October 2012, the second Quality Summit will be held for members. This will be a Working Session during which the implementation of the Consortium standards for Quality Agreements and Metrics will be discussed amongst members.

Future areas the Consortium will be those that the Consortium and its members believe will have the greatest impact within the shortest period of time, and where high value can be ensured. These will be areas in which there are needs for collaboration to ensure efficiency and cost savings while focusing on high quality.

Over the two days of the Quality Summit, we heard repeated calls for action. How do we encourage an industry that has been historically slow to change to move forward in some ways that may not be comfortable at first? Circling back to the quote by Margaret Mead cited at the beginning of this piece, “Never underestimate the power of a small group of committed people to change the world. Indeed, it is the only thing that ever has.”

Page 28: 2012 Avoca Quality Summit Report

Contact Avoca at:

(609) [email protected]

179 Nassau StreetSuite 3APrinceton, NJ 08542