October 2015 | Volume 16, Number 4

Continued on page 3

We are now in the era of inexpensive data. Amazing data repositories exist, and technology can automate the capture of real-time data. Instead of simply generating reports that summarize data, businesses now develop algorithms, capable of predicting things including the future.

Corporate America is trending towards analytical decision-making. This style of management means that one collects detailed data and analyzes it to support all decisions. In the past, that has involved historical data and metrics. Organizations create reports (e.g., billed hours, active projects) and reporting metrics (e.g., profits per partner). Having

vast, diverse data warehouses allows the asking of interesting questions like “Do associates with laptops bill more hours than associates without laptops?” The answer to these sorts of questions informs decision makers and helps to ensure profitable outcomes, like whether or not to issue laptops.

We are now in the era of inexpensive data. Amazing data repositories exist (e.g., data.gov, financial data, social media), and technology can automate the capture of real-

time data. Consequently, we can collect and polish tremendous data assets, store the data inexpensively, and crunch the data with more powerful computers running analytical software. Instead of simply generating reports that summarize data, businesses now develop algorithms that help see into the future. For one business, that may be adjusting the price of an airline route on a particular day to maximize yield, and for another business, it might be determining which consumers may be most receptive to a direct mail offer to minimize cost. Seeing the future, even if it is very blurry, has obvious advantages to having perfect hindsight. One can iterate and develop a better and better vision. It is safe to say, though beyond the scope of this article, that a giant wave of productivity, value, and new opportunities will come from the predictive exploitation of data.

For a law firm, the benefits of developing and using predictive analytics now rather than years from now are both strategic and tactical. Many of the most important benefits are strategic; for example, if one accepts that some of the most valuable client opportunities of the not-so-distant future lie with

Why Lawyers Should Care about Predictive Analytics

By John Hokkanen, Risk Programmer/Analyst Contractor at First Hawaiian Bank, Honolulu, HI

Please direct any comments or questions to either of the editors in chief:

PracticeInnovations

In This Issue EDITORS IN CHIEF

2

EDITORIAL BOARD

A Day in the Life of a Pricing Professional

By Lisa Gianakos

What do law firm pricing professionals do all day? How did they get started? What skills do they have? What do they love about their jobs? And what do they struggle with?

(Probably) No Longer Asking Which Comes First – Project Management or Process Improvement

By Carla Landry

If you use legal project management (LPM) techniques, you inevitably identify ways to improve some aspects of the project as you manage work more carefully and with more tools than in the past. Both methodologies provide opportunities for improved performance.

Managing the Intelligent Global Expansion of Research Services at Squire Patton Boggs LLP

By Scott Bailey, Philip Duffy, and Kelly Underwood

Where lawyers can take advantage of time zones, handing on work around the globe, Research Services also chases the sun answering reference questions.

Why Lawyers Should Care about Predictive Analytics

By John Hokkanen

We are now in the era of inexpensive data. Amazing data repositories exist, and technology can automate the capture of real-time data. Businesses now develop algorithms, capable of predicting things including the future.

Making Project Management Work Better: Earned Value Management for Law Firms

By Don Philmlee

Earned Value Management (EVM) is a proven method and best practice used by large organizations all over the world to understand and control project success.

William ScarbroughChief Operating OfficerBodman PLC6th Floor at Ford Field1901 St. Antoine StreetDetroit, MI 48226office: 313-393-7558fax: 313-393-7579email: wscarbrough@bodmanlaw.com

Janet AccardoDirector of Library Services Skadden, Arps, Slate, Meagher & Flom LLPFour Times SquareNew York, NY 10036-6522212.735.2345email: janet.accardo@skadden.com

Sharon Meit Abrahams, Ed.D.National Director of Professional DevelopmentFoley & Lardner LLP Miami, FL

Toby BrownChief Practice OfficerAkin Gump Strauss Hauer & Feld LLPHouston, TX

Silvia CoulterPrincipalLawVision GroupBoston, MA

Elaine EganHead of Research & Information Services - AmericasShearman & Sterling LLP New York, NY

Ronda FischDirector of Research and Library SystemsJones DayPittsburgh, PA

Lisa Kellar GianakosDirector of Knowledge Management Pillsbury Winthrop Shaw Pittman LLP Washington, DC

Jean O’GradyDirector of Research Services DLA Piper, US, LLP Washington, DC

Don PhilmleeLegal Technology ConsultantWashington, DC

Kathleen SkinnerDirector of Research ServicesMorrison & Foerster LLPSan Francisco, CA

William ScarbroughChief Operating OfficerBodman PLCDetroit, MI

Janet AccardoDirector of Library Services Skadden, Arps, Slate, Meagher & Flom LLP New York, NY

Measure Better to Manage Better—Part 2

By V. Mary Abraham

A good manager will have both quantitative and qualitative metrics in their toolkit. This article explains the relative value of these metrics and provides guidance on how to collect reliable qualitative metrics that are likely to yield useful insights.

3Practice Innovations | October 2015 | Volume 16, Number 4

Why Lawyers Should Care about Predictive Analytics — Continued from page 1

companies that exploit predictive analytics, then it is obvious why understanding these technologies will be a huge asset. These technologies may affect multiple areas of law—intellectual property, privacy, labor—and a firm may benefit by being able to talk the talk and market its expertise.

Building expertise slowly in a key technology means a firm can painlessly and wisely assimilate the technology it uses on “low hanging fruit” projects. For example, one might develop a dashboard of key leading metrics for managing a big case or understanding opportunities in an industry. The point is that a real benefit arises from the increasing capacity of the firm to use these tools rather than the specific ROI that one obtains from the first project. Learning what not to do is often the basis of best practices.

We learn the details of how to make these projects happen by doing, thus giving a strategic capability to those who have done them. For example, let’s take a problem that a lot of large firms have: hiring and retention. What data and processes would it take to try to reduce the cost of hiring and increase the length of employment? What data does the firm have that might yield insight into that question? Does the data need to be summarized or transformed? A relationship observed appears to make no sense; is it possible that something hidden is occurring that explains the relationship? Why does a combination of data elements act together as a leading indicator? What data is missing and how can the firm acquire the information? Finally, how good are the predictions? How can they be used and how can they be improved? As the organization’s assets and skills evolve, it can tackle bigger, more valuable questions. Prescience is a valuable strategic tool, but it is earned.

In summary, for law firm leaders with vision, predictive analytics is a prescient technology that should not be ignored. There are numerous areas where these sorts of approaches might be applied for immediate gain and to add dollars to the bottom line. A few examples will illustrate the point.

MarketingRetailers want to predict who will be receptive to marketing, and they run predictions and tests. This allows them to optimize the selection of whom they approach and how they approach them. Would any of your practice groups want to know the same thing? How do you fund and assess your marketing efforts? This is the science of marketing.

Mining Social FeedsLots of clients and potential clients are on social media. What might one learn from social data? This could be anything from identifying infringement on existing clients’ intellectual property or recognizing an entrepreneur who has the next hot thing and needs corporate representation.

Labor/Human ResourcesHow might you use these technologies to generate insights and provide guidance on recruiting, staffing levels, technology utilization, and administrative costs? How might one’s knowledge of these allow you to provide guidance to other companies on privacy and labor issues?

Long Term Client RelationshipsA litigation firm that handles (or coordinates) a large number of routine claims may point the data-mining engine at all of these cases. It is hard to know what insights might be obtained, especially if one could obtain the corporate records of the nonsuits. Which variables predict the filing of a lawsuit? Could this guide corporate policy on how to handle a situation? Which factors influence a settlement?

Industry DashboardsWhich key leading indicators predict volumes for your various practices? Do you have an analytical prediction method? If you tried to develop such leading indicators, what might you learn? What do you know about the interconnectedness of the people networks known by your firm?

You should not be embarrassed if some of these are difficult questions to answer. You might even conclude that your future business is based mostly on the personality and networks of key partners. If so, then maybe the next tier of partners might benefit not by simply replicating partners with exceptional marketing skills but by supporting some forays into new approaches fueled by predictive analytics.

4

Managing the Intelligent Global Expansion of Research Services at Squire Patton Boggs LLP

Where lawyers can take advantage of time zones, handing on work around the globe, Research Services also chases the sun answering reference questions. If a lawyer in Brussels asks a lot of our Brussels librarian, she can easily request assistance from the UK research team. If they can’t complete the request before close of business, they can pass it to the US offices, where our US team can move it from east coast to west coast.

Global expansion has been a fact of our professional lives in this firm for decades, and change management has been woven into the tapestry of our global approach from the outset. Collectively, the three of us have worked in the one firm for 34 years or, depending on how you look at it, 14 firms in 34 years, because the firms we each started out in have grown and changed significantly, combining to become Squire Patton Boggs. We are a Swiss verein of three arms, a United States LLP, a United Kingdom LLP and an Australian GP. We have more than 1,500 attorneys in 44 offices in 21 countries, with more than 60 percent of offices in Europe, Asia, the Middle East, and Latin America. We are indeed global players, a fact that has been enhanced even more recently by our expanded

presence in the Middle East through our combination with Patton Boggs.

International presence has transformed our business strategically over and over again, and consequently the role and management of the research team has needed to be flexible and dynamically responsive. The constant cycle of change and growth has made our positions feel like multiple roles, embracing and encompassing new resources and challenges every day—rigid adherence to a standard procedure simply cannot cope with the pace of change. Many of the challenges of managing global expansion are somewhat obvious on their face, but the solutions are complex and varied. First, without a good team of insightful and collaborative research professionals, managing a global service cannot happen.

Let’s examine some of the challenges in light of the Squire Patton Boggs LLP case study and the professionals who managed the expansion. As a Global 20 Law Firm, our firm positioned itself through strategic combinations with Hammonds and Patton Boggs to have a global reach beyond its already strong global presence. Research Services by necessity has positioned itself to be nimble and quick to respond. As a result, these challenges are being met with a stronger team of researchers than ever before.

By Scott Bailey, Global Research Services Director (US LLP), Washington, DC; Philip Duffy, Deputy Global Research Services Director (UK LLP), Manchester, England; Kelly Underwood, Senior Researcher (AU LLP) Perth, Australia

5Practice Innovations | October 2015 | Volume 16, Number 4

Managing the Intelligent Global Expansion of Research Services at Squire Patton Boggs LLP

A prerequisite for a successful research team is to embody the idea that the ideal outcome is a valuable service for the firm. Our view is client-centered; it encompasses both internal and external client priorities. Each of the firm’s employees—not just the lawyers—is considered to be a client of the research team, along with every client of the firm itself. Equal importance is given to research questions whether they are from the facilities management team or from a billing attorney. Only deadlines dictate priority.

A major combination of the firm in 2011 into the three-armed verein sparked a collegiate, collaborative atmosphere throughout the offices, which was embraced by the research teams scattered around the world. John Poulsen, the Managing Partner of the Australian arm of Squire Patton Boggs, calls us the “one firm firm.” Squire Patton Boggs operates as one firm, in multiple locations, supporting and assisting clients around the world. Thirty-four percent of work generated in one office is worked on by other offices, and 460 of the firm’s top 500 clients have used attorneys from two or more offices around the world.

Where lawyers can take advantage of time zones, passing on work around the globe, Research Services also chases the sun answering reference questions. If a lawyer in Brussels asks a lot of our Brussels librarian, she can easily request assistance from the UK research team. If they can’t complete the request before close of business, they can pass it to the US offices, where Squire Patton’s US team can move it from East Coast to West Coast. A couple of hours later, the Perth, Australia team gets into the office and can pick it up, finish it off and deliver the completed research before the Brussels lawyer and librarian arrive at the office for the day.

While the Hong Kong office library resources are arranged through the UK LLP and therefore “officially” their library assistance ought to be through the UK research team, Hong Kong staff members regularly email the librarian in Perth, because both Perth and Hong Kong operate in the same time zone. In the early morning, the San Francisco librarian assists Sydney staff members via the firm’s Research After Hours email address before the Perth office opens up. This flexibility in provision of services is useful to staff, but it does raise the problem of billing. The United States generally bills out usage of research resources, whereas the United Kingdom and Australia include them in firm overhead calculations and only bill out any disbursements incurred.

Another issue with this collegiate, flexible provision of services is when global good intentions are defeated by localized services—it doesn’t matter where you are in the world, all UK Land Registry searches—online and in person—can only be performed during UK business hours. That means a US librarian will need to bounce back a late night Research After Hours email request from a London lawyer to the UK research team, so the search can be arranged the following business day.

Time zones are equally a blessing and a curse. While they facilitate a chase the sun reference service, they also make it exceedingly hard for the entire team to meet without at least one person being inconvenienced by a very late night or early morning teleconference.

We are undertaking a project to merge all the different office library catalogs into the UK library catalog software, EOS. Once this is complete, we will have eliminated one of our smaller challenges. Currently, an Australian or Hong Kong lawyer may email a nonurgent request for a court-ready copy of a decision from a specific UK law report series. The Australian librarian has the option to request it from another firm locally, or ask the UK office for a copy. If the librarian requests it from the UK LLP libraries, and none of the UK libraries actually hold that series or the particular date period, then a UK librarian will often request it from another firm local to them, and forward it on to the lawyer once they have received it. This small inefficiency uses the time of two Research Services staff members, instead of one. A workflow best practices team from within Research Services is working on these, as well as other “jump ball” issues of service.

Another challenge we are facing is trying to combine our individual publisher contracts into global contracts. By doing this, we are likely to save money, but also widen access to resources—if the Madrid librarian has picked up a reference question from an attorney in Arizona, it is infinitely easier if the Madrid librarian has access to the same online resources as the Arizona attorney. Where we fall into difficulties is if the publishers aren’t willing to provide access on a global basis.

This globally-staffed research platform does have tremendous advantages in collaborating on cross-jurisdictional questions. With multiple language skills and different access points to online resources around the clock, our team is able to complete and maintain multijurisdictional projects in a variety of practice areas with regional expertise. Clients can access extranet research that has been curated by a knowledgeable global team through their own customized interface designed to their specifications.

6

Managing the Intelligent Global Expansion of Research Services at Squire Patton Boggs LLP

There are a few publishers that see the benefits of a firm using a global subscription, recognizing that while the entire firm technically has access, the likelihood is that only the “French speaking,” “[Middle East country] jurisdiction,” or “European-based” lawyers will use the service, and charge accordingly—not charging for a 1,500 person legal headcount but rather for an appropriate percentage. But some publishers have a corporate structure which has divvied up territory and simply cannot countenance the concept of sales being allocated to multiple entities, and therefore block the combination of territory contracts into global ones.

Recognition of these global arrangements is already here but the facility to achieve them in reality will take time as we establish and insist on the business case. Deciding to develop our separate local library and research services into a global research team was the most intuitive decision in the world. What followed was a series of challenges that will continue to be navigated by our expert research team as we encounter more possibilities with fewer obstacles in the future.

Text tk

7Practice Innovations | October 2015 | Volume 16, Number 4

Should we manage our existing process more proactively? Or should we change the process and then manage the new process more effectively? Does it matter? More than four years ago I wrote a blog addressing “which comes first, legal project management (LPM) or legal process improvement (LPI)?” At the time, there was a lot of confusion about the two terms.

They were used interchangeably and the concepts of both were included in defining one or the other. Two years later, I addressed the topic in another blog. There was more clarity around the definitions at that point, but the question of which should come first was still debated.

Ultimately, the legal arena decided, for the most part, that LPM comes first. Is this a classic case of the tail wagging the dog? Not really. Both techniques are important. The truth is the two methodologies support each other. If you use LPM techniques, you inevitably identify ways to improve some aspects of the project (i.e., process improvement) as you manage work more carefully and with more tools than in the past. The difference is in the level of scrutiny of the steps in the process. LPI is about designing an “ideal” process and

LPM is operating that process. The resulting change from LPI is a more efficient process. LPM results in a project that is well managed and completed on time and on budget. LPM skills are essential in order to manage to the new, more efficient processes that are developed, if you want those new processes to be successful. Both methodologies provide opportunities for improved performance.

Not long ago, only a few firms were recognizing the need to analyze the steps in some matter types more closely and to identify where there are short-term and longer-term opportunities to streamline those steps and those types of matters, in addition to improving performance with LPM. Today, more and more firms are tackling the topic of LPI. Many are still just talking about it, while others are identifying matter types where LPI could have the greatest impact.

So what’s inhibiting broad acceptance of LPI? I’m certain some cynics would claim that lawyers and law firms chose to focus on LPM first, and sometimes exclusively, because it’s easier or less threatening than LPI. But I believe other reasons are involved as well.

In my experience, the resistance to LPI falls into several categories and each is worth exploring:

• Recognition that other things are impacted by changing how services are delivered

(Probably) No Longer Asking Which Comes First—Project Management or Process Improvement

The truth is the two methodologies support each other. If you use legal project management (LPM) techniques, you inevitably identify ways to improve some aspects of the project as you manage work more carefully and with more tools than in the past. Both methodologies provide opportunities for improved performance.

By Carla Landry, Senior Consultant, LawVision Group LLC, Washington, DC

8

(Probably) No Longer Asking Which Comes First—Project Management or Process Improvement

is where LPI skills and techniques are helpful. If LPI is employed, then one is able to identify the repeatable and more routine aspects of legal services. Those services are streamlined, outsourced, replaced with supporting technology, sourced back to the client, or eliminated completely. These are the aspects that become evident with experience and they comprise the “decades of experience” a firm has in handling these types of matters, when they’re pitching their services to clients. The unique aspects of a matter, or even of similar matters, are what remain, and those require significant legal expertise. Clients are willing—and perhaps even reluctantly happy— to pay for that expertise and those unique legal skills.

Now, if employing LPI skills and techniques offers firms an opportunity to showcase their unique expertise, and perhaps charge a premium for those, does implementing an LPI initiative necessarily mean commoditizing services? Perhaps it doesn’t. Law firm clients have already decided they will not pay for what they view as commodity or routine work. Routine work is just that—it’s routine and it’s what a firm has already told the client they know how to do based on having done “a lot” of this type of work. If LPI helps a firm identify the routine pieces of a matter and reduces the cost of delivering those pieces, then profits are no longer squeezed out of those routine pieces, or at least not as much. What comes to the surface with all that experience is how to handle the unique aspects of similar matters. These are the aspects of the matter where premium rates can be charged.

Resistance to ChangeWhat is it about lawyer personalities that would hinder the implementation of LPI techniques? Turns out there might be a few things. According to Dr. Larry Richard, Principal Consultant for LawyerBrain LLC, lawyers share a number of personality traits that distinguish them from the general public. Dr. Richard has been studying the personality traits of lawyers for over 20 years and has measured dozens of traits among thousands of lawyers. Results from one test, the Caliper Profile, indicate that lawyers are very different from the general population in 6 of 18 traits. Three of those six—Abstract Reasoning, Skepticism, and Urgency—likely contribute to the delay, postponement or abandonment in rethinking the delivery of their legal services.

Abstract Reasoning—This trait indicates the potential to solve problems and understand the logical relationships among concepts. People who show a high level of abstract reasoning ability are typically capable of understanding complex issues and integrating

(e.g., staffing, fees, compensation) and too overwhelmed to know where to start

• Belief that services and people will become commoditized by “standardizing” the process

• Lawyer personalities and resistance to change

Too OverwhelmedLPI does impact other things—and it should—but that doesn’t mean it all has to be tackled (or “fixed”) at the same time. When a process is streamlined, staffing needs change. Ideally, lower rate professionals will be able to complete more standard tasks. That’s the point. Those tasks that are repetitive across matters should be done in as uniform a manner as possible and by the lowest cost professionals possible. The unique tasks should be the focus of the highly experienced lawyers.

Fees will change. Again, that’s the point. The idea is to have routine matters handled at the lowest possible level—use technology or non-timekeepers, if possible—in order to reduce the cost of the service to clients. This means that lawyers and firms need to rethink how to maintain profits under that scenario. Fixed-fee billing, which is accurately priced and managed to budget, is often the answer. This is where strong LPM skills are critical.

Compensation systems are still frequently linked to bulk revenue, and not enough consideration, if any, is given to efficiency, low cost, or profitability. This may be the last thing to change, but that doesn’t mean firms should continue to deliver legal services in an inefficient manner. Delivering value to your clients should be the driving force behind your choices, not your compensation system. If not, you’re likely to lose your clients.

Fear of CommoditizationA key concept behind LPI is efficiency. It’s true that historically lawyers have not been paid to be efficient. They were paid primarily for their legal expertise, but the primary measurement tool, billable hours, encouraged inefficiency. To be fair, over time firms differentiated themselves on a number of other factors—e.g., price, geography, industry expertise, responsiveness, relationships, etc. In today’s market, those differentiators are no longer enough. Law firm clients expect their lawyers to be efficient and to deliver high quality at the same time—and maintain all those other qualities.

Does that mean a lawyer’s work is commoditized in the process of creating efficiencies? The answer is “sort of.” Part, but not all, of a lawyer’s work is recognized to be commodity work through an LPI initiative. This

9Practice Innovations | October 2015 | Volume 16, Number 4

(Probably) No Longer Asking Which Comes First—Project Management or Process Improvement

information. Lawyers score an 82 as compared to the general public score of 50.

Impact of trait on implementing LPI: The potential for “paralysis by analysis.” Lawyers tend to overanalyze something or challenge something because they enjoy challenging another’s ideas. So, if someone is suggesting that they deliver their legal services differently (i.e., look at the current process and identify a more streamlined approach or improve the process), they can either analyze it to death (paralysis) or challenge it to the point where nothing is accomplished.

Skepticism—This trait indicates one’s inclination to doubt or question others’ motives. Caliper’s research has shown that this attribute is a “performance inhibitor” in some jobs. Highly skeptical individuals tend to be guarded and doubting of others’ intentions. Lawyers score a 90 as compared to the general public score of 50.

Impact of trait on implementing LPI: High skepticism correlates with being cynical and/or argumentative. As such, a skeptical lawyer is likely to be adversarial in all encounters, especially if he or she perceives that delivery of legal services is being challenged. Mapping a process and identifying potential areas for improvement require a collaborative effort, not a confrontational one. Skeptical lawyers derail those efforts.

Urgency—This trait measures the tendency to take quick action in order to obtain immediate results. It’s a measure of an individual’s sense of immediacy and a need to get things done. High scores on this attribute tend to be driven to act quickly. Lawyers score 71 as compared to the general public score of 50.

Impact of trait on implementing LPI: High scores correlate to impatient and results-oriented. Analyzing a process takes time and patience. Identifying how to streamline that process takes even more time and patience. Monitoring and controlling to that new process takes yet even more time and patience (and attention to detail). If lawyers have high urgency, they’re unlikely to want to spend the time analyzing a process. They would rather get to work on solving the client issue (the legal issue).

Combine these three traits with lawyer high autonomy scores (dislike for being told what to do) and low resilience scores (resistance to criticism or perceived criticism) and it’s clear that it’s a challenge to try to get lawyers to consider LPI. Does that mean we shouldn’t try? Or is it all in the approach?

So how do you get started? Where should you start? Which should be first—LPM or LPI? It doesn’t matter. Just get started.

10

Earned Value Management (EVM) is a proven method and best practice used by large organizations all over the world to understand and control project success. The main advantage of EVM is to anticipate project problems early and to take action to prevent overspending, and to stay on schedule. EVM is a new and unexplored practice for most law firms.

IntroductionYour client is depending on you, but their critical project is over budget and now they are upset. How did this happen? Why didn’t you know there were problems sooner?

Are you having trouble keeping your projects within budget and on schedule? Would you like to forecast project performance

problems accurately? You can identify and prevent a runaway project before it gets out of control using an analysis tool called Earned Value Management (EVM).

EVM is a proven method and best practice used by large organizations all over the world to understand and control project success. You can measure and control your project by understanding your project’s “earned value.”

As work is accomplished, it accrues “earned” value. Understanding your project’s earned value can help you to predict cost amounts and schedule dates at project completion when the project is as little as 20 percent complete.

A Quick History of EVMThe concept of earned value has been around in one form or another since the industrial revolution in the late 1800s. It was used to track cost and schedule performance of large government projects in the U.S. in the 1960s. It came into its current form in the 1990s when the U.S. government started requiring the use of EVM for government contracts. Since then it has been adopted as a project management methodology by the Project Management Institute, and today many organizations and governments use earned value as a means of keeping projects on schedule and within budget.

Understanding EVMEVM is a sophisticated and adaptable methodology. In any description of EVM it is easy to go deep and wide. This article is an introduction to the basic concepts of EVM and what it can do.

At its core it has two parts—primary data and derived data.

• Primary data is basic core information you gather about your project.

• Derived data consists of formulas that use the primary data to report and predict project performance. The derived data is where the real magic happens.

Making Project Management Work Better: Earned Value Management for Law Firms

By Don Philmlee, Legal Technology Consultant, Washington, DC

11Practice Innovations | October 2015 | Volume 16, Number 4

Making Project Management Work Better: Earned Value Management for Law Firms

Using Primary DataTo understand your project’s earned value you need to know these four basic bits of project information:

• The date when you want to report project status (Status Reporting Date)

• Your total budget (Planned Value)

• The actual costs incurred (Actual Costs)

• The value of the work you have completed on the project (Earned Value)

Status Reporting Date is when the status of the project is reported and it is key to calculating the below values.

Planned Value (PV) is your project budget. It shows you how much money you have planned to use to complete the project in a given time, your budget. PV is (the total budgeted cost of the project) x (percent of work planned to be completed).

For example:

• Project will build 10 storefronts (1 per month for 10 months).

• Total budget for a 10-month project is $100,000.

• 4 months have now elapsed. So, according to the project plan 40 percent of the work should be complete.

• To calculate: (total budgeted cost of $100,000) x (planned work completed 40 percent).

• So the PV is $40,000.

Actual Cost (AC) is the actual cost incurred to complete project work. AC includes all hourly costs and fixed costs as of a given date.

For example:

• On the fourth month of a project, your billing department reports that your project has spent $30,000 to date.

• So the AC is $30,000.

Earned Value (EV) is the value of the completed work. As work is accomplished, it accrues “earned” value. EV shows you how much work you have actually completed and its value as of a given date. It is sometimes referred to as the budgeted cost of work performed. EV is (the total budgeted cost of the project) x (percent of work actually completed).

For example:

• Total budget for a 10-month project is $100,000.

• 4 months have now elapsed, but only 25 percent of the work has been completed.

• To calculate: (total budgeted cost of $100,000) x (actual work completed 25 percent).

• So the EV is $25,000.

Simple EVM Example ProjectLet’s take the above examples and combine them for a simple EVM analysis:

• Our project will build 10 storefronts (1 per month for 10 months).

• We have a total budget for a 10-month project of $100,000.

• 4 months have now elapsed.

• According to the project plan 40 percent of the work should be completed.

• So the planned work completed (Planned Value) = $100,000 x 40 percent = $40,000.

• Your accounting department reports that your project has spent $30,000 to date.

• So the actual costs expended (Actual Cost) = $30,000.

• Your project manager reports that only 25 percent of the work has actually been completed (only 2.5 storefronts have been done).

• So the actual work completed (Earned Value) = $100,000 & 25 percent = $25,000.

The diagram below shows our Primary Data mapped against a calendar. Sometimes project managers only compare Actual Costs against the budget (Planned Value). If you did that and saw the chart below, you might conclude that your project is doing pretty well. You planned to spend $40,000, but have actually spent only $30,000. It looks like good news since less money has been spent than allocated.

12

Making Project Management Work Better: Earned Value Management for Law Firms

However, EVM provides us with a deeper analysis of our project status by applying our Primary Data to the Derived Data formulas.

Using Derived DataThe true costs and future of the project are shown more clearly when you use Derived Data to inspect the status of your project. If you have gathered all of your Primary Data, calculating Derived Data can be as simple as plugging numbers into a spreadsheet. What is amazing is that it takes the guesswork out of determining project status. You run a formula, and in a few seconds you have a solid no-nonsense answer.

The following are Derived Data examples that show cost and schedule performance and forecasting for the above sample Project. Each example asks an important status question.

Is the project over or under budget? Use the Cost Performance Index (CPI).

• The formula: EV / AC = CPI.

• Applying this to the simple example above: $25,000 / $30,000 = 0.8333.

• If CPI is less than 1 the project is over budget.

• If CPI is greater than 1, the project is under budget.

• RESULT: This project is over budget.

How much is the project over or under budget? Use the Cost Variances (CV).

• The formula: EV-AC = CV.

• Applying this to the example above: CV = $25,000 - $30,000 = -$5,000.

• If CV is less than zero project is over budget.

• If CV is greater than zero project is under budget.

• RESULT: This project is over budget by $5,000.

Is the project ahead or behind schedule? Use Schedule Performance Index (SPI).

• The formula: SPI = EV / PV.

• Applying this to the example above: SPI = $25,000 / $40,000 = 0.625.

• If SPI is less than 1 the project is behind schedule.

• If SPI is greater than 1 the project is ahead of schedule.

• RESULT: This project is behind schedule by about 38%.

How much is the project ahead or behind schedule? Use Schedule Variances (SV).

• The formula: SV = EV-PV

• Applying this to the example above: SV = $25,000 - $40,000 = -15,000

• If SV is less than zero project is behind schedule.

• If SV is greater than zero project is ahead of schedule.

• RESULT: This Project is behind schedule.

What will it cost to complete the entire project? Use Estimate At Completion (EAC).

• The formula: EAC = BAC / CPI.

• Applying this to the example above: EAC= $100,000 / 0.8333 = $120,000.

• RESULT: The cost to complete all work in the project = $120,000 ($20K over budget).

What will it cost to complete the remaining work? Use Estimate to Completion (ETC).

• The formula: ETC = EAC –AC.

• Applying this to the example above: $120,000 - $30,000 = $90,000.

• RESULT: The cost to complete remaining work in the project = $90,000.

EVM has unique valueIdentify projects before they fail—The main advantage of EVM is to anticipate project problems early and to take action to prevent over-spending, and to stay on schedule.

Improve project visibility and accountability—EVM requires more interaction and data input (hours/expenses) from workers and managers in order to accurately generate project reports. Project predictions and outcomes (good and bad) are reported, and where everyone can see the results both personally and for the project. EVM helps to identify problems quickly for immediate management attention.

13Practice Innovations | October 2015 | Volume 16, Number 4

Making Project Management Work Better: Earned Value Management for Law Firms

Improve the planning process—EVM requires good data. It forces you to clearly and concisely plan the details of a project.

Marketing advantage to new and potential clients—EVM is a well-known methodology that is used by many companies nationally and internationally and requires commitment and organizational maturity to implement successfully. It allows your firm to have statistical and objective information about a project and can provide accurate reports for clients on their project’s status.

Challenges to using EVMBad baseline = Bad reports—EVM requires a project to have a solid starting baseline for costs and schedule. It is critical to any earned value analysis. Collection of solid Primary Data is a critical component in making EVM work. A hastily or poorly done baseline means that any earned value analysis may be inaccurate. A good baseline is critical.

Quality—Quality of work is a critical component in any project. Unfortunately, EVM has no way to quantify quality. A project may be getting done well according to EVM, but the quality of work may be low.

May require new software applications—All of the data EVM requires is data that should already be routinely collected by project managers. Often the data exists, but is in different systems (time and billing, accounting, scheduling) and is cumbersome to routinely collect. It is tempting to buy a new application that does everything. Unfortunately, the initial cost of a new system, training, and data migration are usually cost prohibitive, and a dedicated system is usually best purchased when your organization really needs and values such a system. With a bit of ingenuity, EVM may be implemented using existing applications. Data can be exported from existing systems into an EVM repository. This might be an existing SQL database or something as simple as a desktop spreadsheet.

EVM may be too rigid—EVM emphasizes detailed front-end project planning and often appears to be intimidating. While data must be as accurate as possible in the beginning, EVM can be extremely flexible. A flexible EVM implementation is the result of creating a flexible work breakdown structure (WBS) to help scope the work. The WBS is a hierarchically related list of tasks in a project.

EVM creates extra work and cost—Sometimes it is very time consuming to collect actual costs in an organization. Data required for EVM may be spread across several enterprise applications and departments in an organization. Integrating the data from disparate systems and departments is a challenge, but once done, the benefits from the acquired knowledge and increased project control are well worth the initial effort it takes to collect and integrate data for EVM.

EVM is only for large-scale projects—EVM is usually implemented as a requirement or control measure on larger projects. However, the project control gained by using EVM makes it a very attractive tool for anyone to use with projects large or small. In order to adapt EVM to smaller or shorter projects, you may need to make some accommodations:

• Increase the project reporting frequency to weekly or even daily for very short projects.

• More frequent reporting puts pressure on an organization’s cost reporting systems, which may not be able to accommodate these short periods. Project managers may need to do manual tracking of actual costs or make other arrangements like project timesheets to all project participants.

EVM in Law FirmEVM is a new and unexplored practice for most law firms. EVM has too many advantages to ignore. As firms continue to embrace project management disciplines to help control work product, expect to see some form of EVM adopted. Also, tracking billable hours is already a discipline at law firms. This may make understanding actual costs easier when required for an EVM analysis.

ConclusionEVM may require some work to set up initially, but the payoff of having an “early warning” project management tool that enables the firm to forecast project performance accurately is invaluable.

14

What do law firm pricing professionals do all day? How did they get started? What skills do they have? What do they love about their jobs? And what do they struggle with? I spoke with three professionals from firms in different stages of formal pricing initiatives to find out.

A Day in the Life of a Pricing Professional

The number of pricing groups in law firms has been on the rise for a few years now. There is even a dedicated pricing conference (P3) which recently held its third annual event in 2015. The conference has roughly doubled in attendance each year. The number of firms with one or more dedicated pricing person is following the same trend. At the time this was drafted, around 40 percent of AmLaw 100 firms have at least

one full-time Pricing Specialist/Manager/Director. Some have as many as five or six! (For a relatively complete list of such firms, see Patrick on Pricing’s “roll call.”)1 ALM Legal Intelligence’s 2015 report on law firm pricing professionals reports even higher numbers; more than half of top U.S. firms have someone dedicated to pricing.2

That same report describes the characteristics of the “average” pricing professional. Most often known as the Director of Pricing & Project Management, he or she is working in a 500 to 1,000 attorney firm, is considered a member of the firm’s senior leadership, sits in the finance department and reports to the CFO, and manages a team of two to five direct reports. This professional focuses on alternative fee

arrangements (AFAs), but is also responsible for (in order of priority) profitability analysis, RFPs, budgeting, client fee discussions, and pricing training, and earns above $200,000, including salary and bonus.

Sounds pretty good to me! How do I sign up? In my quest to learn more about how Knowledge Management Professionals, who often naturally complement LPM programs, which naturally complement pricing initiatives, can support this new area of focus (see also “KM Professionals: a Natural Fit for LPM”),3 I sought more details on what pricing professionals do. What do they do all day? How did they get started? What skills do they have? What do they love about their jobs? And what do they struggle with? Many of the job functions are covered in an excellent read, Law Firm Pricing: Strategies, Roles and Responsibilities4 but what’s it really like to be in this kind of role? What does the average day-in-a-life feel like? To better understand this, I spoke with three professionals from firms in different stages of formal pricing initiatives.

Matthew Laws, Senior Director of Practice Management, Crowell & Moring, LLPMatt has worked at three large firms over the last ten years in ever increasing pricing-related roles. He has an MBA in finance and began in the finance department at Shook, Hardy & Bacon LLP, where he helped monitor and manage billing for a single, but very large, client with a multimillion-dollar yearly fixed-price arrangement to cover a docket of cases. In this role, he gained a deep

By Lisa Gianakos, Director of Knowledge Management, Pillsbury Winthrop Shaw Pittman LLP, Washington, DC

15Practice Innovations | October 2015 | Volume 16, Number 4

A Day in the Life of a Pricing Professional

understanding of the practice and the legal processes surrounding the work itself, which eventually led him to move from Finance to the Practice group. There, he continually analyzed the timekeepers and working teams, tracked budgets, etc., and utilized a Legal Project Management (LPM) tool to manage all case details. Those details, like how many complaints were filed, how quickly things seem to move in a jurisdiction, a history of decisions made on motions, etc., were used to help analyze and predict the likely direction of similar cases, which ultimately fed back into pricing.

Matt also holds a Master in Project Management. This background has undoubtedly allowed him to play both ends of the field (pricing on one end, LPM on the other). After the 2008 financial crisis, he moved to Reed Smith where there was a firm-wide effort to establish and manage alternative fees, including substantial fixed-fee projects. This gave him a chance to work with many large clients, although there he was focused primarily on pricing.

Pricing at CrowellMost recently, Matt is the Senior Director of Practice Management at Crowell & Moring LLP, which is taking a holistic approach to pricing and LPM (his practice management group is responsible for both). On the project management side, the firm uses set criteria to determine which matters are likely to need active PMing. For instance, all AFA/value-based billed matters with significant risk are managed. Worth noting, around 15 percent of the firm’s work is based on contingency fee or success-based models. Still another 15 percent of their business is fixed-fee. (This sum total of 30 percent is much higher than the industry average, based on various surveys.) Two lawyers and a paralegal that work on his team handle the project management role. Within the matter team itself, a project manager is assigned, most often a senior associate or counsel. They interface directly with the partner for the matter, as well as with clients, themselves, relying on Matt’s LPM team for overall guidance.

On the pricing side, Matt has two financial analysts who monitor matter profitability and help define budgets. They also analyze financial data to identify emerging trends within a practice, for a client, etc. For instance if they start noticing heavy write-offs on hourly matters for a particular client, they will start digging into why. They may even limit the timekeepers who can bill to these matters. Crowell’s CFO, with responsibility for reviewing and approving write-offs, engages Matt for input as it pertains to AFAs. Matt’s team can provide such support such as additional LPM training or assigning one of his

LPM attorneys to help out. Matt’s team is also responsible for the continual revision of budget templates.

When I ask Matt to describe his typical day, he mentions that, while he reports directly to the managing partner lending a level of prestige to what he does, a little advertising still never hurts! To increase his team’s utility, Matt spends time attending associate, partner and similar cohort lunches as well as practice group meetings, to advertise they are “here to help!” His approach is not to try to sell LPM but to focus on helping his colleagues create realistic budgets, using the many templates available. The project management aspects ultimately follow in time but without scaring off partners with more training or what may be perceived initially as overhead.

In broad strokes, he estimates spending 75 percent of his time working with partners on proposed budgets for existing clients with new matters, often under very short time frames. The other 25 percent falls into LPM, making sure profitability is good and meeting the client’s budget expectations.

What surprises him the most about what he does is the breadth of legal understanding that his position requires. “You never know when the phone rings if it will be about a multi-district litigation case, a class action suit, an investigation, or some kind of transaction,” he says. “Yet you need to be ready to ask the right questions regarding project scope. It takes time to learn the different processes, but our attorneys are usually understanding and good at guiding the conversation as needed.”

When I ask what is one of the more interesting or challenging aspects of the job, Matt points to the assessing of software needs that complement this new area. “Off-the-shelf software typically doesn’t meet the need,” he says. “Building your own means you get to define the requirements, yet we are not a software company.”

It’s a double-edged sword, but they ultimately chose to build a custom solution. The first release of their tool is internally facing, although ultimately they want to make a client-facing version too. The system starts with the intake of new matters. If the matter possesses certain AFA criteria, then a matter plan is required and the system branches off into a matter-planning module. Once the plan is developed, it is reviewed for approval. Approved matters are then surfaced in a project management module to be used by the project manager and Matt’s LPM team.

The responsibilities of Matt’s team are constantly increasing, he said. “This is a really pivotal moment for the industry. Both clients and attorneys are interested in developing AFAs as a credible medium to provide high

16Practice Innovations | October 2015 | Volume 16, Number 4

A Day in the Life of a Pricing Professional

quality legal service at a value. I look forward to the continued growth of this space.”

Kristina Lambert, Director of Strategic Pricing, Akin Gump Strauss Hauer & FeldKristina became Akin’s Director of Strategic Pricing in 2015, after serving as a Strategic Pricing Manager and in other pricing roles for the past four years. “My background is primarily legal. I was a litigator for eight years and then obtained a Master’s in Library and Information Science. My transition into pricing was actually a direct result of my moving towards a research/information services role.” Kristina’s knowledge of the legal process in general has helped in these roles. “As a litigator, I was required to submit budget estimates for the matters I was overseeing and for the department in general. There wasn’t really a focus on ‘strategic’ pricing at the time, but it forced a closer look at matters and to be more thoughtful about the level of effort/cost involved.”

Pricing at AkinAt Akin, the pricing function falls under a larger department, Pricing and Practice Management, headed by their Chief Practice Officer. Practice Management, Knowledge Services, Trial Service, and eDiscovery are also encompassed within the department. The pricing team includes Kristina, a legal project analyst and a pricing assistant. The CPO is also involved in pricing larger, more complex deals and in face-to-face client meetings. “Currently, the LPM function falls under the pricing role,” she says. “The original goal was to have a full-scale LPM program, but as the needs of the firm became more apparent, the approach has shifted to more of an opportunistic one. As we move forward with the roll-out of our new pricing/practice management tool [Umbria], it is safe to assume that the requests for LPM will grow organically.”

When I ask her what kinds of skills are required in this type of position, Kristina replies, “Budgeting and pricing go hand-in-hand. I think that pricing just takes it to the next level when you add in a profitability component. The ability to speak directly and candidly with partners about their matters is crucial in assisting them in crafting the appropriate pricing structure.” She also says a certain level of mathematic proficiency is necessary.

So what does a typical day or week look like? “One thing I love about my job is that it varies from day to day,” Kristina says. “My week typically includes a combination of fee conversations with partners, modeling potential pricing arrangements and providing content for pitches and proposals. The pricing group

also handles the approvals for all write-offs firm wide, which includes analyzing the impact of write-offs on the profitability of a matter. Reviewing write-offs also helps us to highlight practice management issues.”

When asked for aspects of the job she didn’t expect to be dealing with, Kristina says she was surprised by the amount of time spent educating partners and others within the firm about the overall concept of profitability and the drivers of profitability. “I actually really enjoy the educational aspect of my job and have come to realize that this function is ongoing. I also didn’t expect that within this role I would work with as many departments within the firm as I do.” she said. “I anticipated working closely with financial services and the practice management team; however, I also work frequently with the business acceptance/risk management group, business development team, and the IT department.”

Regarding the most unique aspects of the position, she says, “Most interesting to me is what I learn in speaking with partners and understanding the needs of their clients. Increasingly, clients are demanding different ways of pricing and managing matters. This presents an opportunity for our group to get involved and assist the partner in meeting those demands. I would have to say the most difficult aspect involves gathering information from multiple internal systems. We are currently implementing Umbria and have already seen how improved technology and data structuring can elevate the level of service we can provide.”

Bart Gabler, Director of Pricing and Legal Project Management, K&L Gates LLPK&L Gates hired its first Director of Pricing, Bart Gabler, in mid-2014. Bart has a JD as well as an MBA. In his consulting background, working for both McKinsey and PricewaterhouseCoopers for many years, he focused on a variety of management topics, including strategic planning, growth strategy and organizational transformation. He has experience in both client service as well as practice management that have been invaluable in his transition, which all started over a lunch with a senior partner of the firm who discussed a new role they were thinking about creating. Eventually, K&L’s Global Managing Partner hired him for that position.

A Lean ApproachBart jokes that his department, Pricing & Legal Project Management, is currently a “Party of 1.” But he is part of the senior leadership team, and reporting directly to the GMP affords him the freedom and flexibility to work “across” the organization. “I have a close connection with other key departments, like Finance

16 17Practice Innovations | October 2015 | Volume 16, Number 4

A Day in the Life of a Pricing Professional

and Marketing, but also have the latitude to diagnose and assess things broadly across the organization,” he said. “I’m able to meet with clients, the management committee, office leaders, practice leaders, etc., to help identify the agenda for pricing and LPM based on the pain points I hear about.” Based on all of these meetings, Bart ultimately presented his ideas for the direction the firm should take to the management committee and firm partnership. After gaining their general agreement on priorities, he got to work.

“The good news is that I am starting from a solid platform but hoping to make constant improvement in what we do and how we serve clients. I’m initially focusing on matter planning and monitoring. This provides the foundation to address a range of other needs from more creative matter arrangements structures, more advanced budgeting, increased process improvement, etc. These efforts are not entirely new to the firm, so I’m trying to build some uniformity across the firm on these types of processes,” Bart said. He has been in this role for one-and-a-half years. “The goal from the start was to be thoughtful about the change effort, to focus on acute pain points and create a longer-term transformation plan while fostering much of what the firm already does well.”

Bart works with every single department from Business Development to IT to the PMO. He has a “virtual team” that includes partners, as well as project managers, analysts and program developers as he builds up the tools and best practices. Although he plans to keep a lean model and drive as much of the knowledge out to the timekeepers having daily client interactions, he is currently hoping to grow his centralized team to continue to support the growing demand. Some of the key roles Bart says he sees a larger team focused on include “data analytics and monitoring budgets, process improvement and mapping, supporting the rolling out of new tools, and working with client service teams on LPM activities like matter planning and budgeting.”

I asked Bart about the skills needed for his role. He said, “I have to pull on lots of different levers, so a general business acumen that includes (1) an understanding of the legal business as well as the clients’ businesses, (2) knowing the legal process and phases of work around different types of matters, (3) performing both financial and matter performance analysis, and (4) an ability to engage clients and colleagues in discussions around their business needs that necessitates clear, structured communication. But being a problem solver is what will ultimately get you a seat at the table. You need to be a creative thought

partner—one who comes in with both open eyes and ears to hear the problems and talk through potential solutions. At times you need to be a change agent and help put vision into place, which is really exciting!”

He describes a typical week by the three roles he plays:

1. Client-Focused Work—Monitoring and communicating, reviewing ongoing matters for red flags and looking at pricing proposals, and working directly with client service teams.

2. Process-Oriented Work—Trying to establish a consistent process for the firm, so finding where in the process we are doing well so we can expand that effort, or where we are having some inefficiencies or pain points that we can address.

3. Problem-Solving Work—Looking internally for opportunities to improve and finding creative solutions to problems he encounters.

“One thing that has been quite interesting is the big impact that small changes bring,” Bart says. “For instance, offering a client a few options in pricing instead of just hourly billing allows the client to engage in deeper discussions and ultimately fosters a better relationship. Focusing on a single pain point, like giving partners direct and easy access to matter performance data, provides clients with better, faster and more proactive service. Creating an automated process for the annual review of fee arrangements is another example. These were very discreet efforts that had huge impact on partners and clients.”

On the most difficult aspects of his job, Bart notes, “There are an awful lot of stakeholders. Effecting change often requires a range of buy-in, and the vetting of ideas can be a bit cumbersome and time-consuming. However, I have benefited from those discussions, and it ultimately gets you to a better product, but debating with lawyers requires you bring your ‘A game.’ I really enjoy it and it helps protect us from mediocrity and ultimately leads to better adoption rates, as we move quickly once we get aligned.” In short, “it is a challenging role, but I like the challenge and like the direction we are heading.”

SummaryThe demand for Pricing and LPM professionals to fill the expanding industry trend will be strong for a while. There are varying opinions about whether it is better to take a person with legal industry experience and provide the financial and/or project management skills they might need, or to take a person with those

18

A Day in the Life of a Pricing Professional

skills but from a different industry, and teach them about legal processes. Either way, there are many firms that are only now beginning to formalize their pricing strategy, so many opportunities are likely to be available. The unique combination of necessary skill sets can make it hard for firms to find the right talent. So, if you have some of the skills—and also find the above stories interesting—there may be a place for you in this developing area. The biggest challenge for firms will be implementation. Adopting a pricing strategy requires a huge change, and all change is hard. This is especially true for changes that challenge the way financials have been reviewed historically, focusing on profitability rather than simply billable hours.

Sources1. Patrick Johansen. “Roll Call.” Patrick on Pricing, 2015.

2. Chris Johnson. “Your Clients Want Alternative Fees: Is Your Firm Ready?” The American Lawyer. The American Lawyer, 24 Aug. 2015 (access required).

3. Lisa Gianakos. “Knowledge Management.” Digital White Papers, July 2013.

4. Toby Brown and Vincent Cordo. “Law Firm Pricing: Strategies, Roles, and Responsibilities.” Wilmington Plc., 2013.

Text tk

19Practice Innovations | October 2015 | Volume 16, Number 4

If you are committed to increasing the impact of your knowledge management (KM) efforts, you will have to master the art and science of metrics. The purpose of metrics is to collect reliable data (i.e., quantitative or qualitative information) that can be used to understand the performance of an organization, and then guide

that organization towards more productive behaviors that help it achieve its strategic and operational goals.

There is no one-size-fits-all approach to metrics. In “Measure Better to Manage Better,” I provided examples of some types of metrics that you can use to gain a deeper understanding of the impact of your work:1

• activity metrics

• operational metrics

• behavior metrics, and

• outcome metrics

These four metrics are just a few of the wide range of metrics available to managers. For example, there are metrics such as key performance indicators or KPIs that are used to measure an organization’s progress towards its strategic and operational goals, and there are efficiency metrics such as cycle time.

The way you collect data for each of these metrics is quite different. Of the four types of metrics mentioned in my earlier article, the first two, activity metrics and operational metrics, are the easiest to collect because they are usually tracked automatically by various KM platforms and tools, which then present the data in an administrator’s dashboard or report. Behavior and outcome metrics however, cannot be tracked yet by common KM systems.

However, the metrics that are easiest to collect may not always be the best. While activity and operational data may suggest the existence of a behavior change, they cannot prove it. Similarly, while they suggest a possible impact, they cannot prove it.

What versus Why  Nonetheless, as long as you review system metrics regularly and stay abreast of their trends over time, you will know WHAT is happening with those platforms and tools. WHY something is happening is a different, and more difficult, question to answer. To make this clearer, consider the following examples:

1. When intranet usage goes up, the backend metrics should be able to tell you who is using the intranet, when they are using it, and what they are doing once they log in. But why is there an increase in usage? You will not find an absolute answer in those numbers. So a human will have to intervene. By analyzing the timing of the increase you might be able to determine that the uptick in activity occurred just after the arrival of summer associates. By

Measure Better to Manage Better—Part 2

A good manager will have both quantitative and qualitative metrics in their toolkit. This article explains the relative value of these metrics and provides guidance on how to collect reliable qualitative metrics that are likely to yield useful insights.

By V. Mary Abraham, Above and Beyond KM, New York, NY

20

Measure Better to Manage Better—Part 2

analyzing the identity of the users you should be able to confirm if the increased usage is attributable to summer associates. So it might be safe to assume that those summer associates were responsible for the increased usage.

2. When usage of your intranet goes down, what do the backend numbers tell you? That (1) usage is down and (2) particular people are not using the intranet. That may be interesting, but it misses the key issue: Why has usage declined? Once you have eliminated causes such as maintenance down time or acts of God, do you have an answer from these data? No. In this case, you have to leave the relative comfort of your automatically generated numbers to interact with the users (or, in this case more properly, the nonusers). Only by asking them point-blank why they are not using the intranet will you be able to ascertain why usage is down. Further, their answers will likely suggest ways in which you can improve your intranet, and thereby the usage of it.

These examples show the differences between quantitative and qualitative data at play.

Methods for Uncovering Why In the prior example I suggested that you ask, point-blank, the critical question you need answered. That is just one way of getting to the answer. There are, in fact, several accepted ways in which you can collect useful qualitative data. In this next section, we will focus on four methods: observation, survey, interview and focus group.

1. Observation. Observation simply means watching your internal clients as they go through their day (or a specific process) to understand what they are doing. Better still, because you are in the room with them, you can ask them why they did a particular thing after a key activity occurs. Gathering data by observation yields two important benefits. First, it sometimes can be done without the person (or people) you are observing being too conscious of your observation. As a result it can be more accurate than self-reported data. Second, it gets you out of your office (where illusions and delusions may persist) and into your clients’ workspace (where you must confront reality). To maximize the chances of acquiring quality data, be clear about your objectives, anticipate what types of activities you are likely to observe and the type of data you can reasonably draw from those activities, record your observations accurately without judgment, and do not interrupt the subject of observation more than absolutely necessary.

2. Survey. With the advent of Survey Monkey, SharePoint, Google Forms and other tools, surveys are now a little too easy to conduct. In our rush to ask questions, we sometimes disregard what the social sciences have learned about how to conduct a survey properly: how to choose the right questions to ask, how to create pre-set responses that are mutually exclusive and exhaustive, how to interpret the responses received, and how to report the responses and your analysis in an ethical manner. The other key methodological issue is understanding what participation and response levels provide the most reliable data. A final point is to only ask questions that can provide to actionable information. For instance, asking if people are satisfied is interesting, but by itself does not allow you to act. What you really need to know is why they are dissatisfied as a follow-up question to those that indicated dissatisfaction in the previous question.

3. Interview. In law firms, the hardest part of conducting an interview may be actually getting your foot in the door of the office of the person you want to interview. Unless the topic is directly related to a current client matter, lawyers tend to reschedule or cancel interviews that happens to conflict with billable work. That said, to increase the likelihood of having a productive dialogue, you need to come prepared with a plan for using the time efficiently, including a list of critical questions that should yield the data you need, while reassuring the lawyer involved that you are not wasting their time. A data-collection interview is not the time to wing it or improvise. Go in with a script, while remaining flexible. With experience you will learn when it is safe, or advisable, to go off script in order to further explore the responses you receive. Remember, if all you want to do is gather set answers, use a survey. If you want to be able to move beyond set answers to a more nuanced discussion, conduct an interview.

4. Focus Group. Marketers have used focus groups for years. Technologists have focus groups called user groups. Consider creating a KM user group that can provide early qualitative feedback on your products and services. If you intend to collect useful data through user groups, here are some things to keep in mind:

▪ Make sure the right people are participating, which means that they have enough experience with the system or tool in question that their responses will be based in reality rather than theory;

▪ Provide the necessary preparatory materials to ensure that they arrive ready to participate

21Practice Innovations | October 2015 | Volume 16, Number 4

Measure Better to Manage Better—Part 2

and comfortable that you have a plan to use their time in the meeting efficiently;

▪ Create a script with questions geared to elicit the honest responses of your group;

▪ Have a trained facilitator who can lead the meeting and manage the personalities in the room without compromising the quality of the feedback;

▪ Collect their feedback as accurately as possible; and

▪ Report back to the group promptly so they see that their participation was valued and useful.

CaveatsThere are a few things to keep in mind with all of the methods described above:

With all approaches, know that people behave differently when they know someone else is paying attention. When asked to report on their thoughts or behavior, they tend to report what they think the questioner wants to hear. This is known as the social desirability bias, and is a major challenge in survey, interview, and focus group situations.2 You only have to think about the classic job interview or office meeting to understand how this works in practice. So it is critical that you do what you can to ensure the accuracy of the response data. For example, if the survey data does not lose its value by being detached from the identity of the respondents, then assure all respondents of the anonymity of their responses. That should prompt more honest responses. In an interview or focus group setting, explain with sincerity why it is important that they be brutally honest with you and do not do anything that might suggest a particular answer is desired. To promote honest responses, check your own speech and body language—make sure that you do not act defensively in any way. Otherwise, you signal that you want less than honest responses.

When collecting data through observation, try to be inconspicuous without being unethical (e.g., recording behaviors without disclosure and consent). Expect that you will have to wait until someone gets comfortable with you before you can see their “true selves” emerge for observation. Until then, they will likely self-censor in order to demonstrate what they think you want to observe.

In focus groups, be very sensitive to the presence of groupthink; the natural desire people have to conform to a group to minimize conflict and promote harmony. Turn up your personal radar so you can detect quickly when it emerges. In addition, develop strategies for

reducing its occurrence, minimizing its impact, and redirecting a group that seems in danger of groupthink.

In surveys, interviews, and focus groups, be sure that you are not asking leading questions. The purpose of these data-gathering opportunities should be to provide new insights. If you rig the process to achieve specific ends, then you deprive yourself of the insights that might help make your work more effective.

The Quantitative versus Qualitative Trap In our numbers-obsessed world, we sometimes fail to appreciate the value of information that is not automatically generated by a system. Without a doubt, qualitative data in the wrong hands can be more subjective than quantitative data, and that potential for subjectivity has led some to prefer quantitative over qualitative. Don’t fall into that trap. Good qualitative data are every bit as useful—and sometimes even more useful—than good quantitative data. As mentioned above, quantitative data can explain what is happening, but you often need qualitative data to explain why it is happening.

Be careful as you collect and analyze your data. The key to success is to handle the data ethically and with the type of professional controls that minimize the possibility of distorting the qualitative data to achieve specific results. For that matter, it is just as possible to distort quantitative data to achieve specific results. Remember the old warning about the sometimes inappropriately persuasive power of numbers.3 There are three kinds of lies: “lies, damned lies, and statistics.”

Conclusion Now that you know more about metrics, consider what behaviors and outcomes are critical to the success of your KM work and then focus on the metrics that shed light on those behaviors and outcomes. If you track these metrics diligently and analyze them honestly, you will be guaranteed insight into your work and its impact on your law firm.

Sources1. V. Mary Abraham. “Measure Better to Manage Better.”Practice Innovations Newsletter, July 2015.

2. “Social Desirability Bias.” Wikipedia. Last modified 9 June 2015.

3. “Lies, Damned Lies, and Statistics.” Wikipedia. Last modified 9 Sept. 2015.