Legal Operations

Metrics? What Metrics? KPIs That Legal Ops Professionals Need To Track (Part II)

Part II of a two-part examination of what it means to define and measure legal project success.

By  
on

You want to know why the billable hour is so entrenched and not going anywhere any time soon? It’s because it provides a direct, measurable metric or benchmark for the value of a lawyer to a law firm.

The billable hour is generally reliable (assuming accuracy), it demonstrates a form of ROI, and it provides the basis for some of the most important metrics law firms rely upon, like leverage, utilization, and realization. Few, if any, firms are tracking the type of work, and none that I’m aware of are looking at the quality of the work (at least not in any systematic way). Heck, many firms don’t even bother to use phase and task codes to categorize their work.

While we’ve come a long way from the days when an invoice from a law firm simply said “For services rendered” and included a dollar amount, there still appears to be some dissonance in the legal industry about how best to track and measure legal work and outcomes of legal projects.

Last week, we suggested that legal operations professionals need to identify at the outset of a legal matter what the success of a legal project will look like. This week, we explore ways to measure the success of a project.

According to the CLOC, the Corporate Legal Operations Consortium, in 2018 medium- to large-sized corporations averaged 400+ litigation matters. These corporations also spent on average in excess of $2 million dollars each on eDiscovery. Interestingly, only about half of these organizations use a case or matter management system. Yet somehow, 70 percent of them track budgeted costs to actual costs on a per matter basis.

I spoke to my friend John Rowley, the Director of Corporate Sales for ELM software company Onit. John and I have been talking about project management and legal matter management metrics for 10 years. “Tracking outside counsel spend and managing to a budget on a matter are quite common, even when organizations are still doing it in spreadsheets and haven’t implemented an ELM tool yet,” says John.

But these metrics are just the tip of the iceberg. As John works with corporate legal departments, he recommends to “invest some time to figure out the types of metrics you need at a matter level, metrics to measure risk, and outcomes.” John said, “Take a look at the metrics you are reporting on today, who are you sharing those metrics with and are they helping you to successfully drive business decisions at the department level, within business units and the company. Every organization has their own thoughts about how best to measure these things; about what metrics they want to measure and track and what the next steps are in their journey on metrics maturity.”

Sponsored

Outside counsel spend and budget to actual on each project are no-brainers. In order to fully understand the value of litigation, however, there are other metrics corporate legal departments could be tracking.

My thinking on this, as I pointed out last week, is that every legal matter tends to involve very similar events and follow a pattern of tasks and activities. So, looking at a litigation matter, for instance, one might categorize work into the Pre-Litigation Phase, a Pleadings Phase, a Motions Phase, a Discovery Phase, and a Trial Phase.

How might we measure the work in each of these phases? For too many, the default is the time and money spent to draft or answer a complaint or research and write a motion or brief or respond to interrogatories. But what about the organization’s exposure in the matter? What’s the value of this case? What are the consequences (think: risks) associated with losing? Is there a phase and task-based budget for the matter? What about the jurisdiction, the court, the parties? What type of matter is it? Which firms are involved? And what about the outcome? Did you even win? These are some of the questions I’d be seeking answers to if my company were involved in litigation. The good news is that this is all information that is easily available.

In discovery, an aspect of litigation with which I am particularly familiar, I keep track of all sorts of eDiscovery metrics. Years ago, before we had the smart business intelligence tools capable to reach inside any data store, I used to do these case studies — mostly because some lawyer asked me for the numbers, not because I enjoyed gathering metrics. We would measure every data point we could think of across the spectrum of the EDRM. How much data was identified? How much data was collected? What type of data? Where did it come from? Who are the custodians? How much data per custodian? Per source? How much data was left after filtering and culling? After processing? What data was promoted to review? What is the volume? How many reviewers were involved? How long did it take? Time spent on each document, by file type or data source? How many documents were produced, from what source or by file type?

If you’re not measuring these data points across the EDRM, you’re not getting a full picture of where the money is going, how you might realign resources, or whether you’re overpaying for services.

Sponsored

These are just a few of the metrics organizations should be tracking. But today, with technologies like machine learning, there are a host of additional data points that may be tracked with relative ease. There’s really no excuse for not gathering metrics.

In the end, it’s true that most of this — gathering and measuring metrics — really does just come down to money. We all use dollars to assess the value of the products or services we purchase. But other informative metrics could lead to new and innovative ways to measure and manage legal risk and new ways of thinking about and making strategic decisions.

Imagine a world in which we know in advance that a wage-and-hour suit filed by Firm X in the Southern District of New York before Judge Y is going to last three years and cost $500K in legal fees to prosecute or defend, and we know the value of the case is only $250K. Wouldn’t that be useful? Think “money ball” for litigation, as one of my good friends likes to say.

Mike Quartararo

Mike Quartararo is the managing director of eDPM Advisory Services, a consulting firm providing e-discovery, project management and legal technology advisory and training services to the legal industry. He is also the author of the 2016 book Project Management in Electronic Discovery. Mike has many years of experience delivering e-discovery, project management, and legal technology solutions to law firms and Fortune 500 corporations across the globe and is widely considered an expert on project management, e-discovery and legal matter management. You can reach him via email at mquartararo@edpmadvisory.com. Follow him on twitter @edpmadvisory.

Topics

, , , ,