2019 KM Summit Survey Results

I report here on the 2019 Knowledge Management (KM) Summit survey results. The Summit is a meeting of about 80 senior knowledge management professionals from large US, UK, and Canadian law firms.

Mary Abraham, Oz Benamram, and I organize the meeting. To prepare for it, we survey invitees toward the end of each year to learn about current KM priorities and plans. We share detailed survey results with respondents and I publicly share a subset of the results here. (See the 2018 KM Survey results here.) An Appendix at the end of this post has demographic information and qualifiers about the survey.

We ask three primary questions; the quoted text in parentheses corresponds to chart labels that appear later this this report:

1. What are your top priorities for 2019? (“2019 Priority”)
2. What did you focus on in 2018? (“2018 Focus”)
3. What would you like to discuss with the group in 2019? (“2019 Discuss”)

Summary of 2019 Responses

The chart below presents 2019 KM Survey results. We administered the survey in late 2018; it asks about 2019 priorities, what respondents focused on in 2018, and what they want to discuss in our 2019 meeting. Respondents provided text answers and also checked tags. The tags are the values displayed on the horizontal axes in each chart.

New in the report this year is a designation of the highest or lowest value in the answers with an asterisk after an X-axis tag. Where two or more minimum or maximum values are equal, we star multiple tags.

As has been true now for several years, responses are fairly stable year to year. Visually, this chart differs little from the prior two years.

Confirming Stability in the Data: Changes in Focus Are Small

This year, we wanted to confirm the apparent stability of the survey results pictured above. We created additional tables and charts in Excel, not included here, which confirm that stability. We do, however, want to highlight two changes in focus (what KM professionals actually worked on in the prior year).

From 2017 to 2018, only two tags changed by 10 percentage points or more. Portals increased by 10 points. We are not surprised given the number of firms that have relatively old portals. With rapid evolution in web design standards, driven by consumer sites, even a five-year-old portal today may appear quite dated. Beyond look and feel, as firms want to tap more and more data sources, portals are a natural environment to present these data.

Separately, security is up by 12 percentage points. One factor driving this is cybersecurity threats. Another is the increasingly likely move of many (most?) firms to the least privileged security model, which means only matter team members and those staff with a need to know can see documents. With these two factors at play, seeing more KM professionals focusing on security is no surprise.

Did We Act on Priorities?

We find it instructive to examine the gap between Priority and Focus. Priority refers to 2018 priority intent stated in 2017, and Focus refers to what respondents actually worked on in 2018 as reported at year-end, in this 2019 edition of the survey. The gap reflects shifting attention intent from 2017 to 2018. Positive numbers mean more attention in 2018 than planned from the prior year. Negative numbers mean less attention in 2018 than intended or planned. (The data are normalized to account for differences in the average number of tags selected per person.)

Gaps this year are bigger than last year but still not that large given typical variations between intent and follow-through and normal variations in survey results. We focus here on a few changes of note.

• Work on AI + data science lagged its stated priority. Two related reasons may explain that gap. First, AI hype has declined. With that fall-off, some firms likely felt less urgency to pursue AI. And second, KM professionals face many competing demands for their time. Though AI retains much of its sex-appeal, both firms and KM professionals likely saw more immediate benefits from other pursuits.
• Both content management-DMS and forms-precedents-doc assembly activity exceeded stated intent. Both are core KM functions. While many KM professionals may prefer spending more time on sexier pursuits such as AI or client-facing KM, core functions exert a strong gravitational pull. Also, many firms are upgrading document management systems: the complexity of these upgrades likely meant some respondents spent more time than they initially expected they would need.
• Two of the hottest KM trends are enterprise experience management and security. And they are related: as firms improve security by moving to the least privileged model of document access, the importance of experience management grows. We are not surprised, therefore, that both would draw strong intent. Doing less than intended likely reflects competing time demands, plus the difficult effort required to do both right. We suspect quite a few firms punted this hard work to the future.
• Doing more on portals and less in enterprise search may be related. Portals often take more time than expected, partly because of the need to clean up content and data, and partly because new portals are more complex than old ones. Many firms make search available via the portal and others use search to power portals. Our hypothesis here is that firms focused on upgrading portals before addressing search. Even though the two can be intimately connected, the portal framework exists separately. And that framework is required if search is to work well. Also, the enterprise search products have changed significantly in the last couple of years. Some firms may have decided to allow more time for product evaluation and/or to let the market settle.
• The jump in selling KM internally reflects the biggest difference at 17 percentage points. We think this reflects that “hope springs eternal”. As KM has matured and become established in large law firms, KM professionals slip back to thinking that “KM will sell itself”. Then reality hits and KM teams realize that change does not happen on its own, that they must actively promote adoption.

Shifting Priorities Across Years

The chart below shows how priorities have shifted over time. Below the chart, we discuss some apparent trends.

Priorities for 2019 largely mirror those for 2018. With five years of consistent longitudinal data, instead of pointing out changes, we highlight a few apparent trends.

• AI remains hot – exactly what we said last year. Hype is down and the real work has begun. Even deploying proven off-the-shelf AI products takes time and effort.
• The continued decline in AFA / Budgets / LPM reflects the growth of specialized departments in law firms that have taken on these functions. (See, e.g., the P3 conference.)
• Expert systems and client-facing KM are closely related because many expert systems are deployed for clients, not as a purely internal tool. The steady increase in both likely reflects growing interest in serving clients digitally, driven by competitive pressure and client demands for value.
• As noted above, enterprise experience management is (1) a new product category and (2) critical once firms move to the least privileged security model. It is thus not surprising to see it grow in priority.
• While not a new product category, the enterprise search category is rapidly gaining penetration. That likely explains the steadily growing priority. With the changes in vendor alignment and improvements in functionality, we are surprised not to see even steeper growth.

2019 Priorities Compared to Discussion Interests

Each year we also compare respondents’ priorities (“Do”) with what they would like to discuss (“Talk”). The chart below shows this comparison (normalized by the number of tags per respondent, which yields a 74% multiplicative adjustment factor).

As was true last year, the variance between Do and Talk is fairly small for most tags. The two biggest gaps, one in each direction, warrant brief discussion. We believe the AI gap is explained by everyone wanting to do and discuss it to keep up with the market, even if not everyone will have the budget or time to do it. Forms and precedents, in contrast, show a big gap in the other direction. Forms are a mature discipline, so we are not surprised interest in talking about them falls below their priority level.

Conclusions

We draw two main conclusions from the survey, the same as the prior two years. First, KM remains robust. Each year we find more firms with KM professionals; the interest in the Summit grows steadily each year.

Second, KM has certain core activities but also tackles an ever-changing set of initiatives. KM professionals will likely always need to attend to core activities such as forms + precedents or upgrading core systems such as DMS and search. But they also work on a shifting range of activities, from pricing, and process improvement, to new methods of collaboration. New disciplines such as AI and data analytics and new software categories, e.g., data analytics and experience management (née experience location), will keep us busy!

Even with the AI hype factor down, we expect it will occupy us and many others for years to come. As AI hype declines and our collective learning about it in legal increases, we venture a prediction as well: AI efforts will increasingly shift to data and data analytics. We see a growing understanding in the market that data analytics – whether pure AI or not – is extremely valuable. Moreover, without good data, there is no good AI.

Appendix: About this Survey

The Questions Asked and Chart Labels

At the end of each year, we ask three primary questions, listed below. The quoted parenthetical text corresponds to chart labels that appear in this report.

1.        What are your top priorities for 2019? (“2019 Priority”)

2.        What did you focus on in 2018? (“2018 Focus”)

3.        What would you like to discuss with the group in 2019? (“2019 Discuss”)

Survey Demographics and Caveats About Interpreting It

Here are demographic and other details about the survey that highlight potential limitations:

1. Firm Size and Location: Most respondents work at US, Canadian, and UK firms with 500 or more lawyers.
2. Survey Not Representative of Market. We survey large law firms where we have identified one or more senior KM professionals. The survey therefore selects for firms with a KM commitment.
3. Respondents Provide Free-Form Text Answers and Tag Answers: Respondents answer each question with free-form text and tag their answers with one or more of 16 defined topics (tags). These tags, listed below, inform the survey interpretation
4. AI or data science
5. AFA / Budget / LPM
6. Client-Facing KM (except expert systems)
7. Collaboration / Internal Social Media
8. Content management / DMS
9. eDiscovery / litigation support
10. Expert systems
11. Experience location
12. Forms, precedents & document assembly
13. Matter management
14. Portal redesign or upgrade
15. Process improvement
16. Security / access management / info. gov
17. Search (install, upgrade, or improve)
18. Selling / marketing KM internally
19. Question + Tag Variation Over Time. The 2017-2019 surveys asked the same questions using the same tags. Questions in the 2015 and 2016 were almost the same.
20. Caution on Year-to-Year Comparisons. When interpreting surveys over time, understanding changes in respondents and response options is key. The tables below show the number of respondents by year and the average number of tags by question and year. We changed a few tags between 2016 and 2017.