[Ed Note: Please welcome guest blogger, Susan Kostal. Susan is a longtime legal affairs journalist who also offers marketing advice and media coaching. Follow her on Twitter at @skostal.]

LMA Tech’s annual in-house counsel panel is always one of the biggest draws of the conference, and this year was no different. Last week’s discussion in San Francisco, moderated as always by Nat Slavin of Wicker Park Group, could be called the “one-size-fits-one” lesson. However, I’ve dubbed it the “come-to-Jesus” panel. Repent of your sins, outside counsel, and sin no more.

The panel consisted of tech-company general counsel, Michael R. Haven, Senior Corporate Counsel Legal Operations and Litigation at NetApp; Olga Mack, Head of Legal at ClearSlide ; Sharon Segev, VP of Corporate Development & General Counsel at Elo Touch Solutions; and Alexandra Sepulveda, Deputy General Counsel at Udemy.

Beyond the panelist’s tips and advice came a fair portion devoted to horror stories involving various errors by outside counsel. As Sepulveda commented, “How is it we are still having the same conversation about how to get personalized pragmatic advice?”

One running theme for these tech-savvy in-house counsel is that outside firms must use the billing and tech management software their clients use. It is the firm that should adapt and adopt, not the client.

Each in-house counsel said that excellence and understanding a client’s business will only get a firm in the door. In other words, it is expected, and not the exceptional. NetApp’s Michael Haven said the differentiators are:

  • new fee models, 
  • new technology to manage projects and 
  • proactive efforts to improve the attorney-client relationship. 

Haven drove the point home by commenting that “we want e-billing, so we can derive metrics. We would love to see them firms giving us metrics about their own business, and showing us how they are working more efficiently for us.”

Each panelist said they want hours and bills updated daily or three days at the latest. Haven likes ViewaBill, which allows him to see daily expenses. Serengeti also received high marks. “I can’t endure the expense of waiting for bill and then trying to mitigate the damage.” In other words, avoid surprises at all costs.

Haven says about 150 firms have adopted ViewaBill. “We see only our matters in real time. I can see what my firm billed me yesterday on a matter,” he said. Such software avoids nightmares that could cost a firm its slot on the outside counsel panel.

Olga Mack told of one patent matter which came in $500,000 over budget. As she went back through the bills, she discovered the law firm had increased its rates 15% Jan. 1, in the middle of the project, and never informed her. The firm lost its place on her panel almost immediately.

In addition to billing issues, there are technology habits within law firms that drive tech company GCs crazy. An enormous pet peeve is attorneys and firms that won’t use electronic signatures. Nearly all tech firms strive for paperless offices, and demanding inked hard copies is SOOO 20th century. Emails with “FYI” in a subject line are equally hated. In-house counsel want timely information they can scan immediately, with email subheadings such as “EVENT,” “IMPORTANCE,” and “ACTION” to be TAKEN.”

Tech GCs want quick-and-dirty answer in hours. They will wait longer for more complicated questions, but want to know their attorney is on it by acknowledging their email. A reply within 24 hours is ideal; three days is the outer limit. If attorneys can comply, in-house promise they won’t stage “fire drills” for information they don’t need immediately.

Mack also said she immediately judges an attorney on the quality, both the writing and content, of an attorney’s client alerts, as well as what’s on social media, such as LinkedIn and elsewhere. “I make an immediate judgment as to do I want to meet this person, and do I find this useful. I admit it is a bit like judging a book by its cover.” See her recent LinkedIn post on “The Art and Science of Being Useful to In-House Counsel.”

Additionally, more in-house counsel are now on Facebook and Twitter, not just LinkedIn.
Equally important are attorney bios, where in-house counsel typically start their research. 78% of general counsel use bios when choosing outside counsel.

Internally, in-house counsel share information about various attorneys and firms. A lot. Haven, who manages a global team of 80, says NetApp has “our own little version of FaceBook that we use internally. We collaborate on how we are dealing with outside partners, and discuss the pros and cons of certain partners.”

Regarding maintaining a strong relationship with a client, these panelists said they are amazed more firms don’t ask for 360 reviews of how the firm is doing.

In summary, here are the takeaway themes:

Selection/Competition

It has to be spot-on experience to get hired. It used to be OK to have general experience and good service, but now that’s just a gating issue.

Communication

Unless someone is on vacation, responding within 24 hours is the low bar. GCs expect outside counsel to at least communicate that you got the email even if you can’t do anything right now.

Get in Our Shoes and Stay There

If you are going to be a very effective advocate for the client, you need to understand what they do, how they make money, and what risks they face. Then your approach is more tailored to their goal.

Image [cc] Lucy Kimbell

I was watching Tim Corcoran’s video on “Useful Metrics & Benchmarking” and it made me think about how some of the metrics and benchmarking strategies apply to information professionals within a law firm environment. Are we, as managers of these professionals, giving them the right type of feedback that contributes to the overall strategy of the group, or are we asking them to hit benchmarks which do not mesh with the strategy? Tim makes a statement that “You can’t manage well what you can’t measure well.” Of course, Tim is talking about law department metrics, but I’m pretty sure the same concepts are applicable to law firm information professionals.

 Here’s Tim Corcoran’s video. I have borrowed heavily from around the 7:36 through the 10:46 portion of the video (with Tim’s permission), and applied it to measuring the efforts of the Information Professional.


 Metrics and Benchmarks for Law Firm Information Professionals
So what are Law Firm Information Professional metrics today? Let’s borrow and edit the list that Tim uses for law departments:

  • Costs (rates, hours, total cost)
  • Ratios (attorneys:InfoPros)
  • Write offs/downs
  • Repeat Work
  • Additional Costs / Adherence to Budget
  • Service (responsiveness, experience/specialty)

Let’s think about each of these and how we benchmark them and score our InfoPros to these metrics. I want to talk about these on a high-level, and not get into the minutia of what does and doesn’t apply on an individual level.

Costs – pretty straight forward metric of finding out how much the firm is actually paying for this person.
Ratios – this could be attorney to InfoPro, or it could be Practice Group or Office to InfoPro. However you want to measure it, just make sure it is consistent across the staff so you don’t start getting apples to oranges comparisons.
Write offs/downs – Again, pretty straight forward measurement of what did they bill versus what did we bill the clients… and what did the clients actually pay.
Repeat Work – Are attorneys or groups returning and asking for more work from the individual? I know that many of us, my group included, set up a pool of InfoPros to handle work as it comes in, but I think most of us have a reality that attorneys become comfortable with certain people that do good work for them, or are conveniently located to them.
Additional Costs / Adherence to Budget – Are the InfoPros aware of budget restraints for certain clients, and following those rules? If clients do not pay for online legal research, are they going in and using these tools anyway, or are they finding alternative methods to get information that will not cause additional write offs on the client invoice?
Service – How responsive are they? Do individual InfoPros have specialty knowledge of certain practice areas or are subject specialists with unique access to individual resources (usually allocated to them because of licensing issues or overall cost of these specialty resources)?

Mission and Strategy vs. Evaluation and Goals
Do the metrics we measure match our strategy? I’m going to guess that most of us have a mission statement (whether implied or official) that says something like this:

Our department serves the broad needs of our internal and external clients with a highly knowledgeable staff providing exemplary research results in an effective and efficient manner.

I’m sure there are a thousand different ways to say it, but effectively we have a mission of having high-quality people with great knowledge and research/analytical skills who get great results back to the client in a quick and cost effective manner. If that is our mission and strategy, are we measuring our people on those items and encouraging them to fulfill the strategy, or are we rating them and giving bonuses/pay increases or promotions on other metrics? Are bonuses and raises tied only to length of time at the firm, or are there measurable items used to determine how these are allocated? Are there non-monetary incentives available to reward those who score well on certain benchmarks.

Scorecards
Can we take Tim’s idea of using Scorecards to let the InfoPros know where they stand within the firm and perhaps even against their peers? If the only time that your staff knows how well they are doing against the benchmarks which they are being measured is at review time, then that’s bad management on your part and unfair to everyone.

On Tim’s example of scorecards, he lists a number of measurable subjects that can easily apply to Information Professionals, but in reality, it needs to be modified to fit the benchmarks you are asking them to hit. The scorecard should both be presented as a constantly updated piece of information, as well as a snapshot of how well the person is doing in a set period of time (monthly, quarterly, etc.)

Again, borrowing from Tim’s list, let’s think about what InfoPros would see on their scorecards, and where they stand in relationship to those benchmarks.

  • Practice Summary – What have they worked on? Bullet points of projects assigned and completed. Perhaps this piece of the scorecard is kept by the InfoPro themselves, or a joint effort. 
  • Top Billed Matters/Clients – What matters or clients are InfoPros constantly asked to cover? Perhaps expand that to practice areas if they are subject experts.
  • Spend by Practice Group – What are the costs associated to the InfoPro when they are asked to perform a task? Look at time spent, resources used and charged to clients. Don’t forget to measure things like training or client development tasks that may not show up on the client invoice.
  • Tasks or Hours Worked vs. Peers – I know that some of us don’t like this type of competition within the group, but if we are going to allocate bonuses or future pay based on how well they compare to others within the group, it may be fair to expose that information throughout the year so that they know where they stand.
  • Client satisfaction – what type of feedback are they given from those requesting their assistance? Can you get measurable feedback (1 to 5 scale) from internal clients? Perhaps your reference 
  • Adherence to Budget – are they following guidelines, or are there additional expenses incurred when using certain InfoPros?
  • Time Entry Turnaround – Is there a lag between time worked for a client, and the time entered for that client?

Metrics vs. Gut Feeling
Tim doesn’t get into this area directly, but I think it is a logical step in this conversation as most of us may be very uncomfortable using these types of metrics to judge people who report to us on a daily basis. It would be much easier to simply give feedback and measure performance based on what we experience with the Information Professional on a personal level. I’m not saying that personal interaction and experiences should not be used in evaluations, but it should not be the only measurement. There simply has to be benchmarks that have clearly defined measurements, and transparent to those being measured as to where they stand within the benchmark.

Where the gut-feelings and individual experience comes into play is putting a narrative to the metrics. Metrics give us data, but may not tell a complete story. One prime example would be that some individuals may have high write-offs simply because they do a lot of work for attorneys that won’t pass along their costs to the client. This is where the periodic reviews of the scorecard comes in handy. By setting the metrics and monitoring them, you begin to understand the story behind the data at an earlier point in time. This will allow you and the InfoPro to take corrective actions either by correcting internal behavior, or client behavior.

I’ll bring Tim’s comment back of “You can’t manage well what you can’t measure well.” Establishing metrics and benchmarks based on the overall strategy of the department and the firm will help the InfoPros work toward that strategy, and will allow you to measure how well they are doing in achieving that strategy, as well as how well your overall department is working toward promoting the firm’s overall strategy.

Benchmarking the Department
I’m going to borrow one last thing from Tim from around the 10:30 mark in his video where he talks about the law department leveraging the scorecards as an integral part of the overall business needs of the company, and apply it to the efforts and mission of the Information Professional department.  In establishing clear benchmarks that drive overall strategy of the firm, you can leverage this process and become so closely linked to the law firm’s strategy, that your groups becomes a competitive advantage for the firm. Firms are always looking for a cost advantage, or a unique skill set for its marketing to clients. How about leveraging your department to help the firm get to clients faster, overcome obstacles more quickly, manage information and knowledge more effectively so that we operate more effectively in our markets. Tim concludes this section of establishing metrics that show your impact on the overall business with a thoughtful statement. “Think about that. A metric that helps us understand the impact we’re having on the business throughput can be pretty powerful.”

Thanks again, Tim, for letting me morph your legal department concepts for metrics and benchmarking onto the world of legal information professionals.

Today, Jones McClure Publishing is now simply O’Connor’s. For those of us in Texas, this won’t be a huge surprise or much of a change in the way we think of the company’s brand. Most of us call everything they publish after Judge Michol O’Connor, who started the company nearly twenty-five years ago, and is still involved in the company although her son, Baird Craft, is now the President of the company. The biggest deal will probably be sending a note to the Accounting Department letting them know to change the vendor name on the accounts payable system, and some changes in the website address (now oconnors.com) and having to update my email contacts.

Now I could have mis-remembered the story of how Judge O’Connor came up with the “Jones McClure” name of the company, but it was definitely a made up name, since there were no Jones or McClures involved in the creation of the company. At one of the AALL annual conference dinners, Judge O’Connor got up and told the story, and I seem to remember that the company was named after a couple of the delegates from the Second Consultation and Convention where Texas sought independence from Mexico. Delegates Oliver Jones, and Bartlett D. McClure were present during the April, 1883 convention and thus, the name Jones McClure was created. I see that Charles Baird was also a delegate, so I assume not only was the company named after some of the delegates, but also the current President of the company. (Baird, forgive me if I am jumping to too much of a conclusion here… as, there may have been a couple of glasses of wine consumed during the story.)

One of my favorite people, Jason Wilson, is the Vice-President of O’Connor’s, and I asked him for a story to tell about the name change. Here is what he had to say:

For years, whenever I would meet someone cold—say an attorney or librarian—and told them I was with Jones McClure Publishing, I would always get a polite nod. But when I followed it up with, “We publish O’Connor’s,” the reactions were always the same: “Oh, I love you guys! I have all your books.” This is an experience each of us at the company has had. So, when we moved into the digital space with O’Connor’s Online and then started planning for the new web store, we decided to retire Jones McClure Publishing in favor of the brand we’ve worked so hard to create and something our customers recognize immediately as quality products and services. So now we are just O’Connor’s.

For those of us that rely upon the twenty-five titles published by O’Connor’s, or the new online resources, and those attorneys we support who enjoy the detailed information provided within those titles, in an easily understood writing style, we applaud the official name change to better recognize Judge O’Connor’s vision of creating legal books, minus the legalese.

[Ed. Note: Updated at 2:02 to include Wilson quote. – GL]

When I graduated from high school, I knew three languages. I was fluent in two and had a fairly good working knowledge of the third. Today, my second and third languages are a bit rusty but I can get by when spoken to or making an inquiry.  Yet, I feel compelled to learn another bunch of languages. I fear that if I don’t I won’t be able to talk to my children let alone my future grandchildren, I won’t be able to maintain my job or advance in my career and I most definitely will lose any trace of being as a worldly individual if I can’t “speak” Ruby, Python, SQL, Java or any other of the languages Mashable tells me I need to learn right now. I’m not kidding, I have looked at Women Who Code, Lynda.com and other sites, unsure of where to start and how to deal with the overwhelming sense of coding disability.  I feel pressure to be something I am not in response to the changing market. Apparently, even long after high school, peer pressure doesn’t go away, everyone is coding and I need to too! 

For me, coding is symptomatic of a bigger fear. A fear that until recently I thought I was shielded from in some respects, being a law firm marketer, business development, and CI person.  My real fear is Big Data.  The stuff collected passively in the background, sometimes for a reason, other times just because the technology is there to collect it.  Law firms are jumping on the Big Data/BI train, a train travelling so fast that Dennis Hopper would be proud.  True, I did collaborate on a book some three years ago with the Ark Group on Business Intelligence for law firms and in that publication, I did write about data and how firms can increase efficiency and provide better value to clients using scraped data and robust analytic techniques. All of which, still stands. I do believe that there is power in the data we collect and even more power to be harnessed if we can find meaningful ways to make sense of that data.
I have no illusions of becoming a data scientist, nor do I suggest that all law firm marketers need to be comfortable dabbling in data, but data, like “information” circa 2005, data has become ubiquitous and the legal industry is far from immune.  In August, DataFloq the self described “One Stop Shop for Big Data” published a post on how a variety of law firms are tapping into the data revolution.  You can read it here: How Big Data Can Improve the Practice of Law . Companies and thought leaders blogged about on 3 Geeks in the past, such as LexMachina or David Perla of BloombergLaw openly discuss the impact data has, will have and should have on both the practice and business of law.   Some firms, like Littler Mendleson have even created roles for national directors of data analytics in an effort to woo general counsels. 
Big Data, like the coding languages and algorithms needed to support extracting insights from the data is real and not going away.  We live now and forever more, in a data driven world where we have the capacity with relative ease to know, for example, that a clause we are putting into an agreement is used 5% of the time in a industry, 76.6% of the time in a type of contract, and .9% of the time in a particular jurisdiction.  The same intensity of analytics can be applied to the kind of coffee we drink, the clothes we buy or litigations our firms undertake and their probable success rates.  Statistical probably will soon run the world.    That’s what scares me.  I’m all for data driven hypothesis and fact based learnings.  But I think there is something to be said for the human element – the syntax, the colour, the qualitative nature and value of being graciously flawed and human. In law firms there are vast amounts of unstructured data, the art and craft of lawyering is often about interpretation and word nuance. 
There have been cases won and lost on the placement of commas in an agreement, beautiful examples of the dynamic nature of language that no machine learning, no data as I see it, will ever be able to understand.  We need data and the ever increasing pace at which we are expected to respond to client needs demands that we use the data available to us in new ways, requiring us to speak to new languages. I am not naïve to this I am scared though, that as we wade through the data, we will drown in a pool of objective facts and figures.  I liken it to knowing the score at a sporting event without knowing what happened, player and team stats will go up and down, predictions about winners and losers can be made but the great plays, the epic bat flips and the amazing free shots would be lost.  Imagine sports coverage as score keeping without the colourful narrative. This is what I fear will happen if we spend too much letting Big Data and analytics aid and even replace our subjective discussions and decision making. Not everything can or should be reduced to a series of numbers and equations.  There is value in the narrative, nuggest of gold in the telling of the story. 
The idea of Big Data is hard to resist, like the shiny glint of a smoking silver bullet in a haystack (metaphors mixed for emphasis). We like Big Data because its clean and seemingly perfect. It lets human imperfection off the hook.  But we know from experience and the history of humanity that we are not perfect, there is rarely a silver bullet, and statistics, though based in numbers can often be skewed or misinterpreted.  So humour my fears as we propel ourselves forward at a rapid pace on the Big Data train in law firms and in life and lets add some colour to our use of Big Data. 

Tips on coding lessons welcome and accepted. 

90% of people don’t know how to use CTRL+F to find a word in a document or web page. Instead, they search the old-fashioned way, manually skimming the text.

This preponderance of ignorance is stupefying to me. But I want to be very clear that I am using the word “ignorance” in its most neutral form–i.e., lack of information or knowledge–rather than to convey any judgment or pejorative connotation. Ignorance is unavoidable. The only settled part of the debate as to who was the last person to know everything is that the person is long dead.

The curse of ignorance is that you don’t know what you don’t know. Previous posts have touched on this obstacle of metacognition, and our ignorance of our own ignorance. But there is another side of the coin: the curse of knowledge. The curse of knowledge is that once we know something, it is really hard to imagine not knowing it. This incapacity undermines communication and, especially, instruction because of the lack of shared information and assumptions. If I, for example, were going to put together some tips on internet research, I doubt that, absent the article cited above, I would have thought to include CTRL+F. I would have assumed that most everyone already knew it. I would have been wrong.

Indeed, I am a posterchild for both curses. I’ve told the story many times that my inflection point in using technology involved a client discovering that I printed and scanned to create PDFs. But how was I supposed to know what I didn’t know–there’s an app for that–without already knowing it? Yet, several years later, I delegated a task where one of the steps involved converting a large volume of documents into PDF. I was shocked (shocked!!!) to find that the person was spending hours printing and scanning. I assumed that because I knew how to convert a file to PDF, they knew it, too, despite the fact that I had been Exhibit A that this was not knowledge everyone possessed.

Thus, whether we know something or not, we too often assume that others know it. The tech-averse frequently fall into the trap of thinking the tech-comfortable know everything there is to know about tech. And those who know tech sometimes assume that others do, too. Both curses are reason that competence-based assessments are such excellent training tools. Figuring out what people do and do not know is superior to speculation. But assessments alone are not enough. The primary objective of identifying gaps is to tailor the training to fill them. In this regard, I have been an abject failure in speaking to law school classes.

I speak to law school classes for free. I provide them a copy of my Legal Technology Assessment (“LTA”) for free. I then provide a copy of the LTA Training Edition (which pairs the competence-based assessment with synchronous, active learning) for free. Finally, they can retake the LTA (for free). Not only do have the opportunity to address identified deficiencies in their skill but a qualifying score is also something they can add to the bottom of their resume to replace the meaningless “proficient in MS Office.” After speaking to hundreds upon hundreds of students, I’ve had exactly zero take me up on my full offer.

The class I wrote about last week is representative. Twelve students took the LTA because it was a class assignment. The results (below) were bad, as usual. I spoke to them for 40 minutes and offered the Training Edition to anyone who wanted it. Only two of the twelve emailed to ask for the Training Edition. And, if history is any guide, neither of them will return to take and pass the LTA.

In approaching these classes, my idea is that taking the LTA beforehand will puncture delusions of adequacy. We won’t get bogged down in an abstract conversation about how fluent they are with technology. 32% correct on some fairly simple Word tasks leaves little room for debate:

Pretty bad but not unexpected. As I try to communicate to them, it is not their fault. Everyone just assumes that they know things that they had no way of knowing absent training. They are not stupid, lazy, or untalented. They are smart, hard working, and full of promise. They simply lack training in one particular area that has the potential to make their lives better.

On the issue of their immediate future, I point out that their most recent predecessors are miserable human beings. In fact, the students are auditioning for the unhappiest job in America.

I then try to persuade them that technology plays a role in this dissatisfaction. Before technology takes our jobs, it can make them easier. At least, in theory. The technology has to actually be good, and we have to use it correctly. Otherwise, it is a source of frustration rather than leverage. Technology initially substitutes for labor at the most severe pain points. Machines can reduce the hours spent reviewing, proofing, conforming, collating, updating, and otherwise fiddling around the edges of the substantive legal work. Using technology well can improve both speed and accuracy, as I try to convey in the video below, and thereby alleviate a fair amount of the agony associated with being a young lawyer:

My contention is that having the right technology and learning to use it correctly will permit legal professionals to reduce the amount of their finite time and attention that is directed towards misery-inducing busywork. I’ve added to my spiel some recent confirmation of this theory from the cover story of last month’s American Lawyer. AmLaw’s annual associate satisfaction survey found that technology, including technology training, has a material effect on satisfaction:

One unsung key to retention could be technology. We found that overall satisfaction of midlevel associates, as measured on our survey, was strongly statistically correlated to their law firm’s scores on four questions involving technology. (The questions ask respondents to rate their firms’ technology generally, as well as technology training, support and use of technology in meeting client needs.) 

….In fact, eight of the top 11 firms in the national satisfaction rankings also were at the top on the technology questions. Conversely, many of the firms that occupy the bottom of the national satisfaction rankings also place low in the technology survey.

The AmLaw conclusions comport with an earlier study I cite from the National Conference of Bar Examiners that surveyed recent law graduates about the most important skills for young lawyers. Out of 30 skills, using basic office technology ranked 6th:

Seeing basic office technology ahead of legal reasoning is a bit jarring, even for me. But the incongruence is heightened by the fact that, unlike the rest of the skills listed above, using technology is not taught in most law schools (or, generally, in most colleges or high schools). 
Then again, the idea that law school is not geared towards turning out practice-ready lawyers is well-worn territory. As discussed in Mark’s previous post, a LexisNexis survey found that “95% of hiring partners and associates believe that recently graduated law students lack key practical skills.” The dissatisfaction of associates is mirrored (and, maybe partially driven) by the dissatisfaction with associates. This is not just abstract griping. Anecdotally, partners report writing off massive amounts of associate time for perceived inefficiency. These claims appear to be borne out by the Georgetown Law and Peer Monitor realization data (which I dug into here):
 

So that’s my story. You’re great. You just haven’t gotten the training you need in technology. This training will benefit you directly in the form of improved satisfaction and performance. Here it is, for free. Followed by crickets.

 I’m not quite sure how to interpret my utter inability to make any progress with these students (thankfully, the people who actually pay me are considerably more engaged). Am I, yet again, suffering from the curse of knowledge? Is there some assumption that I am making about these students that is impeding communication? As I try to put myself in their shoes, I increasingly come to conclusion that there isn’t anything I can say.

In general, it is challenging to get anyone to use their precious spare time to buckle down and really learn something new, even if they are persuaded that they should. The last time I decided to tackle a new area of study, I felt compelled to pay for online courses that included tests and graded assignments. I needed real stakes and real structure to have the discipline to systematically engage with the material (all of which I could have found for free on the internet). Here, the students took the LTA as a diagnostic because it was an assignment, and, I have no doubt, that they would have trained for and passed the LTA if that were assigned. As a law student, I suspect I would have behaved much the same way (I know my scores would have been just as bad).

Stakes and structure matter. These students have had both all their life. From speaking to them, I get the sense that they believe this will continue. They believe that law school is designed to prepare them for law practice. They believe that whatever they do not know upon leaving law school, their firms will teach them. And, more than anything, they believe that they do not need to worry about this tech stuff because they will have secretaries to do it for them. More on that last point in my next post.

For me, the primary myth of the digital native is that, by virtue of their age, they already know what they need to know with respect to using technology. The corollary myth is that which they do not already know is not worth learning. But there exists a softer formulation that hits much closer to the truth. Rather than automatically knowing that which they need to know with respect to technology, we (and they) tend to believe that people who grew up with technology have the capacity to learn it and will do so when the situation requires. It’s that last part, however, where there continues to be a disconnect.

The older generations seem to think that the situation will somehow mandate the acquisition of new skills. In this, they are not totally wrong. Most people, including the older generations themselves (with their fancy new iPhones and Surface Pro 4’s), learn what they need to learn to get by with technology. Some people learn more. But most satisfy the bare threshold of survival. This results in massive underutilization of extant technology. And study after study has shown that younger generations are the same as their predecessors in this regard–i.e., learn the minimum to get by.

The younger generations, on the other hand, think that they will quite literally be required to learn it. Someone in a position of authority is going to lay out a curriculum, objectives, and a timeline. At that point, they will do what they’ve always done: work hard to meet the expectations set for them. A few will fall short. Some will excel. But most will quite effectively do what they are asked to do. I, for one, think we ought to oblige them.

At some point, I will dig deep into my data. But, on average, people (lawyers and staff) in practice outperform the kids in school on the LTA. In part, this reflects a general raising of the baseline as the skill set required for bare survival expands upon entering the professional workforce. But there is still significant interorganizational and intraorganizational variance.

The variance between organizations appears to be entirely attributable to mandatory training. Different organizations have different attitudes towards training (is it available, is it mandatory, does it include competence-based assessments) that, unsurprisingly, have an appreciable impact on how well trained their employees are. The variance within organizations stems from outside training. Frequently, I learn that the person who outpaced her colleagues on a diagnostic assessment had some previous career that demanded a more robust technology skill set. Sometimes, I meet people who, like me, had some sort of rude awakening and decided they did not like being embarrassed. Every now and then, I encounter a true tech geek (meant with love and affection) who happens to also work in law. My own data reinforces previous empirical findings that, rather than age, facility with technology is a product of “breadth of use, experience, self-efficacy and education.”

Technology training is important for everyone, including the digital natives. I just wish I could convince them of that.

++++++++++++++++++++++++++++++++++++

Casey Flaherty is the founder of Procertas. He is a lawyer, consultant, writer, and speaker focused on achieving the right legal outcomes with the right people doing the right work the right way at the right price. Casey created the Service Delivery Review (f.k.a., the Legal Tech Audit), a strategic-sourcing tool that drives deeper supplier relationships by facilitating structured dialogue between law firms and clients. There is more than enough slack in the legal market for clients to get higher quality work at lower cost while law firms increase profits via improved realizations.
The premise of the Service Delivery Review is that with people and pricing in place, rigorous collaboration on process offers the real levers to drive continuous improvement. Proper collaboration means involving nontraditional stakeholders. A prime example is addressing the need for more training on existing technology. One obstacle is that traditional technology training methods are terrible. Competence-based assessments paired with synchronous, active learning offer a better path forward. Following these principles, Casey created the Legal Technology Assessment platform to reduce total training time, enhance training effectiveness, and deliver benchmarked results.

Connect with Casey on LinkedIn or follow him Twitter (@DCaseyF).

I am disappointed every time I guest lecture a law school class.

Because anecdote is often more compelling than data, I’ll start with an example from two weeks ago. An adjunct professor who teaches one of those great law school classes with cool titles like Tomorrow’s Lawyer had his students take the Word module of my Legal Tech Assessment. They performed exactly as you (well, I) would expect of an untrained group: poorly. Their anonymized scores are below:

I’ll explain the scoring in a subsequent post. But, for now, just focus on the accuracy. On average, students were able to correctly complete less than a third of the following tasks in a live Word document:

  • Accept/Turn-off changes and comments
  • Cut & Paste
  • Replace text
  • Format text (font, margin)
  • Footers
  • Insert hyperlink
  • Apply/Modify style
  • Insert/Update cross-references
  • Insert page break
  • Insert non-breaking space
  • Clean document properties
  • Create comparison document

To put it in anecdotal context (though these numbers are representative of the larger data set), around the same time the class was taking the assessment, a similarly-sized pilot group at a large law firm achieved an average accuracy of 68% while a group of lawyers and staff who had just been through LTA-specific training achieved an average accuracy of 95%. Training matters.

Training matters even for so-called digital natives like the law school students I’ve tested. Acquiring a Twitter account in utero does not engender natural facility with technological tools because most technological tools are not intuitive.

My wife thinks my sons are geniuses. One piece of evidence she submits in their favor is how well they use an iPad. I agree with her that the fact that my 1.5 year old can use an iPad is a testament to genius. But not his. For me, it is a testament to the genius of the designers at Apple who created a device so intuitive that 1.5 year old can use it. The kid touches a picture, it moves. Congratulations to him!

While we can discuss design principles that would move work software in the direction of our consumer experience, I don’t actually believe that individual apps are the best basis of comparison for complex business software. Rather than thinking of Word as an app, we should think of it as a bundle of apps. Each of the icons on the ribbon is a solution to a particular problem. The challenge is that there are so many icons (just as there are so many apps). While design principles can bridge some of the gap, there remains a tradeoff between depth and intuitiveness. Most of us therefore only become comfortable with a few functions while ignoring the rest (just as most of us use a limited number of mostly single-purpose apps):

In this, the younger generation is no different than their predecessors. Survival is the threshold they achieve with most of their technology. That their milieu demands facility with a few more social apps sooner does not change the fundamental fact that using technology properly is a collection of acquired skills, not some innate talent that Lamarckian evolution bestowed on those under the age of 30. Expecting them to automatically know how to use complicated technology because of their familiarity with basic technology is like expecting them to automatically know how to prepare a gourmet meal because they know how to cook a Hot Pocket in the microwave.

Digital native first entered the popular lexicon in 2001. In an article entitled Digital Natives, Digital Immigrants, education consultant Marc Prensky explained that “Our students today are all ‘native speakers’ of the digital language of computers, video games and the Internet.” In many respects, the article looks terrible 14 years later. Prensky, for example, speaks glowingly of Digital Natives’ ability to multi-task, despite the fact that contemporary research (subsequently extended and validated) had already demonstrated the opposite. But just because it suffers from some poor assumptions and hyperbole (Prensky claims we’ve already arrived at a singularity) does not mean the thesis needs to be totally rejected.

You can reconcile (i) a belief that, on average, younger generations are more accustomed to technology (and the rapid evolution thereof) than their predecessors with (ii) the recognition that this comfort does not automatically translate into proficiency. Indeed, in his seminal article, Prensky talks about the need to train a bunch of Digital Natives on a CAD program that “contained hundreds of new buttons, options and approaches.” Prensky takes pride in efforts to gamify the training and “create a series of graded tasks into which the skills to be learned were embedded.” For Prensky, this approach translated training into the language of the Digital Native. He reports that the main impediment was the reluctance of the Digital Immigrant professors to adjust their pedagogical approach. But even Prensky realized that the Digital Natives still needed training (competence-based assessments paired with synchronous, active learning).

Prensky’s article was anecdotal, not empirical. It did not address the fact that most of what Digital Natives did with technology was related to consumption, not application. It also ignored the inconvenient fact that it was Digital Immigrants who had invented the technologies on which the Digital Natives relied. And it introduced a term that conflates general familiarity with specific facility. A decade later, the London School of Economics would publish a paper entitled Digital natives: where is the evidence? The paper concluded that there was no real evidence of fundamental differences between generations. What differences existed were best explained by “breadth of use, experience, self-efficacy and education.” Or as another academic paper would find:

Young people’s engagements with digital technologies are varied and often unspectacular – in stark contrast to popular portrayals of the digital native. As such, the paper highlights a misplaced technological and biological determinism that underpins current portrayals of children, young people and digital technology.

Subsequently, the Organization of Economically Developed Countries (OECD) ran an international study. Instead of asking people about their general comfort with technology, the study asked them to actually solve basic problems using technology (again, a competence-based assessment). Millennials did not fare well. In the words of The Washington Post, “U.S. millennials performed horribly.” Or, as Fortune summarized it:

We hear about the superior tech savvy of people born after 1980 so often that we tend to assume it must be true. But is it?

…. It turns out, says a new report, that Millennials in the U.S. fall short when it comes to the skills employers want most: literacy (including the ability to follow simple instructions), practical math, and — hold on to your hat — a category called “problem-solving in technology-rich environments.”

The advocacy group Change the Equation put out a related report about the High Cost of Low Technology Skills, which included the following graphics:

Digital natives are not at fault for the fact that comfort does not automatically translate into skill. The myths surrounding the digital native, however, have done them a disservice. The belief that they already know everything about technology has convinced us and them that they do not need training in technology. As discussed in the last post, the notion that they are already tech savvy introduces barriers associated with metacognition and mindset.

Metacognition is thinking about thinking. The related concept of metaignorance is ignorance about our own ignorance. We don’t know what we don’t know. Not recognizing how incompetent we are results in unfounded confidence in our own skill level. For this reason, the people most in need of training are the least likely to recognize it. Moreover, confidence begets ego. Those who have a high opinion of themselves are the least inclined to admit facts that undermine their self image.

The problems of ego are compounded by mindset. People with a fixed mindset believe that cognitive traits are stable. You are either smart or you aren’t in the same way that you are either tall or you aren’t. You are born with it, or you are not. To the extent tech acumen is treated as product of age, it will be approached with a fixed mindset, which means that people will try to hide their deficiencies rather than recognize and remedy them.

Because we take a fixed mindset approach, we tend to act as if neither older professionals nor younger professionals will benefit from training. The former because they lack the capacity. The latter because they lack the need. We are wrong on both counts.

I am not trying to pick on anyone. I understand why both older and younger generations buy into the myths surrounding digital natives. But they are myths with pernicious consequences. Many of the decisions about training (or lack thereof) made at law schools and legal employers rest on an illusory foundation. It isn’t always what we don’t know that gives us trouble, it’s often what we know that ain’t so. 

Part 2 (originally posted separately)

90% of people don’t know how to use CTRL+F to find a word in a document or web page. Instead, they search the old-fashioned way, manually skimming the text.

This preponderance of ignorance is stupefying to me. But I want to be very clear that I am using the word “ignorance” in its most neutral form–i.e., lack of information or knowledge–rather than to convey any judgment or pejorative connotation. Ignorance is unavoidable. The only settled part of the debate as to who was the last person to know everything is that the person is long dead.

The curse of ignorance is that you don’t know what you don’t know. Previous posts have touched on this obstacle of metacognition, and our ignorance of our own ignorance. But there is another side of the coin: the curse of knowledge. The curse of knowledge is that once we know something, it is really hard to imagine not knowing it. This incapacity undermines communication and, especially, instruction because of the lack of shared information and assumptions. If I, for example, were going to put together some tips on internet research, I doubt that, absent the article cited above, I would have thought to include CTRL+F. I would have assumed that most everyone already knew it. I would have been wrong.

Indeed, I am a posterchild for both curses. I’ve told the story many times that my inflection point in using technology involved a client discovering that I printed and scanned to create PDFs. But how was I supposed to know what I didn’t know–there’s an app for that–without already knowing it? Yet, several years later, I delegated a task where one of the steps involved converting a large volume of documents into PDF. I was shocked (shocked!!!) to find that the person was spending hours scanning and printing. I assumed that because I knew how to convert a file to PDF, they knew it, too, despite the fact that I had been Exhibit A that this was not knowledge everyone possessed.

Thus, whether we know something or not, we too often assume that others know it. The tech-averse frequently fall into the trap of thinking the tech-comfortable know everything there is to know about tech (i.e., they can’t tell the difference between someone who knows slight more than them and someone who knows infinitely more than them). And those who know tech sometimes assume that their tech knowledge is widely shared.

Both curses are reason that competence-based assessments are such excellent training tools. Figuring out what people do and do not know is superior to speculation. But assessments alone are not enough. The primary objective of identifying gaps is to tailor the training to fill them. In this regard, I have been an abject failure in speaking to law school classes.

I speak to law school classes for free. I provide them a copy of my Legal Technology Assessment (“LTA”) for free. I then provide a copy of the LTA Training Edition (which pairs the competence-based assessment with synchronous, active learning) for free. Finally, they can retake the LTA (for free). Not only do have the opportunity to address identified deficiencies in their skill set but a qualifying score is also something they can add to the bottom of their resume to replace the meaningless “proficient in MS Office.” After speaking to hundreds upon hundreds of students, I’ve had exactly zero take me up on my full offer.

The class I wrote about last week is representative. Twelve students took the LTA because it was a class assignment. The results (below) were bad, as usual. I spoke to them for 40 minutes and offered the Training Edition to anyone who wanted it. Only two of the twelve emailed to ask for the Training Edition. And, if history is any guide, neither of them will return to take and pass the LTA.

In approaching these classes, my idea is that taking the LTA beforehand will puncture delusions of adequacy. We won’t get bogged down in an abstract conversation about how fluent they are with technology. 32% correct on some fairly simple Word tasks leaves little room for debate:

Pretty bad but not unexpected. As I try to communicate to them, it is not their fault. Everyone just assumes that they know things that they had no way of knowing absent training. They are not stupid, lazy, or untalented. They are smart, hard working, and full of promise. They simply lack training in one particular area that has the potential to make their lives better.

On the issue of their immediate future, I point out that their most recent predecessors are miserable human beings. In fact, the students are auditioning for the unhappiest job in America.

I then try to persuade them that technology plays a role in this dissatisfaction. Before technology takes our jobs, it can make them easier. At least, in theory. The technology has to actually be good, and we have to use it correctly. Otherwise, it is a source of frustration rather than leverage. Technology initially substitutes for labor at the most severe pain points. Machines can reduce the hours spent reviewing, proofing, conforming, collating, updating, and otherwise fiddling around the edges of the substantive legal work. Using technology well can improve both speed and accuracy, as I try to convey in the video below, and thereby alleviate a fair amount of the agony associated with being a young lawyer:

My contention is that having the right technology and learning to use it correctly will permit legal professionals to reduce the amount of their finite time and attention that is directed towards misery-inducing busywork. I’ve added to my spiel some recent confirmation of this theory from the cover story of last month’s American Lawyer. AmLaw’s annual associate satisfaction survey found that technology, including technology training, has a material effect on satisfaction:

One unsung key to retention could be technology. We found that overall satisfaction of midlevel associates, as measured on our survey, was strongly statistically correlated to their law firm’s scores on four questions involving technology. (The questions ask respondents to rate their firms’ technology generally, as well as technology training, support and use of technology in meeting client needs.) 

….In fact, eight of the top 11 firms in the national satisfaction rankings also were at the top on the technology questions. Conversely, many of the firms that occupy the bottom of the national satisfaction rankings also place low in the technology survey.

The AmLaw conclusions comport with an earlier study I cite from the National Conference of Bar Examiners that surveyed recent law graduates about the most important skills for young lawyers. Out of 30 skills, using basic office technology ranked 6th:

Seeing basic office technology ahead of legal reasoning is a bit jarring, even for me. But the incongruence is heightened by the fact that, unlike the rest of the skills listed above, using technology is not taught in most law schools (or, generally, in most colleges or high schools). 
Then again, the idea that law school is not geared towards turning out practice-ready lawyers is well-worn territory. As discussed in Mark’s previous post, a LexisNexis survey found that “95% of hiring partners and associates believe that recently graduated law students lack key practical skills.” The dissatisfaction of associates is mirrored (and, maybe partially driven) by the dissatisfaction with associates. This is not just abstract griping. Anecdotally, partners report writing off massive amounts of associate time for perceived inefficiency. These claims appear to be borne out by the Georgetown Law and Peer Monitor realization data (which I dug into here):
 

So that’s my story. You’re great. You just haven’t gotten the training you need in technology. This training will benefit you directly in the form of improved satisfaction and performance. Here it is, for free. Followed by crickets.

I’m not quite sure how to interpret my utter inability to make any progress with these students (thankfully, the people who actually pay me are considerably more engaged). Am I, yet again, suffering from the curse of knowledge? Is there some assumption that I am making about these students that is impeding communication? As I try to put myself in their shoes, I increasingly come to conclusion that there isn’t anything I can say.

In general, it is challenging to get anyone to use their precious spare time to buckle down and really learn something new, even if they are persuaded that they should. The last time I decided to tackle a new area of study, I felt compelled to pay for online courses that included tests and graded assignments. I needed real stakes and real structure to have the discipline to systematically engage with the material (all of which I could have found for free on the internet). Here, the students took the LTA as a diagnostic because it was an assignment, and, I have no doubt, that they would have trained for and passed the LTA if that were also assigned. As a law student, I suspect I would have behaved much the same way (I know my scores would have been just as bad).

Stakes and structure matter. These students have had both all their life. From speaking to them, I get the sense that they believe this will continue. They believe that law school is designed to prepare them for law practice. They believe that whatever they do not know upon leaving law school, their firms will teach them. And, more than anything, they believe that they do not need to worry about this tech stuff because they will have secretaries to do it for them. More on that last point in my next post.

For me, the primary myth of the digital native is that, by virtue of their age, they already know what they need to know with respect to using technology. The corollary myth is that which they do not already know is not worth learning. But there exists a softer formulation that hits much closer to the truth. Rather than automatically knowing that which they need to know with respect to technology, we (and they) tend to believe that people who grew up with technology have the capacity to learn it and will do so when the situation requires.

The older generations seem to think that the situation will somehow mandate the acquisition of new skills. In this, they are not totally wrong. Most people, including the older generations themselves (with their fancy new iPhones and Surface Pro 4’s), learn what they need to learn to get by with technology. Some people learn more. But most satisfy the bare threshold of survival. This results in massive underutilization of extant technology. And study after study has shown that younger generations are the same as their predecessors in this regard–i.e., learn the minimum to get by.

The younger generations, on the other hand, think that they will quite literally be required to learn it. Someone in a position of authority is going to lay out a curriculum, objectives, and a timeline. At that point, they will do what they’ve always done: work hard to meet the expectations set for them. A few will fall short. Some will excel. But most will quite effectively do what they are asked to do. I, for one, think we ought to oblige them.

At some point, I will dig deep into my data. But, on average, people (lawyers and staff) in practice outperform the kids in school on the LTA. In part, this reflects a general raising of the baseline as the skill set required for bare survival expands upon entering the professional workforce. But there is still significant interorganizational and intraorganizational variance.

The variance between organizations appears to be entirely attributable to mandatory training. Different organizations have different attitudes towards training (is it available, is it mandatory, does it include competence-based assessments) that, unsurprisingly, have an appreciable impact on how well trained their employees are.

The variance within organizations stems from outside training. Frequently, I learn that the person who well outpaced her colleagues on a diagnostic assessment had some previous career that demanded a more robust technology skill set. Sometimes, I meet people who, like me, had some sort of rude awakening and decided they did not like being embarrassed. Every now and then, I encounter a true tech geek (meant with love and affection) who happens to also work in law.

My own data reinforces previous empirical empirical findings that, rather than age, facility with technology is a product of “breadth of use, experience, self-efficacy and education.” Technology training is important for everyone, including the digital natives. I just wish I could convince them of that.

++++++++++++++++++++++++++++++++++++

Casey Flaherty is the founder of Procertas. He is a lawyer, consultant, writer, and speaker focused on achieving the right legal outcomes with the right people doing the right work the right way at the right price. Casey created the Service Delivery Review (f.k.a., the Legal Tech Audit), a strategic-sourcing tool that drives deeper supplier relationships by facilitating structured dialogue between law firms and clients. There is more than enough slack in the legal market for clients to get higher quality work at lower cost while law firms increase profits via improved realizations.
The premise of the Service Delivery Review is that with people and pricing in place, rigorous collaboration on process offers the real levers to drive continuous improvement. Proper collaboration means involving nontraditional stakeholders. A prime example is addressing the need for more training on existing technology. One obstacle is that traditional technology training methods are terrible. Competence-based assessments paired with synchronous, active learning offer a better path forward. Following these principles, Casey created the Legal Technology Assessment platform to reduce total training time, enhance training effectiveness, and deliver benchmarked results.

Connect with Casey on LinkedIn or follow him Twitter (@DCaseyF).

I am more expert than most of the people in the world about The Wire, James Baldwin, Breaking Bad, David Foster Wallace, the films of Quentin Tarantino, and many, many other topics. This is not because I know much but because I know anything. The baseline used makes my claim to status meaningless. Only a minute percentage of the world’s population has any familiarity with the aforementioned. Of course I know more than people who know nothing. Being more expert than they does not mean I am an expert (I’m not).

Often, when we judge something, including ourselves, we encounter a reference class problem. Last season, was Mike Miller of the Cleveland Cavaliers a good or bad basketball player? Of the already small percentage of the basketball playing population that makes their high school squad, only .03% make it to the NBA. So Miller is, inarguably, among the basketball playing elite. At the same time, in the 52 games his coach chose to play him, Miller only averaged 2.1 points on 32.5% shooting to rank dead last in player efficiency rating among all NBA players. Miller, a 15-year veteran nearing the end of a solid career, was so unproductive that you get 816 hits on Google for  “corpse of Mike Miller”. If you are putting together a pick-up basketball squad at your local rec center, probability suggests that Miller remains a phenomenal addition. If you are building an NBA rotation, you hope to have better options. Playground and NBA players are different reference classes.

When I became a lawyer, I was immediately ordained “tech savvy” because of the reference class. My reputation was sealed when the partner in the adjacent office (an otherwise brilliant lawyer) was screaming about the computer eating valuable documents. I superheroed in to press CTRL+Z (undo). When the files resurrected, he stared at me like I was some sort of wizard. From then on, I handled ediscovery on his cases despite the inconvenient fact that, in the beginning, I knew jack all about ediscovery (that’s changed). It required dangerously little knowledge of, and facility with, technology to be tech savvy among the reference class of lawyers.

It was, of course, one thing for technophobic partners to think I knew everything about technology because I demonstrably knew more than they did. It was yet another for me to believe I was tech savvy in any meaningful sense. Yet, I started to believe just that because, well, ego. After all, I spent my days knowing more about technology than almost everyone I encountered. And those who did know more (IT professionals, word processors, and our bad-ass librarian who switched her keyboard to Dvorak) were not lawyers. The word “lawyer” was doing most of the heavy lifting in the appellation “tech-savvy lawyer.”

I had my delusions of tech adequacy punctured by a client. He happened to be in my office and asked me to turn a document into a PDF. I obliged. I printed the document, walked to the printer, walked to the scanner, and returned to my desk where he was sitting mouth agape. For months, I had been the sole associate on a PDF-intensive arbitration. He was doing the math on how much time I must have already wasted. He was apoplectic. If I had not already proven my value as a lawyer, he would have had me thrown off the case. But I had. And he didn’t. Still, he had words with the partner (the same partner who had designated me as a tech expert).

Let me be clear: I believed I was tech savvy despite the fact that I did not know how to convert a Word file into a PDF.

I was absolutely embarrassed. I changed my whole approach to technology based on the incident. But, at the time, how was I supposed to know? Everything is obvious once you know the answer. It is obvious to me now that converting one filetype (e.g., a Word document) to another filetype (a PDF) is something that the machine should do. But, to that point, no one had ever taught me that. Without training, how was I supposed to know that which I did not know?

This is the problem of metacognition. In the book, Talent is Overrated: What Really Separates World-Class Performance from Everybody Else, Geoff Colvin explains the role metacognition plays in superior performance (h/t Farnam Street):

The best performers observe themselves closely. They are in effect able to step outside themselves, monitor what is happening in their own minds, and ask how it’s going. Researchers call this metacognition – knowledge about your own knowledge, thinking about your own thinking. Top performers do this much more systematically than others do; it’s an established part of their routine.

Metacognition is important because situations change as they play out. Apart from its role in finding opportunities for practice, it plays a valuable part in helping top performers adapt to changing conditions…[A]n excellent businessperson can pause mentally and observe his or her own mental processes as if from the outside:…Am I being hijacked by my emotions? Do I need a different strategy here? What should it be?

…Excellent performers judge themselves differently from the way other people do. They’re more specific, just as they are when they set goals and strategies. Average performers are content to tell themselves that they did great or poorly or okay. The best performers judge themselves against a standard that’s relevant for what they’re trying to achieve. 

Not only do people not know what they don’t know, but their ignorance begets confidence. Metaignorance as the source of unfounded confidence is known as the Dunning-Kruger effect. Illusory superiority means that the people most in need of assistance (e.g., training, education, help) are the least likely to recognize their need.

And with confidence comes ego. Many people who have declared themselves tech savvy (or some other positive designation) are only interested in that which confirms their self image. Faced with the choice of changing their self conception and proving there is no need to do so, most get busy on the proof. For lawyers, this urge towards ego preservation combines with a psychological profile that sees any admission of fallibility as an admission of incompetence.

Thus, in administering my tech competence assessment, I often come across ‘tech-savvy lawyers’ who, like I was, are merely tech savvy for a lawyer. They do well on the assessment. But they do not do perfectly. They then go to great lengths–one penned a 1,500 word memo–to explain why the features they struggled with should not be tested. While I think there is a worthwhile discussion to be had about who should be training on which skills (a later post), it amazes how people are able to delude themselves that they already know everything worth knowing. Any assessment that fails to completely confirm that self image is flawed.

When I then explain to them where the disputed features fit into a rational legal workflow, as well as the important concepts of fluency and fluidity (another later post), they, more often than not, begrudgingly concede that I may have a point. Maybe, just maybe, they might have some worthwhile things left to learn. But…there is always a but…but, they inquire, how can I expect the majority of lawyers or staff to score perfectly on a first attempt when they themselves–the cream of the crop within the profession–did not do so? I, of course, have no such expectation.

There are lawyers and staff who have flown through my assessment modules because there are people who are both legal professionals and tech savvy. But not many. For most people, there are areas where they could use training. That’s the point. Competence-based assessments are designed to identify who needs training on what, and then to verify the training has been effective. An assessment that everyone, trained and untrained, can pass on the first attempt is pointless.

Our problem is not the dearth of individuals who operate above the profession’s tech baseline. Our problem is that the baseline is so low. My goal is to raise the baseline.

ADDENDUM: All that said, I cannot help but feel like an impostor because I am one. As I detail above, I had my bubble burst with respect to my own tech savvy. It was the epiphany I needed to start taking tech and tech training seriously. Converts make the greatest zealots. But I’ve never really recovered. Indeed, the more I learn, the more I recognize how little I know. Lawyers are no longer my reference class for properly using technology. People who are genuinely expert at using technology are my reference class (though some of them are lawyers). And they make me realize that I still have so far to go. It makes my evangelism feel hypocritical.

But this particular brand of humility is also helpful. I know enough to know how much low-hanging fruit is within our immediate grasp. But my lack of confidence in my own expertise also helps me empathize with those who are unaware of their own ignorance or who struggle with their ego. I’ve been there. Just because you have room to improve at using tech does not mean you are stupid, lazy, or bad at your job. It just means that you have room to improve at using tech, which you should do. Progress, not perfection, is the objective. And getting better is evidence of a commitment to excellence, not an indictment of past performance.

++++++++++++++++++++++++++++++++++++

Casey Flaherty is the founder of Procertas. He is a lawyer, consultant, writer, and speaker focused on achieving the right legal outcomes with the right people doing the right work the right way at the right price. Casey created the Service Delivery Review (f.k.a., the Legal Tech Audit), a strategic-sourcing tool that drives deeper supplier relationships by facilitating structured dialogue between law firms and clients. There is more than enough slack in the legal market for clients to get higher quality work at lower cost while law firms increase profits via improved realizations.
The premise of the Service Delivery Review is that with people and pricing in place, rigorous collaboration on process offers the real levers to drive continuous improvement. Proper collaboration means involving nontraditional stakeholders. A prime example is addressing the need for more training on existing technology. One obstacle is that traditional technology training methods are terrible. Competence-based assessments paired with synchronous, active learning offer a better path forward. Following these principles, Casey created the Legal Technology Assessment platform to reduce total training time, enhance training effectiveness, and deliver benchmarked results.

Connect with Casey on LinkedIn or follow him Twitter (@DCaseyF).

  

This is the fourth talk from the the ILTA Session – Legal Technology Innovation – Bolstering and Destroying the Legal Profession. This is from Stuart Barr, COO at HighQ.


A lot has happened in the AI world in the last year. Robots can now create art, learn how to play computer games, categorise buildings and even determine how creative a painting is. But I think we can all agree that the most exciting development in computing this year has to be the IBM food truck, which combines cognitive computing, big data and the cloud to invent delicious new food. The perfect combination for us techies 🙂

However, no one has taught a computer how to be a lawyer just yet. They’re working on it and as we discussed last year, I think it’s only a matter of time before we do eventually create machines that can learn how to be lawyers and they will replace many functions in the legal sector. In my mind there is no doubt about that. White collar jobs will be taken by machines in the same way that many blue collar jobs have been. It’s just a matter of when, not if. Arguably it’s already happening.
The famous economist John Maynard Keynes popularised the term “Technological unemployment” in the 1930s to give a name to the process of losing jobs to technical innovations. This is a problem that humans have been wrestling with for centuries, since we started using tools to help us become more efficient and there have been many examples of it in the past.
The Industrial Revolution is a great example of technical innovations having profound effects on jobs and on wider society. It was brought about by a combination of three primary innovations in textile manufacturing processes, steam power and iron founding. Together, innovation in these three sectors acted to transform what were traditionally specialised, cottage industries into highly industrialised, automated, mass production processes, displacing many highly skilled workers, such as artisan weavers and cotton spinners, in the process. Their skills had been commoditised by machines that were faster, more efficient and cheaper.
More modern examples of technological unemployment include self-checkout kiosks in grocery stores and biometric scanners at checkpoints in airports. So I think it’s clear that the incredible pace of technological advancement definitely has an effect on jobs and entire industries. 
But it’s not all bad news. Technology also creates new types of work and new jobs. All of the panelists at the ILTACON session “Legal Technology Innovation: Bolstering AND Destroying the Legal Profession” where I spoke about this subject were only there because our jobs and our companies have arisen out of technological advancements and we’re trying to apply them to the legal sector. Think about the “App economy” created by the boom in smartphones. In 2007 it didn’t exist and in 2015 it’s expected to be a $100 billion industry. Literally millions of jobs and massive wealth is created as a result of these innovations. Have workers had to adapt and learn new skills? Absolutely, but the overall amount of work in the world economy continues to grow, not shrink, as technology advances.
So in the short to medium term, I think technology will create at least as many jobs as it destroys. But skills will shift and people will need to become more technical in order to stay relevant. Some jobs will completely disappear but new ones will emerge to take their place.
In the legal sector this means many legal processes, such as contract reviews, to pick one example, are being automated and optimised. We will get to the point in the not very distant future where junior lawyers will not need to sit and wade through thousands of contracts in a due diligence process; a machine will do it for them. But new opportunities will arise for hybrid “legal engineers” who will need to understand law AND technology in order to best utilise those machines and leverage their capabilities to gain a competitive advantage for their firms.
So I can see that, in the short term, basic legal processes will be automated and then, gradually, as machines become more sophisticated, they will be capable of performing more complex legal functions and there will be a shift from being “lawyers” in the traditional sense, to being legal developers or technicians. At this point though, we will still need the most bespoke and sophisticated legal work to be carried out by humans. Indeed, the very concept of machines taking over more and more human work may actually increase the demand for lawyers to sort out the societal complexities and disputes that will inevitably arise.
But what about the long term, 30-50 years from now?
Last year I talked about the “technological singularity” which is the theory that the exponential growth we are seeing with technology will ultimately lead to artificial intelligence exceeding human intelligence.

There is a general consensus among futurists that this will happen sometime in the 21st century, probably in the next 50 years or so. At this point, the question is no longer about whether lawyers will still have a job, it’s about what will happen to society. Will it be a Star Trek-like utopia where man will leverage machines to better themselves and explore the Galaxy? Or will it be a Terminator-like dystopian nightmare? Who knows. But one thing is is for sure, technology is changing everything and it will happen in our lifetime.

Pat Lamb, who I mentioned in my last post, recently wrote a piece that while excellent was not exactly groundbreaking. Or so I thought. Pat’s premise was that everyone makes mistakes. Everyone includes lawyers. Mistakes happen. Mistakes are bad. We should therefore learn from mistakes to avoid repeating mistakes. Towards this end, Pat explained the usefulness of After Action Assessments and root-cause analysis. Great stuff. But, for me, it was akin to a health expert recommending that couch potatoes exercise, consume less junk food, and eat vegetables. Perfectly sound, uncontroversial, well-worn advice that is too often ignored and should therefore be repeated as often as possible.

As usual, I was so very wrong. The ABA Journal comments section exploded with adverse wisdom like:

CHRISTIE WAGNER said:
Yes, dear Patrick J. Lamb. Let us embrace our mistakes, such as missing statutes of limitations and whatnot. Don’t our mistakes just empower us? Gag. How about doing all you can NOT to make a mistake and then doing all you can to rectify it and if you can’t do all you can to make amends? Hmm?

associate said:
Mistakes are simply unacceptable.  That’s the reality of our profession, especially since the law schools and ABA have flooded it with excess attorneys.

These perspectives struck me as so silly that I added satirical support to their keen ‘insight’:

Casey Flaherty said:
Mr. Lamb:

Christie and associate make some compeling points, which is unsurprising given that they are menbers of our hollowed profession. Look, I know that you reference Einstein, Drucker, the Navy Seals, Kimye, Drake, and Meek Mill. What do they all have in common other the fact that they make mistakes? None of them are lawyers.

As Justice Mustang wrote for the Untied States Supreme Court in in the semenal 2010 case of Clouseau v. Malapprop, “whatever a lawyer is doing is, ipso facto, correct because a lawyer doing it. Lawyers are, by definition, incrapable of error.” As you know, the Court held that having a J.D. (which stands for Jactanter Dealbator, or “the one who is (always) right”) was an absolute defense to a claim of malpractice. In her concurrent, Justice Abercrombie even traced the origins of the lawyer infallibility doctrine from The Federalist Papers to the Papal bull Unam Sanctam. The impact of Clouseau on the insurance industry was the primary cause of the Great Recession. And 43 of the 46 state bar associations subsequently suspended their grievance process because no one could maintain a good faith claim that a lawyer had been wrong in thought or deed.

Not only do I think that we should continue to deny that lawyers make miss takes (thereby guaranteeing that mistakes will never happen), I believe that we should also deny that any lawyer has ever written an article suggesting that might lawyer mistake make (guaranteeing that your article was never written). How’s that for an After Action Plan?

My mockery did absolutely nothing to stem the tide of abuse directed toward Pat. Pat, of course, does not need my help. Along with Jeff Carr, he is building an entire business around the idea that identifying and addressing the root causes of mistakes is key to not repeating them. But the incident left me sad for my profession.

The pursuit of perfection begins with admission of imperfection. How do we get better at what we do when reflecting on mistakes is a thought crime? when it is a sin to admit that what we do can be improved? when formalizing a mistake-reduction process is the height of unprofessionalism?

Michael Jordan believes Michael Jordan is the best basketball player of all time, just ask Michael Jordan (see his Hall of Fame induction speech). Yet, Michael Jordan is, arguably, right. Michael Jordan may have been the best basketball player in history, in part, because he understood that being the best required being fanatical about eradicating his weaknesses (which, of course, he had to admit in order to address). Jordan even made a commercial entitled Failure.

You can be the best at what you do without being perfect. You can stand in high regard in your profession (and your own mind) without pretending you are infallible. Indeed, framed properly, your ruthless pursuit of better can be a signal of your stature, not a threat to it.

As Seth Godin says, “Yo Yo Ma isn’t perfect… he’s just better than everyone else.” Like Jordan, Ma is also, arguably, the best at what he does. He has pretty much always been great. He was a child prodigy. He chose the cello at the age of four. By seven, he was playing for American presidents. Yet, Ma still practices 2,000 hours per year–a solid year of lawyer performance. Yo-Yo Ma is Yo-Yo Ma precisely because he is the kind of person that will practice 2,000 hours per year even after he is established as the best in the world.

In the book, Talent is Overrated: What Really Separates World-Class Performance from Everybody Else, Geoff Colvin explains that the best of the best take a different approach to self-analysis and error (h/t Farnam Street):

Excellent performers judge themselves differently from the way other people do. They’re more specific, just as they are when they set goals and strategies. Average performers are content to tell themselves that they did great or poorly or okay. The best performers judge themselves against a standard that’s relevant for what they’re trying to achieve. Sometimes they compare their performance with their own personal best; sometimes they compare with the performance of competitors they’re facing or expect to face; sometimes they compare with the best known performance by anyone in the field….

….If you were pushing yourself appropriately and have evaluated yourself rigorously, then you will have identified errors that you made. A critical part of self-evaluation is deciding what caused those errors. Average performers believe their errors were caused by factors outside their control: My opponent got lucky; the task was too hard; I just don’t have the natural ability for this. Top performers, by contrast, believe they are responsible for their errors. 

The kind of people who proclaim that mistakes are unacceptable even when they recognize mistakes are unavoidable do not really believe they are perfect. Rather, they believe that people like them are not supposed to admit imperfection. This remains one of the biggest impediments to change in our industry. As I try to convince law firms and law departments to engage in structured dialogue, I constantly run into people on both sides of the relationship who are threatened by the idea of an open discussion about doing better. If I concede that we might do better then I am confessing that I’ve been wrong. We have to get beyond the idea that improvement is an indictment of the past. Our job is not to be perfect. Our job is to do the best we can until we can do better, and then do better.

But better means different. And different requires experimentation. Unfortunately, this runs counter to the lawyer psychological profile, as explained in this stellar post on Adam Smith, Esq.

It’s a truth universally acknowledged that Law Land resists change like the plague. And, yes, this is pretty widely attributed to the phenom known as the “lawyer personality,” characterized by an almost pathological aversion to risk. But let’s unpack this a bit to better understand why lawyers, in particular are so risk averse.

Lawyer-psychologist Dr. Larry Richard, the leading expert on the psychology of lawyer behavior has quantitatively established that a preponderance of lawyers share (among others) two personality traits that in combination scotch the very notion of experimentation. The first is “Resilience.”  Somewhat surprisingly, lawyers score really low on “Resilience,” essentially the ability to recover quickly after a setback. On this trait, Lawyers score only 30%, which would be enough on its own to seriously dampen any appetite for experimentation.

But wait – there’s more. What won’t surprise anyone is that lawyers score really high on “Skepticism.”  Dr. Richard’s work reveals that lawyers score 93% on this attribute (only 5% of the population is as skeptical). Lawyers can (and do!) poke holes in anything.  So, any proposed change will be scrutinized and picked over to such a degree as to discourage all but the most stout-hearted.

These two characteristics, lack of resilience coupled with a high degree of skepticism are a double whammy to the notion of experimentation. Failure is perceived anything but “noble” in Law Land.

In short, we have the wrong mindset. We are inclined to hide deficiencies instead of overcoming them. That’s the easy way out. The problem with the easy way is that eventually it makes everything so damn hard.

  

++++++++++++++++++++++++++++++++++++
Casey Flaherty is the founder of Procertas. He is a lawyer, consultant, writer, and speaker focused on achieving the right legal outcomes with the right people doing the right work the right way at the right price. Casey created the Service Delivery Review (f.k.a., the Legal Tech Audit), a strategic-sourcing tool that drives deeper supplier relationships by facilitating structured dialogue between law firms and clients. There is more than enough slack in the legal market for clients to get higher quality work at lower cost while law firms increase profits via improved realizations.
The premise of the Service Delivery Review is that with people and pricing in place, rigorous collaboration on process offers the real levers to drive continuous improvement. Proper collaboration means involving nontraditional stakeholders. A prime example is addressing the need for more training on existing technology. One obstacle is that traditional technology training methods are terrible. Competence-based assessments paired with synchronous, active learning offer a better path forward. Following these principles, Casey created the Legal Technology Assessment platform to reduce total training time, enhance training effectiveness, and deliver benchmarked results.

Connect with Casey on LinkedIn or follow him Twitter (@DCaseyF).  
 

This is the third talk from the the ILTA Session – Legal Technology Innovation – Bolstering and Destroying the Legal Profession. This is from Michael Mills, Co-Founder and Chief Strategy Officer at Neota Logic.

Clearing Department, woman sorting cheques and using adding machine - 1960s

I decided to come at the technology question from the human side, to speculate about what humans are still good for in a technology-saturated world of legal services.

I concede. I am obsolete. The robots are winning.

Fastcase is a better legal researcher than I am—despite a University of Chicago law degree, a federal court clerkship, and a hand in hundreds of briefs and memos.

Recommind Axcelerate is a better document reviewer than I am—despite tutelage by demi-gods of the American bar, and years of experience, some of it in unheated warehouses and abandoned salt mines.

And of course Google is a better driver than I am.

Nonetheless, for a while, I have work to do.

Kira is not a better contract analyzer than I am—my pattern-recognizing brain is more precise, more adaptable, and faster than Kira’s algorithms. And the algorithms need training, so I can have a job as an algo trainer—like a dog trainer, but without a whistle or a biscuit.

But … Kira’s algorithms are getting better and its computers are getting faster. My brain is not, alas.

So, one day . . . poof! ZMP for me—that’s Zero Marginal Product, the economists’ term for adding no value at all.

As Harvard professor Bill Bossert said many years ago—“If you’re afraid that you might be replaced by a computer, then you probably can be—and should be.”

Or, as I say to law firm partners who worry that Neota Logic expert systems will cannibalize their billable hour work—“If your business model is to do work that my software can do … you’d better get a new one.”

So what’s left for me? For us? We went to law school, we’re nice people, we’re pretty smart.

Fortunately, there are some things for us humans to do:

Geoff Colvin of Fortune Magazine just published a book with a great title “Humans are Underrated.” (In some contexts, one might argue that he has that backward.)
He writes that the right question is this:

“What are the activities that we humans, driven by our deepest nature or by the realities of daily life, will simply insist be performed by other humans, regardless of what computers can do?”

He then says that the foundation of all the other abilities that make people valuable as technology advances is … empathy.

Yes, empathy.

Discerning what some other person is thinking and feeling, and responding in some appropriate way.

We have evolved to do that—collaboration was essential for survival, in hunter-gatherer and then in agricultural economies.

As Colvin puts the point:

“We want to work with other people in solving problems, tell stories to people and hear stories from them, create new ideas with people. We want to follow human leaders. We want to negotiate important agreements with people, hearing every lilt or lament in their voices, noting when they cross their arms, looking into their eyes.”

We, both individually and as members of groups and organizations, keep changing goals, purposes, understandings, directions, conceptions of the problem, interests—software simply can’t keep up.

People can, groups can.

One might say, then, that what humans will continue to do, so as not to drown in the rolling wave of technology, is what we do best in groups:

  1. Idea-generation, problem-solving, strategy
  2. Persuasion, argument, storytelling
  3. Collaboration

So … if groups are essential to our economic survival in a world eaten by software, to use Marc Andreessen’s phrase, how do we know an effective group when we stumble into one?
Cambridge University psychologist Simon Baron-Cohen developed in 1997 a simple test, the RME—Reading the Mind in the Eyes. Participants are asked to choose a word that best describes people’s thoughts or feelings based only on photos of their eyes.

Group members’ average score on RME has proven to be an excellent predictor of group effectiveness.

More recent research supports a much simpler test—no advance testing required. Just count the number of women in the group. More women, more effective. Period.
Another reason for diversity in STEM disciplines!

MIT professor Alex Pentland invented the sociometric badge, a little tag that hangs around your neck and tracks how you work with others—the amount of face-to-face interaction, conversational time, prosodic style, physical proximity to other people, and physical activity levels.

After sociometrically measuring many groups with his little badges, Professor Pentland found that groups do their best work when the participants:

  1. Generate many ideas in short contributions to conversations. No one natters on.
  2. Constantly alternate between talking and listening, encouraging, and reacting.
  3. Take turns.

It does sound a bit like the prescription for a good kindergarten, but it works. These 3 factors are as important to group effectiveness as all others together—individual intelligence, technical skill, personalities, and so on.

Interestingly, and here we technophiles should take note, this research suggests that online, technology-mediated collaboration is far less effective than we think.

Apple agrees—their new headquarters is gigantic, in order to bring people together, physically, to do the empathy thing, to do the human thing.

Google agrees—they engineered the cafeteria (it’s a metrics-driven company) – optimum wait time in line 3–4 minutes, table spacing to encourage bumping, long tables to encourage sitting with people you don’t know.

So, even at Google, there is room for us humans.

But … and here I think we come to the rough reality of the legal services industry (and others too, which raise profound long-term questions about the civic compact)—technology is pushing the performance bar for humans ever higher, chopping off the bottom tail of the bell curve, shrinking the space in which “just OK” is OK, in which being “pretty good” is good enough. It isn’t any more.