Legal Technology Track: Data Analytics in Law



(upbeat contemporary music) (applauding)
– Thank you, Nicole. And thanks all of you
for being here today. As Nicole said, I'm the CEO of Fastcase. I also teach at Georgetown
University Law Center and at Cornell Tech, a class
called The Law of Robots. I have to say it like that. I've got a 10-year-old
kid who's really into the like, reverb for Law of Robots. I'm also the editor of a
book called Data-Driven Law. Nika Kabiri actually contributed
a chapter in that book talking about a lot of the topics that we're gonna discuss today. So data analytics. What are they, how do they help your firm, how do they help your clients? That's our topic for today, so naturally, I'm gonna start with maps. Permit me like, a brief tangent. Let's talk about the history of maps and cartography for a second. I want you to think about
a time before maps, right, and think about a time before maps from the perspective of a traveler. In antiquity, before there were good maps, travel was dangerous, all right? People didn't travel much. They were born and died in the
places where they were born. Why? Well, travel was risky. You didn't know if you
picked the right way. You didn't know how to get
from one place to another. You didn't know if
you'd get there in time, if you would have enough
money to get there, if you would have enough
food to get there. So travel was risky in
the age before maps. And during this time, travelers would hire navigators, right, and a navigator would take you from one place to another place. And navigators were good because
if you'd never been before, it was comforting to go with someone who had made the journey before. The problem is navigators typically only had their own anecdotal
experience to guide them. They could take you on routes that they themselves had taken. Maybe, maybe they could take you on routes they had heard about themselves, but they were limited for the most part to their own anecdotal experiences. So in an era before maps were common, travel was risky, and expensive, and rare. People just didn't travel
much because it was dangerous. All the risk was on the traveler. So we know over time, these experiences, these individual, anecdotal
trips began to be recorded, first in oral tradition and then written, and then people could
trade them to each other, these written accounts
of what would happen, and then they started to be drawn, and people started to aggregate
these drawn experiences into the early maps. And what happened? So in the early days of maps,
travel became less risky, more people did it, it became easier. So there's like a thousand
years of history of maps, right, so there's innovators like Ptolemy, who first drew the latitude
and longitude lines on maps so you could scale maps. Mercator who allowed
us to see on flat maps what journeys looked like
across a world that was round. They weren't always good maps. I mean, a lot of the maps
would sort of represent the unknown with these like, sea monsters, "Here there be monsters." But they were certainly
better than nothing. I mean, they were better
from the perspective of the traveler than
no information at all. And over time, the maps
got better and better. Cartography techniques got
better, and sooner or later, in the early 1900s, we were actually able to send photography drones in the sky to take pictures of the terrain
so you could actually see what things looked like from the sky. I see a couple of maybe skeptical faces about the drones in the
sky in the early 1900s, but it's true. We would send drones in the
air with cameras on them to take pictures of the Earth. Here's a picture of one. During World War I, maps were a source of significant strategic advantage, and so military leaders
would send pigeons in the air with cameras to take
pictures of the terrain so they could understand from the sky what was happening better. And they would use these maps for their strategic advantage in battle. So the maps continued to
improve, until in the mid-1900s, we would have maps that
were way more detailed. And every time maps got
better, trips became easier. They were done more. From the perspective of the travelers, maps democratized travel, and they shifted the
risk from the traveler, maybe to map-makers, I don't know. Okay so, I talked about
the drones of the 1900s as one source of innovation. There was another in the '70s called Geographic Information Systems. These are the basis for Google Maps and all the maps that
power your cell phone, Bing Maps, MapQuest, things like that. The only thing I'll say about GIS Systems that I think is interesting
is that for the first time, you could separate out the
database of data about maps, the analysis of that data, and then the display of that data. With paper maps it was all in one place. You would sort of hand transcribe it, and they were all in sort of one layer. So after GIS Systems, you get
like, the first MapQuest maps. And then with mobile phones,
the ability actually to see where you are on that map, right? And then the last innovation I'd really like to talk about, is Waze. The ability to crowdsource the data layer, where people who are using the maps can actually transmit data
that is reflected on the maps. We're all familiar with this now, right, so you can see where
there is a traffic jam because the phones of people
who are using Google Maps or Waze will actually reflect
back to the Google databases and say, "These phones,
these sensors have stopped." And so you can see if
there's a traffic jam and take a different route, right? Okay, and so it's really great. So now you can sort of see ahead of time where there are construction
events or road blocks or traffic cameras, all
because of the information provided by the people
who are using these maps. From the perspective of the traveler, the world gets much, much better, and travel becomes much
more safe and less risky as you move from an era
of anecdotal information about the trip to good maps, to great maps, rich with information. Okay, so that's my
nerdy tangent about maps and the history of cartography. "What on Earth," you might ask, "does this have to do
with legal informatics, "with data analytics, with my
damn law practice? (laughing) "Please Ed, tell me you're
gonna tie this back." And I will, so. Clients bring to law firms, they bring to us as lawyers
their most pressing problems. If you are a corporate lawyer, companies will bring
Bet-the-Company litigation to you. If you practice family law, people will come to you with
the most important questions their family will ever face. And so we get these
questions all the time. "How much is my case worth?" Right, if you are a contract
lawyer, "What's market?" If you are drafting contracts,
you're trying to figure out, do you use this clause or not, you want to know, is this standard? If you're a criminal lawyer, is this plea deal a good plea deal or not? Every, single one of your
clients has a common question which is, "How much
will this matter cost?" A mystery to me is around settlements. If you're trying to settle a case, how much do you offer in settlement? Or, if you receive a settlement offer, should you accept that settlement offer? If you're litigating,
your clients want to know, "What are my chances of
winning in this litigation?" Increasingly, in more
sophisticated matters, clients in one way or another are asking, "Help us to price simple risk." In some cases, "Help us
to price exotic risk." And we as lawyers, when we get these Bet-the-Company questions,
when we get these questions that will decide the fate of a family, we answer them with our experience. When I talk to lawyers
about this, they say, "Look, I've handled
dozens of these matters, "and so I know from experience
how much something is worth. "I know from my experience
how to settle the case." Social scientists and
statisticians would say though, "If you're making that decision
based on 20 data points, "30 data points, you barely
know anything at all." And that's for the experienced people. Many times, when you're just starting out, you have to make that call with like, two data points, or zero data points. So statisticians and social scientists would call this experience "hunches," a decision that you make with
very small amounts of data. And so clients bring
these questions to us, and we answer them with our hunches, and as a result, legal services are risky and expensive, and rare. From the perspective of the person who is making this journey through the legal services system, they're really limited by the individual, anecdotal experience
of their navigator, us. When we try to help our clients through this unfamiliar terrain, we are limited by our own experiences. And as a result, four out of five people who have a legal problem
don't address that problem with the help of a navigator,
with the help of a lawyer. To put it differently, if
you as a client have a choice between getting legal help from someone who has anecdotal experience,
and someone who has data, you will always choose
someone who has data. Our clients have all the risk
in these transactions, right? "How much will this matter cost?" Frequently, the answer is, "Don't know. "It's gonna cost you $220
times the number of hours "it takes for me to handle this matter." It's risky. I think a lot of times,
we imagine as lawyers that the reason that
four out of five people don't pursue legal problems
with the help of a lawyer has something to do with price. And Jack alluded to this before. There is, beyond price, a
product/market fit problem. And part of that's about
structure, part of it's about risk. So I think of it this way,
if you went to a restaurant and saw a lobster on the menu,
and it said, "Market price," and you asked your server, "What's the market price for lobster?" And your server said, "It depends. "It depends on the
temperature of the water, "the day the lobster was caught. "It depends on the price
of gas for the boat "getting from the lobster
traps back to port. "It depends on the tides and the waves. "It depends on the price
of gas to get the truck "from the port to this restaurant. "It depends on the price
of natural gas to cook it, "and how long it takes at this altitude. "But don't worry, "we'll give you the price
when you get the check." So nobody would ever
order a lobster, right? In fact, people would order
something else on the menu that has an average price higher than the market price of
lobster to avoid the risk that the lobster's gonna be
$72 or $109 or $4,000, right? And if you had a restaurant
where all the prices were market prices, and
all the prices depend, and you only ever find out
the price of these entrees when you get the check, no one would ever go to that restaurant. If they have any alternative at all, they will eat somewhere else, or they will cook for themselves. And this is where we are. We're in a place where
we're trying to help people navigate unfamiliar terrain, but we don't have any
data to help them do that. We don't have maps of this terrain. Sometimes we barely
understand it ourselves. The way we understand it is
conjectural and anecdotal. And from the perspective of clients, they face that lobster-like
risk every time they come to us. All of the risk in that
transaction is on the client, and as a result, legal services
are risky and expensive, and people just don't do 'em
if they have any alternative. They'd rather cook their own food. All right, so how do we get out of this? Well, I don't have a complete solution, but one part of it I think
is to map out the terrain. And this is data analytics for law firms. We can pull out of our experiences wisdom. We can share it. We can see across a number
of different matters and create maps of legal services. We can create new legal
products out of them. All right, so here's
like a judicial opinion. A judicial opinion rendered in the best, most wonderful legal research service in the history of the universe. By the way, a legal research product that most of you get for free through your state bar association and integrates 100% with
your Clio subscription, so that's also nice, but I digress. A judicial opinion, very
familiar to us, right? When we see this, we
typically see a document, but the document is full of data, right? I mean, you can see things
like who the parties are, or what the appeal is,
where it's coming from, who the judges are on the
case, what date it was decided. You can actually pull out of this who wins or who loses that case. This is a docket sheet. A docket sheet rendered in
best docket tracking software ever created in the history
of the world, Docket Alarm, which is now a part of Fastcase. I think we tend to see
this as a list of documents in the litigation in chronological order. It's way, way more than that. This is a map. This is the story of a journey
through a legal matter. So it's full of all this information like the date things were filed, how long that is from the
original filing in the suit. You can see what happens at
each stage of this litigation, and how long it takes. Every one of these docket sheets is a map to a legal journey. If you open the documents
in the docket sheet, you might see a complaint. Let's say this one was picked at random. Full of information, right? You can find the court,
and at the end of it, you can find the parties. You can extract from it what happens. Was the motion granted or not,
on what date, by what court? What were the law firms? Every one of these is
a point in the journey, and when you assemble
those points together, you're able to create maps, right? When you roll up the
information that is collected in every one of these individual
entries in a docket sheet, in every one of these judicial opinions, in every one of these
documents in litigation, then, for the first
time, you're able to say, "Here are the litigants
in this jurisdiction. "Here are people who have
had a similar experience "to my clients, and here's what happens." And not in an anecdata way, not in a "my limited
experience of 12 matters" way. Here's what happens in 120,000 matters. Here's what happens in cases like ours before this court and before this judge. And by the way, I don't need
to pave this course myself. I can look at the documents in the case, and try and figure out
what's happened before so I don't have to start from scratch. There's a lot of people
who use information like this to prospect as well. You can get alerts when certain
kinds of cases are filed, and then you can use that as
a way to build new business. And when you go to pitch
that business, you can say, "Not only am I a very experienced lawyer, "but I can show you analytically "what happens in cases like this one. "So we're not starting from scratch. "You're not limited to my
individual, anecdotal experience." "How much is this case worth?"
is not like, a single answer. It's not like you say, "The
answer is $1.6 million." The answer to "How much is my case worth?" is a distribution. This is the distribution of outcomes in cases like mine, right? The curve looks like this. The mean is 1.1, the median is 1.6. "Who is this judge?" The answer isn't, "I know
this judge because I know "that she was appointed by
a Republican president," or, "I helped this judge to
get elected the first time "by playing golf with her
and raising money with her." "Who is this judge?" is "How does this judge
rule in cases like mine?" And you can't know that
from individual experience. You're not a litigant in all those suits. You need to roll up all that experience, and you need to do it from
the perspective of a map. And when you roll up this data from a lot of different data
points, for the first time, you're able to express the answers to these questions not as hunches. You're able to express the answers to these questions as data. And this is what clients are asking us. They're asking us data questions, but we're giving them conjectural answers. What they really want is data answers. So (laughs), "Should we settle this case? "Should we accept the settlement offer?" You might want some
information about that. For example, what happens
if we go to trial? Are we gonna win or lose? What's your exposure if you go to trial? You might also want to know, what are settlements like
in cases like this one? So if the settlement offer is like, in the 90th percentile of
settlements of similar cases, and your chances for
success at trial are 16%, that's important information to know. That sounds like a pretty
good settlement offer. If the settlement offer is in the eight percentile of settlement offers, and your chances for
success at trial are 89%, even if the settlement number is the same, those two cases are
entirely different postures. So instead of using our best guess, instead of averaging out how
much is being asked for in zero and trying to figure out whether
to settle the case or not, we can use data to see what
happens in cases like this one. So these are all Docket Alarm slides. Michael Sander, the
founder of Docket Alarm is sitting right here. Look at how these cases end, right. If you look at all of them,
see what the outcomes are, right, and look at cases
that are like yours to try and figure out
what's likely to happen in this case, in this court, right? So when we answer these questions, we shouldn't answer them
with "it depends" answers. We shouldn't answer them
with anecdata or hunches. If they're data questions, we should answer them with data answers. This is not how we do
it today, by the way. Nobody, very few people are able to answer questions this way. But I can guarantee you, this is how the questions will
be answered in the future. Right, today this is the ceiling. In very short order,
this will be the floor. Your clients will have
this kind of information at their fingertips, and if you don't, you will never win their business. Right, you have to at least have these kinds of reports
for matters like yours. You have to be able to use the experiences across judicial opinions
to extract information that nobody else has. That's the new floor. At Fastcase, we built this
tool, the Interactive Timeline, that maps out search results and shows the citation
relationships among them. It is kind of like a data-driven way of showing what's persuasive to courts. Courts know what's persuasive to them. You can't find that out by reading 600 documents in a search result. You need to map out the
citation relationships and see what's authoritative
in a data-driven way. We also created this adorable Bad Law Bot, the world's first algorithmic citator that uses algorithms to try and predict whether cases are still
good law or overturned. I'm showing you Fastcase and Docket Alarm examples, by the way. I'm not pretending like
Fastcase and Docket Alarm are the only companies that do this. Lex Machina from Lexis Nexus
really pioneered this space. They do a really good job
with litigation analytics. Lexis Nexus is making a big push into data-driven law now
itself, which I think is great. Law schools, by the way,
have done this for years. In law school, they call
this empirical legal studies, the study of what happened, right? Studying the past as a guide for what's likely to happen in the future. These are maps. These are ways of taking risk out of the legal services
market for clients. If we want to serve more than
20% of the needs of clients, we need to change the kind of products that we're offering them. And data-driven law, data analytics, are a great way of doing that,
taking a lot of the risk out. "How much is this matter going to cost?" Well, I can tell you that
the distribution shows us the prices range from $21,000 to $46,000. The average is 33, and
there's a cluster over here in the 40s that have this one
particular characteristic, yours doesn't, so I think this matter's gonna cost you around $33,000. You don't even necessarily
have to do it on a fixed price. If you tell a client during the pitch, "I know from experience,
from my billing system, "this is what it's likely to cost." That is way better than what
everyone else is going to do. And if you do it for a fixed price, maybe even a fixed price
that's 120% of the average, people will be way more excited about that than a transaction where
they bear all the risk. So look, I was a lawyer for
years, but today I'm a client. At Fastcase, we have like
four or five law firms who do work for us, and
even the great ones, when I say to them, "What's
going to happen here? "How much is this gonna cost?" their answer is always conjectural. Their answer is always, "Don't know. "It's gonna cost you $775
times the number of hours "it takes me to finish this work. "And if that takes me one
hour, it's gonna cost you 775, "but if it takes me 100 hours,
it's gonna cost you $7,700. "Where it's gonna fall between there, "nobody knows, big mystery, okay?" That drives me bonkers! And if any law firm came to me and said, "I can't tell you with any guarantee "exactly what it's gonna cost, "but I can tell you in
matters just like this one, "we have a range from 22,000 to 49,000, "and the median is 32,
and the mean is 38," I'm gonna go with that law firm before the lobster law firm any day. We gotta stop practicing
lobster law, all right? So Jack in his keynote talked
about product/market fit, and it's not just about
the products that we offer. In large part, that can be
about the prices we charge, but more than that, it's about risk. It's about taking a
journey that you don't know that you're gonna live through, taking a journey that you don't know if you're gonna reach the other end of it, taking a journey where you are worried that you're gonna run out
of food in the middle of it. I was talking to a lawyer
today who said that he was in a six-year
litigation with somebody who paid his fee the whole
way, and at one point, had to declare bankruptcy,
like right before trial. The litigation bankrupted him, right? So we're practicing lobster law. We're practicing hunch law. We need to practice data-driven law, and we need to start answering data questions with data answers. The product that is being
asked for by clients from us is data law, but we keep
feeding them lobster, and we gotta stop doing that. All right, so if we take
data analytics as maps, maps of legal matters, right, a way for clients to better
understand the journey through a legal service, we should think about maybe risks, opportunities, and frontiers. What can we learn from maps as we begin this new
era of data analytics? So one risk is that maps lie. It's often said by cartographers, "No map completely tells the truth." There was a great West
Wing where they show the map of the world upside down, and like, Iceland is like,
way bigger than North America because the Mercator projection
threw everything off. So maps are representations of the truth, they are not themselves the truth, and they are distorted
in all kinds of ways. A great example of this that I love, in the early days of
exploration of North America, a Spanish explorer went to the West Coast and explored the Baja Peninsula,
and went to the coast, and started up the Baja Peninsula, and got to a certain point and said, "All right, I've explored California, "and I can see from my journey "that California is an island." And so he returned to Europe
with this map of California, the island, and this was the
way California was represented in maps for almost a
hundred years in Europe. Because this individual explorer said so. This is not something limited
to like, antiquity, right? We have these kinds of data
problems in maps today. My friend Andrew Arruda recently was at SFO waiting for his Uber, which appeared to be in the
middle of San Francisco Bay. Every once in a while,
there are data errors. There's another risk too. The risk is something like our smart phones will make us dumb. I have a 10-year-old son, and I worry that he may
never be lost in his life. He will always and forever
know exactly where he is, and he will never develop
the skill of getting un-lost, of working his way out of a jam. We are increasingly dependent
on these maps in our lives. You can tell this as people
are walking through traffic, like, arched down over their smart phone. Not a very intelligent posture. So recently, I apologize to the Clions in the room in Canada. A woman was driving her new
Toyota Yaris following a map, and it told her to go straight, and so she followed the map
directly into the Georgian Bay. She didn't realize that she was underwater 'cause she was looking down
at the phone. (laughing) So divers were able to rescue
her, her Yaris, not so much. The map said, "Go straight." She went straight, but she missed the experience
in the real world as well. By the way, the other thing
about crowdsource maps, is that following the
maps changes the maps. We experience this all the time, right? You're driving, and Waze, or
Google Maps or Apple Maps says, "There's two ways to go. "One way has traffic
congestion, go the other way." And so you head the other way, and there's traffic congestion. Why? Because 10,000 people had the exact, same instruction on their maps, and to avoid the congestion, they all went to the friction freeway. I'll take some questions at the end. Yeah, no, I know, yeah. – [Man] Did she sue Google? – She did not sue Google. I would love to see that lawsuit though. And I think we have this kind of like, over-reliance on these maps too, right? We shouldn't always trust them. A lot of them are algorithmic. A lot of them are machine-generated. There's such enthusiasm right now for artificial intelligence. We sort of want to believe
that artificial intelligence we can sprinkle like pixie dust
over almost anything we do, and it will magically make
everything much better. It won't. People who are trying to sell
you a robot lawyer right now, the only thing I'll tell you is, "Run." There's no such thing. And artificial intelligence
is itself in its infancy. This in itself is a risk. So an AI researcher recently said, "Let's see how good AI is
at some soft skills stuff." When you buy paint today, you can't buy the paint color orange. Right, it's like, Toast
of New York, right? (laughing) You don't buy yellow, you buy
like, Buttercream Biscuit. And so this AI researcher said, "I wonder how good artificial intelligence "would be at naming paint colors." (audience laughing) The answer is, "terrible." So I grew up pretty close to
here actually, in Baton Rouge. I think I went to high school
with a guy named Stanky Bean. (audience laughing) In defense of AI, I think
Turdly is exactly right. (audience laughing) So I don't want to paint like, data analytics as this panacea. They're not a cure-all. There are certainly risks
with data-driven law. Carolyn Elefant, who saw this
presentation at ILTA said, "What about loser law? "What about cases where
it's important to lose? "You need to bring the case, "even though you know
you're going to lose. "Will analytics stop important
cases that are losers?" And I think my answer
to that would be really, "That's up to the client." Right, that's not up to the lawyer. If the client wants to bring a loser case, you can certainly hand
them the analytics and say, "You have a 9% chance of winning, "so we know ahead of time this is a loser, "and it's damn important. "We're gonna bring it anyway. "And you should know at the
start, we're going to lose." Right, put that in the client's hands, and let them make informed decisions based on data and not on hunches. So I think there are
opportunities here as well. One opportunity that's obvious is that maps create more travelers, and if you are an expert guide,
that means more business. I think this is maybe obvious. If there are more people
who are engaging a lawyer for legal help, that means
more business for all of us. If we take some of the risk
out of these transactions, and put more things on the menu that they don't have to
worry so much about ordering, people will eat more, right? So there is a role here for expert guides. I mentioned before that analytics will be like kind of a new floor. Everybody will have the analytics. Michael and I are gonna
make sure of that, right? Clients will have it,
law firms will have it. Everybody will sort of know ahead of time what the lay of the land is. It doesn't guarantee outcomes. When you have a 30% chance of winning, or three times out of a
hundred, this party wins, it doesn't mean you're gonna lose, right? I think sometimes we look at
like, the Weather Channel App, and it says, "a 70% chance of rain," and we say with moral certainty,
"It is going to rain." That's not how probabilistic
thinking works. As we saw in the 2016
presidential election, when you have a 30% chance
of winning probabilistically, three times out of 10,
that party wins, right? So it's a way of understanding
litigation differently. A 70% chance of winning doesn't
guarantee you're gonna win, and a 30% chance of winning doesn't guarantee you're going to lose. We might have to think
about it differently. When everyone has these
kinds of analytics though, it creates this interesting opportunity to move from read-only analytics
to read-write analytics, which means that we can
create our own data products. We can use information from
our proprietary law firm data to create new kinds of
data products for clients. We can show them things that
they had not seen before, and in this way, make legal
services less expensive, and less risky, and more common. I say this all the time. The Legal Executive Report
by Thomson Reuters estimates the size of the legal services market in the United States at
$437 billion, with a B. That's 20%. Now I will grant this is
probably the most lucrative 20% of legal market problems,
faced by big companies. How much is the other 80%? I mean, maybe not four times as much, but I think there's
every reason to believe that the 80% that's not being served as is big as the 20% that is, which means that the latent
market for legal services, the unserved market, could be as big as $500 billion, with a B. The total size of the market could be one trillion, with a T, dollars. This is exactly what Jack
Newton was talking about in his keynote, where
the market for Netflix is way bigger than the
market for Blockbuster. If we create new legal services, we can vastly expand the market for law in the United States and
who gets access to it. So these are the frontiers, right? We can create these new data products for the benefit of clients. I like to think about
simulations a little bit, where you might be able to wargame out litigations a little
better with the benefit of many similar cases like it. You could actually game them out and see how often you win or lose with that particular
fact set or situations. But I think by far the
best opportunity here is access to justice. If we can create new data services, we can expand the pool of
people who have access to it, we can take a lot of the risk out. Many more people will be served by us. We will all have more business. More justice is served across the board. Before there were maps, nobody traveled. They were born and died in the same place. Travel was expensive and
risky, and therefore rare. That's where we live today. Our legal services
market doesn't have maps. Our clients are limited to the anecdotal, individual experiences of us as guides. It doesn't have to be that way. It won't be that way in the future. In the future, we'll have maps. It changes the risk
profile of legal services, and offers an opportunity to expand legal services
to many more people. I feel like this is almost Pollyanna-ish when I tell you we can
have it all, but we can. If we create new legal services,
more people are served, more business for the legal profession, more justice overall. Thank you very much. (audience applauding) I would love to take some
questions from you all. And we have a microphone
too, Nicole's passing around. – I'm really glad to see
that you ended with that access to justice 'cause
it's exactly what I think. What I've been trying to
do in my firm is to try and track out what settlements,
what cases are worth. And I mean, in the litigation setting, this knowledge is absolutely valuable. It allows me to predict what can happen. It allows me to be able
to predict for my clients what their cases are worth. This is all wonderful,
but there's a huge gap, there's data assymetry, and
institutional defendants in litigation settings have
massive amounts of data. The insurance industry
collects all the data about all the settlements,
and the reality is that having that data is
incredibly powerful for them. It's like I tell my clients, "When we go into a settlement conference, "we're basically playing poker,
and what they're doing is, "they're dealing out the cards. "We put the cards on
our forehead facing out, "they read them, and
we don't see anything." So this is all, everything
you say is true, except we don't have access to anywhere near the data that they do. As long as we have protective orders and courts that allow secrecy,
none of this means anything, except in my office, where I can collect my own anecdotal data,
and make a prediction. I know anecdotally what
some of my peers do when I work with them, but
none of us can work together, and those people who don't
have data don't have power, and the people who have data are never gonna let us
have access to that. How do we fix that problem so our clients can begin making decisions
based on information, and not individual attorney anecdotes? Is there any conversation that's going on where we're getting access
to this information and data? – Yeah, awesome question. So if I can paraphrase, doesn't this widen the division
between haves and have-nots? People who are in litigation all the time will have troves of data about this stuff, and individual litigants
will have very little. So I will be the first to say that we don't yet have
all of this data, right? And many of these things
are not unknowable, they're just unknown. We will have to create some of these data products ourselves. And I will say this too. With new services that are being created, and we're trying to create
some ourselves right now, individual lawyers can have more information than insurance companies. Like, in Docket Alarm,
there's 250 million pleadings, complaints, answers, docket sheets, motions, and the outcomes. You can see at a glance,
like who the big players are, and how well they do, how the
lawyer on the other side does. And a lot of insurance companies don't have this information, right, so we are democratizing
that information right now. You'll see a lot of companies who do this. The settlement data you're talking about I think is a goldmine. I can't believe that goldmine is gonna stay locked up forever. So I feel very strongly, although settlement data
is not broadly available, anonymized settlement data
is about to come online. We're pushing pretty hard to do it at Fastcase and Docket Alarm, but I think I'd be very
surprised if five years from now, there's not very good settlement
data available to anybody. So right now, it is definitely true that a lot of the information
we would want is not online, but some of it is, right? Some of it is coming online right now, and so we do have the opportunity to democratize the data that matters. In the age of maps, this was true as well. Maps used to be a collection of royalty. There were times when castles were raided, and the only things that
were taken were maps. Maps were a source of major strategic advantage in world wars. This is why you always see,
like, in these world war movies, like the Map Room, right,
moving stuff across the map. 'Cause if you understand the terrain, you have a strategic advantage. But we know how the story ends, right? These things all get
democratized over time. We live in an era where
this kind of information is more freely replicated and distributed than at any point in the
history of the world. So I don't believe for one
minute that this aggregate data about litigation will stay locked up. In fact, we're gonna try
and open it up ourselves. So there is a good story
and a bad story there. The good side is that it absolutely over time will democratize, right, and it will change the advantage that insurance companies and large litigants
have always had, right? But I would say the downside of that is we're just not there yet. We're gonna get there, though. Next question. – Hey Ed, it's Stan Leer. I was basically just gonna
say what you just said. I was just thinking, so I used to work for
a company called Avvo, which I imagine most of you know, and like, Avvo's initial, key
strategic advantage was that all of the lawyer data was made available by licensing authorities,
and so they took that data and put it in a very
consumer-friendly space, and made that available. Well, increasingly to your point, and I was just talking here
with my buddy Sam about this, like, local court docket data, and Avvo was looking
at this to some extent, to get access to that data, not to hoard it for insurance purposes, but to make it available to consumers so they could have better
insight into which lawyers actually did the type of
law that they say they did, and which lawyers had litigated
against certain judges, and which ones had been more successful. So I was just gonna basically
say that I completely agree, that right now there may
be an information assymetry as it relates to where this data is, but some enterprising entrepreneur is gonna find a way to unlock that data from what is already
publicly-available sources, and make it available in
a consumer-friendly way so that we see some evening out there. I think it's inevitable. – Amen brother, amen. And if I could add, Dan and Nika were instrumental in this chapter of Data-Driven Law talking about Avvo's
methodology in using data to help consumers
understand lawyers better. Data is not the whole picture
to product market fit, but data is one way to help unbreak a market for legal services
that is broken, right? It is a part of the answer,
but not the complete answer. If we want to make the
product/market fit better, from the perspective of
travelers, and clients, data is absolutely a part of that story. I self-consciously in this talk put you in the shoes of someone
making the journey, right? Not of the navigator,
not of the ship captain, but of the person who is on that trip because we should be thinking about this from the perspective of clients, and that's how we get
product/market fit right. Yes ma'am – I work in family law. So you have many, many
more human conditions. I just don't see. I mean, we could, like the judge idea, of seeing what the
judges came through with, I think was a really positive thing there. I just don't see how, there's
just too many variables with the human condition
when it comes to family law. – Yeah, it's interesting. I think that there is this instinct that everything is different,
each individual circumstance, each individual judge is unique. But I think when you start to
look at the data rolled up, patterns really do emerge. You will see that some judges are more likely to grant a
TRO, and some judges less. Some judges will be more likely to grant custody to a
husband, and some less. And over time, when you're able to roll up the empirical data of what really happens, trends will emerge. – But what I'm saying is us
being able to tell the client, "This is what you can expect," or just to help them begin to price it. – [Ed] Right. – We can't tell whether they're
gonna want to fight longer, or their spouse is gonna
want to fight longer. That's all over the board. – Right, so the question is, for the benefit of the recording, "Aren't we gonna still have these, like, "wildly variant fact patterns "that will be hard to
price out ahead of time?" We don't know how ambitious
the spouse is going to be in family law matters, or
what contingencies will arise, but that's always true. That's true of sea travel. It's true of air travel. And what happens over
time is that first of all, you can see what the patterns look like, even if the distribution is broad, you understand what that
broad distribution is, and things tend to revert to the mean. So I'll tell you a story
about data and stock trading. People thought that you couldn't actually use algorithms to trade stocks because there's too much
variability in the value of stocks. Things happen that increase or decrease the value of stocks in ways
that are unpredictable. Algorithms can't understand them, and so there's no way
to do it effectively. Well, we've seen what happens, right? What happens is that the
prices fluctuate less. They achieve some sort of equilibrium. Take another example, baseball, right? So Moneyball was a very
controversial idea. It was practiced by this
guy named Bill James who everyone thought was a kook, right? They said, "There's no
way you can understand "whether someone is gonna be
a good baseball player or not. "There's too much human variability "in the baseball statistics. "You can't really tell
the character of somebody. "You can't really tell whether
there's hustle there or not." When in fact, when you really
boil down the statistics, there's a lot that you can figure out, even in very human industries. So I asked this recently to a big room. Who is the Bill James of legal? Who is the crazy person out
crying into the wilderness, saying that even though
there's variability, there's a signal in the noise, right? There is some economic
rationality to this. There is a way to make a
product and market fit together. That person will likely sound a little bit like a crazy person for a little while, and then everyone will say, "Of course; I always said it was so." – So I feel like data can
often be misleading for people or confusing, maybe based on
different situational factors. Do you have any resources
that can help people ask the right questions about data, and also explaining that to
clients so that they understand? – Yeah, I actually don't. I think, I hope that wasn't a softball for me to recommend my
own book, Data-Driven Law. I think it doesn't fulfill that promise. If there were a guide today that says, "Here's the right resource
to think about data "in a law firm, and how do
I explain it to clients," I don't know about it. I like a couple of resources
on the frontier here, like Kevin Ashley's book
about artificial intelligence in law, but it doesn't really
answer the data questions. So we're right on the cusp of this, we're not in the middle of it. That's also a good news/bad news thing. As we talked about from
the gentleman in the back, we don't yet have all of the
information that we want. That's the bad news. The good news is the bar
is so incredibly low, that almost anything
you do will distinguish your practice from everyone else's. The first time you go
into a beauty contest with any data at all, you
think I'm exaggerating, try it. Any data at all will distinguish you from every other law firm
trying to win the business. It will make you stand out. You will deliver a better legal services. What kind of data? Pull it out of your billing system, right? Take a bunch of matters out of Clio, and just figure out how
much they cost, right? It might take you a day. Have a paralegal hand-tag
'em, and just say, "Here are matters like this,
here's what they cost." I bet you some enterprising soul, at the next LAUNCH//CODE competition will have an app that will start pulling this stuff out of Clio, so you can figure out how
much things cost, right? Will it benefit your law firm, yes, but this is for the benefit of clients. Sir. – Just a quick anecdote
about data gathering and how it can be used quickly. We started gathering this data about our settlements last year, and we just went back and
put 'em on a spreadsheet and tried to figure out
what the average was, and within a month, I
had an averaged number for a certain kinda case that we file, and what I figured out
that we could do with this immediately was, I gave this
average number to my associates so that they could walk
into settlement conferences with magistrates who would say to us, "Well, what do you want?" I mean, we didn't have a number, and we didn't have any way to value cases that have no physical
damages, no economic damages, and the only thing that you were measuring was pain and suffering, and so what my associates
were then able to do is just turn around to
the magistrate and say, "I'm sorry, the average
of these cases is $25,000. "My boss told me that if
I come back with less, "I don't have a job anymore." And so, we walked out with
the $25,000 immediately just 'cause we had an
average data over the last three years to say to the magistrate, "This is what the cases are worth." And they had nowhere to push back on us. We said, "These are
better-than-average cases, "we need a better-than-average
settlement." And those numbers have
just started creeping up because we always bring
better-than-average cases, just like Lake Woebegone. – Right. So we're starting to see the tools, right? Like, pulling the data out of Clio. If you'll permit me, we've just launched inside of Docket Alarm
an analytics workbench that allows you to query
the 250 million documents and dockets for the analytics
that you care about. How many times were
motions in limine granted in Iowa federal courts? What are writs of mandamus
issuance rates like in state courts in Texas? Questions that they can't run reports for in like kind of read-only programs, you can't pull out of
Lex Machina or something, you can now begin to create for yourself inside of Docket Alarm and Fastcase. And so when you're able
to do these things, query your own database, take big data sets and ask questions and get data answers back, it helps improve that product/market fit for the benefit of your clients. It makes legal services less risky and wins you more business. I have given a very nerdy talk
about maps and data analytics for the practice of law right
before the closing keynote. Thank you all for your patience. (audience applauding) (upbeat contemporary music)

One Comment

  1. Анастасия Грачева said:

    Brilliant talk, thank you!

    June 28, 2019
    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *