In 2002, when Brad Bonnington joined Fenwick & West, the large Silicon Valley law firm was in the midst of Compuware v. IBM, a high-profile lawsuit – later settled for $400m – where the former charged the latter with copyright infringement and violations of antitrust laws. The lawsuit included a body of evidence that consisted of more than 120m pages, and the firm was struggling with managing such high volume of data.
The documents were part paper and part digital and required a sophisticated electronic-discovery software of a kind that did not really exist on the market at the time, Bonnington, director of practice support at Fenwick, recalls. Electronic discovery, or e-discovery, is a process of retrieving evidence stored in digital formats on computer systems used by people or companies involved in litigation and organizing it in a way that makes it useful in court. In 2002, e-discovery as a specific practice did not really exist, nor did specific e-discovery technology.
Over the past 10 years, e-discovery has come of age and has become an industry of its own, with service providers dedicated to e-discovery and law firms that make e-discovery a part of their service portfolios. With the amount of data created and stored growing exponentially, the impact of e-discovery in the data center will only grow in strength.
Retrieving and sorting
Kevin Moore, director of IT at Fenwick, says the e-discovery process starts with a meeting between attorneys on both sides of a legal dispute to agree how they are going to go about e-discovery. The attorneys will then make requests for discovery, specifying a date range. . “They’ll collect relevant data within that range and decide who the relevant custodians are.” Custodians are people that possess access to the needed data on their hard-drives or through service accounts.
Fenwick provides the collection service to its clients. “We go out and collect from the client, and that data is relocated to a specific location relevant to that matter,” Moore says. His team puts the extracted data into a “staging area” of its storage network for processing.
The data is extracted in native formats, such as Microsoft Word documents, PDFs or emails, and processed for the review environment of whatever e-discovery software the company is using. Once in the review environment, the e-discovery team organizes the data and tags it in ways that are useful to attorneys. This can include building a keyword index and de-duplication, or marking duplicate email messages included in email replies, for example. For example, the team may collect 10 copies of the same email from 10 different custodians. All 10 copies would be kept in this case but the duplicates would be marked as such, Moore explains.
During the review process attorneys tag the documents as “relevant”, “not relevant”, “privileged” and so on. The documents are then produced to the court and to the other side.
Behind my firewall
Not all law firms do all of this themselves. The majority outsources e-discovery to third-party providers, who port the extracted data into their own data centers and prep it for the attorneys. Bonnington says, “The trend now is to start bringing it in-house. Back a few years ago it would be very rare, aside from Fenwick and maybe a few others. Now you see more and more law firms invest more in their infrastructure in-house.”
And there are plenty of reasons to do it in-house, not the least of which is it ends up costing the law firm less in the long run. This is one of the reasons Fenwick decided to take e-discovery in-house. “Over time you’re going to save a lot of money and you’re going to save clients a lot of money,” Bonnington says. Another reason is privacy. More often than not, the process involves a lot of very sensitive corporate and private information, and there is value in not storing that information at a third-party provider’s data center.
Matt Berry, founder and president of Lateral Data, developer and provider of a leading e-discovery software called ViewPoint, says for law firms that have their own e-discovery capabilities these capabilities are a major income source. Because they are able to reduce the cost of e-discovery to the client, many also use these capabilities to attract more business.
In July 2012, Lateral, Berry’s company, was acquired by Xerox, which folded it into its Litigation Services unit. Xerox paid US$30m for the firm.
Berry says a common approach to the insource-versus-outsource question today is a hybrid solution. Many firms, he says, will want to manage a certain percentage of their matters themselves, while keeping relationships with multiple service providers, using them for overflow when they take on large projects.
Digging in the Cloud
Overflow is common, as the amount of data retained for e-discovery has been growing exponentially over the past few years. E-mail, mobile devices and data stored with various cloud-service providers are making this curve increasingly steeper. Fenwick’s Moore says one never knows how much storage a litigation case may need. It varies tremendously from lawsuit to lawsuit. In one case, for example, about 1TB of data was extracted from seven or eight custodians, he recalls.
“It’s a graph that’s continuously growing north,” Bonnington says, explaining that the challenge of handling the data gets tougher and tougher every day.
“Storage is getting cheaper,” he says. “People are keeping email longer. Email systems have been around for a lot longer. If they keep that data in place, we often go get it. It seems like we’re implementing new [data-storage] stands and shelves every few months.” After a lawsuit is over, the data does not necessarily leave the walls of Fenwick’s data center. It depends on what the client wants to do or what a judge may order, but more often than not, the law firm holds on to the data.
Majority of the company’s e-discovery efforts today, Bonnington says, still focus on email. “Most of your documents are dragged into email at some point or another,” he says. And it’s not just traditional corporate email. ViewPoint now has connections into cloud email platforms like Gmail, Berry says.
Companies also increasingly store documents on platforms like SharePoint or Google Docs, and some potential digital evidence makes its way to Facebook. “We need to get into Facebook stuff on occasion,” he says. While smart phones are a part of e-discovery today, they are not a big part yet, at least for Fenwick. Neither are tablets, such a Apple’s iPad. “I’ve personally not had to mine data on iPads, not saying I won’t be in the near future,” Bonnington says.
Because they are upgraded so frequently, smartphones are a challenge for e-discovery workers. Just a small firmware upgrade on an Android phone may cause a world of pain for a team like Bonnington’s.
Approaching a petabyte
At the backend of Fenwick’s e-discovery infrastructure are two SQL servers and a storage network. At any given time, there are also anywhere between 50 and 75 virtual “worker machines” hosted on two physical servers that manage the data, taking it through the entire process from extraction at the collection point through to the review point. Finally, on the client side, there is an interface attorneys use to view and tag the documents. Fenwick serves this front end through a terminal environment and has the ability to serve it to as many people as needed, wherever needed. Remote locations can be served over a secure line.
About 100TB of storage capacity is currently online as part of the active database at Fenwick’s data center in Mountain View, California. “That’s just the structured managed data,” Bonnington says. There is an additional 500-600TB of storage capacity that is not being actively used by the database. “We’re definitely adding shelves to existing storage arrays at least on a quarterly basis,” he says. “At least once a year, we get a whole new storage array.” The e-discovery set-up is quickly approaching a petabyte (1024TB) of raw storage capacity, Moore says.
The infrastructure hosting Fenwick’s e-discovery environment is segregated from the infrastructure providing regular IT services to the firm.
Guinea-pig customers rock
Most of the work in Fenwick’s e-discovery process is done in Lateral’s Viewpoint. The law firm was tightly involved in the development of the solution, which has now been around for about seven years. Bonnington’s team was on a mission to find an e-discovery solution that was comprehensive and that would not require any data collected to be stored at a third-party data center. This was a challenging task, as most providers would require that at one point or another the data would have to pass through their infrastructure, Bonnington says.
Outsourcing the function had also become too expensive. “There was so much cost involved in taking a single matter and outsourcing it to a third party vendor,” Bonnington recalls. “We had enough talent to do it internally.” The company’s e-discovery strategy became to outsource only if they could not handle the volume of data internally.
They finally found Lateral, which started in 2003 as a software developer and an e-discovery service provider but quickly got rid of the service-provider part of its business. Lateral had a software solution that fit Fenwick’s requirements and the law firm became a guinea pig of sorts in the process of further development of the solution. “They were very instrumental in providing us resources, a lot of attorneys and the practice support to really take the product to where it needs to go,” Berry says about Fenwick. Lateral had day-to-day feedback from the law firm on which pieces of the solution worked and which did not, making the development process a lot easier for the engineers who knew exactly what functionality ViewPoint needed.
While Viewpoint had been used by a number of law firms “in the trenches”, as Berry says, since about 2004, the product was ready for widespread commercial release in 2009. Today, the company is on version 5.35. The solution is helping Fenwick attorneys sift through sometimes tens of millions of pages of documents using sophisticated search algorithms, concept clustering, category labeling, etc. – something lawyers could never even dream of being able to do less than 10 years ago.
“Back in the day, whenever that was, it was a paper world, and today it’s not,” Bonnington says. “We still gather paper, but at such a small scale, it’s unbelievable.” Whether the new capabilities have made the litigation process more accurate is a tough call to make, he says. However, if one was to judge just by the amount of evidence used, it has certainly become a better informed process.