Silicon Valley data analytics company Palantir Technologies will deliver a comprehensive combat intelligence hardware and software suite for the US Army, in a contract potentially worth more than $800m.

The contract award for the Distributed Common Ground System (DCGS-A), first reported by the Washington Post, came after a fierce competition between Palantir and rival military contractor Raytheon.

Separate DCGS programs exist for the Navy, Air Force and US special operations forces.

Digital warfare

The DCGS-A, a mixture of software, ruggedized computer networks, and sensors, acts as the Army's primary system to post data, process information, and disseminate Intelligence, Surveillance and Reconnaissance (ISR) information about enemy troop movements, weather and terrain. It should be easy to use by soldiers in harsh environments, and - for obvious reasons - be very secure.

"The Army Operating Concept, Win in a Complex World, requires intelligence warfighting function training to increasingly focus on employing the DCGS-A as a "weapons system" to support expeditionary operations with light and lethal formations capable of deploying quickly," a 2014 US Army article on DCGS-A by Maj. Gen. Robert P. Ashley and Col. William L. Edwards states.

"The emerging operational environment is increasingly dominated by the proliferation of technology and rapid information exchange."

The existing DCGS-A system cost more than $3bn and took a decade to develop with help from Lockheed, Raytheon, and IBM, but was seen by many in the Army as outdated and ineffective. Palantir hoped to pitch a replacement - based on its existing commercial technology, Gotham - but the US Army instead chose Raytheon for an early phase of the project.

Battles in court, first

In 2016, Palantir - which was co-founded by PayPal's Peter Thiel, an unofficial advisor to President Trump - took the Army to court, arguing that it was required by law to consider buying commercial products when available, rather than building custom ones from the ground up.

The Complaint court documents, published by Government Executive, detailed some of the problems with the original DCGS-A, according to Palantir. Core to soldiers' difficulties with the system were repeated delays over when it would shift to the cloud.

"In 2011, DCGS-A program owners predicted a March 2011 date for the DCGS-A Cloud to reach Initial Operating Capability, with more robust capabilities planned for 2012. In contrast to the positive outlook espoused by DCGS-A program owners, Soldiers reported as late as 2013 that the DCGS-A Cloud had been “offline for months” and was not synchronizing data between data centers.

"Around that time, Lieutenant General Legere subtly shifted her story, claiming that the new version of the DCGS-A Cloud, now dubbed “Red Disk,” would be operational by the end of 2013, and insisting that previous efforts were only “experimental,” despite earlier assurances that these efforts were meeting critical Soldier needs."

Other technology issues persisted: "According to an email sent to Congressman Duncan Hunter in April 2013, 'DCGS-A servers deployed in Afghanistan [did] not share their databases with one another.'"

Palantir also pointed to several small cases of troops using its products, after Lt. Gen. Michael Flynn issued a report claiming "the enemy is able to hide in plain slight" because "intelligence analysts in theater currently do not have the tools required to fully analyze the tremendous amounts of information currently available."

That report led to the Pentagon's Combating Terrorism Technical Support Office supplying Palantir servers to troops in Afghanistan. This move was not without its own controversy - as Wired reported in 2012, some saw lobbying at play. Flynn's report "was clearly ghost-written by a Palantir engineer," a member of the Army's resource planning directorate wrote in a February 24 email.

Around the same time, Palantir massively ramped up its lobbying spend, its Washington, DC advertising spend, and set up a Political Action Committee to back politicians it favored.

"There are multiple requests for capabilities in theater and many are ghost written by commercial vendors," a set of Army talking points, dated July 25, 2012, stated. Others pointed to the fact that Gotham had its own reliability and scalability issues.

In an effort to provide some level of independence, the Army Test and Evaluation Command was brought in to survey troops on Palantir’s product, with an April 25 report noting that "DCGS is over-complicated, requires lengthy classroom instruction, and is easily perishable skill set is not used constantly... [The Army should] install more Palantir servers in Afghanistan [and] incorporate a one-week training class on Palantir."

The report was subsequently removed and replaced with one that no longer recommended buying from Palantir.

Palantir won its case in 2018, forcing the Army to consider between it and Raytheon, both of which were allowed to test and refine their product in trials with soldiers.

The wider picture

This award now marks a culmination of that competition, and the fruits of a long-term effort by Palantir to win Army contracts, after years of success in the intelligence sector, and limited success with corporate contracts.

Founded in 2004, the company soon received an investment from In-Q-Tel, the CIA’s venture capital investment branch. Its software is now used by numerous federal, state and local law enforcement agencies, with some contract - such as one with the Immigration and Customs Enforcement agency - mired in controversy.

It is this controversy, of working with politically difficult agencies and the military industrial complex as a whole, that much of Silicon Valley has balked at.

Even when the corporations themselves have wished to do business with the military, it has proved difficult - with Google's employees writing petitions, staging protests, and even resigning.

Palantir does not have this problem. From the beginning, it clearly defined itself as a company trying to work with the government, painting itself as the business that will bring Silicon Valley's AI and big data smarts to the nation's intelligence agencies.

Elsewhere, we have detailed the US government's national AI strategy, as well as the Department of Defense's cloud and AI plans, which follow the same core themes - the government is eager to tap into the innovations of big business in the realms of artificial intelligence and cloud computing.

How these two realms - one of military needs, and the other of corporate and staff desires - will interact over the coming years is not clear. But what is known is that, whatever happens, it will profoundly impact both the future of computing, and of society.