HP is making efforts to expand its reach into the market for “Big Data” analytics (extracting useful business intelligence out of large and often chaotic mixes of data), using its hardware, software and services capabilities.
Much of this play is offering customers help with leveraging Apache Hadoop, an open-source distributed-computing framework that provides a way to cluster many simple commodity servers to process massive workloads. It has become one of the most popular platforms in the quickly growing space of unstructured-data analytics.
The company announced on Monday a hardware architecture designed to run Hadoop and unveiled consulting services to help companies use the framework. It also announced that Vertica – a company it bought in March 2011 – will integrate the latest release of its analytics platform (Vertica 6) with Hadoop.
Vertica 6 – also launched on Monday – can use data it gets from a variety of sources, HP says, which Hadoop is now one of. It takes this ability to use many different interfaces from its FlexStore architecture, according to HP.
In a statement, the company said this architecture enabled integrates or federation with Hadoop, Autonomy (another recent analytics acquisition) or literally “any other structured, unstructured or semi-structured data source.”
In addition to Hadoop, the latest version of Vertica adds native support for parallel execution of R (a programming language developed specifically for data analytics), and better support for implementations on cloud infrastructure or for Software-as-a-Service delivery.
HP made all of the announcements at is annual Discover conference in Las Vegas.
The hardware architecture for Hadoop unveiled on Monday is called HP AppSystem for Apache Hadoop. It consists of an 18-node cluster of HP’s latest ProLiant Gen8 DL380 servers and HP networking gear, all managed through the HP Insight Cluster Management software.
The vendor can deliver the system as a “turnkey appliance” for customers that do not have the resources or the desire to design and build a Hadoop system in-house.
HP has developed reference architectures, tools and whitepapers to support the top Hadoop distribution vendors: Cloudera, Hortonworks and MapR. These companies provide commercial products and services associated with Hadoop besides distributing Hadoop itself.
Cloudera, one of the three, is also partnered with a range of major HP competitors, including Acer, Dell, NetApp, SGI, Oracle and Supermicro. Hoartonworks’ partner list includes NetApp, Seamicro and Teradata, as well as Vertica, among others.
MapR’s has EMC, Cisco, IBM, Teradata and VMware as its partners in addition to many more.
Autonomy, whose founder and until recently CEO Mike Lynch was let go in May, is also now more tightly integrated with Hadoop. HP can now embed the unit’s Intelligent Data Operating Layer (IDOL) 10 engine in each server node in a Hadoop cluster.
Rafiq Mohammadi, CEO of Autonomy Promote, an Autonomy unit, said, “IDOL 10 is now part of the Hadoop network. Enterprises can drop an IDOL instance right into a hadoop node.”
IDOL is the “heart” of Autonomy’s infrastructure software, the company says. The secret sauce is the company’s proprietary structure for storing data, optimized for fast processing and data retrieval.
Close coupling with IDOL 10 gives users access to hundreds of its functions, including automatic categorization, clustering, hyperlinking and more.
Lynch's departure was announced on the same day HP announced its Q2 fiscal 2012 earnings and a new restructuring plan that included laying off or sending into early retirement about 27,000 of its staff around the world.