What does the world's biggest software company do when it needs a Data Center Infrastructure Management (DCIM) solution to manage its massive data center footprint? Well, it makes one of course. Going with home-baked solutions rather than turning to the market is the norm for the team that runs Microsoft's data center infrastructure, so it comes as little surprise that it would take the same route with DCIM.

“Microsoft is the best software manufacturer in the world,” Nic Bustamante, the company's manager of data center operations engineering, says. “And I smile when I say that because I feel comfortable saying that. We understand things about software development that would be difficult for us to go get from a DCIM vendor right now. I don't have to have that conversation with them if we do it internally.”

The second reason Microsoft has turned in-house for DCIM software is the same reason it designs its own data centers, from server motherboards through cooling systems: the way it builds and runs its infrastructure is one of the competitive weapons it brings to the marketplace. “Microsoft very much values the way that we operate and manage our facilities as a strategic asset, so I need to be able to keep that data and the way we handle that data essentially within Microsoft's walls,” Bustamante says.

Its unique approach to infrastructure is one of the reasons it is able to provide its services – be it Windows Azure, Xbox services, Bing or mobile apps – to end users at competitive rates, Bustamante explains. “All of those service offerings that we sell today are at some point being touched by our infrastructure-management software,” he says. “I don't want to share that with competitors.”

Many things to many people

Microsoft has had its in-house DCIM solution for a number of years, although it was not always called that. The platform is open, Bustamante says, meaning it understands common protocols like Modbus, BACnet and SNMP, but also, more recently, gets a lot of data from the hardware, speaking protocols like IPMI and DCMI. “It's a different set of data that in some cases is actually more powerful than that BACnet data,” he says. If in the past, the system was getting BACnet data from a few end-of-row sensors in a room, “now I'm getting data from every machine in a rack, and that's a lot more powerful to me.”

The solution's applications went from simple capacity management to using it to tune the entire infrastructure, across multiple sites, for performance and efficiency. It enables tweaking the infrastructure to avoid building out unnecessary capacity, for example, or to predict potential failures or unexpected capacity shortages, be it compute capacity, storage capacity or floorspace.

Today, the solution provides different things to different service owners within the organization, Bustamante explains. How much of the infrastructure componentry is obstructed can be tweaked based on the user's specific needs. “I've seen that become really rich,” he says. He has seen a lot more service owners at Microsoft say, “Well, this is the process and this is the scenario that's really important for me, and I'm going to use DCIM to model that. I think that's where the more mature organizations are going with DCIM.”

Rather than one software product, the DCIM solution is a suite of tools, Bustamante says. One of them, which Microsoft has talked publicly about in the past, is SKRY, its data center monitoring software. The company has even said it was using its motion-sensing video-game controller Kinect with SKRY.

Unstructured data in DCIM

Development of the suite has been an iterative process that has gone on for about a decade or more. “We have components of it today in use for over a decade,” Bustamante says. Key developments of the most recent years have really constituted the drive to consolidate different kinds of data. Somebody would build an application that would consider a certain data source authoritative. Another application would then transform this data and add to it in a way that made it even more authoritative. “It became a richer data set,” he says.

The willingness of the “owners” of certain data within an organization to share that data is not always there, and this is a common problem not just in Microsoft but in all other large organizations. “That is a very real problem I see with the DCIM world,” Bustamante says. Different stakeholders have access to different sets of data and they guard that access jealously. “The right answer, I think, is you need to share it with everybody,” he says. “The only way that you improve it is by almost crowd-sourcing the improvement of it.”

Integration of more and more types of data is, or should be, the next step in DCIM, in Bustamante's opinion. Especially unstructured data. The most intelligent data center operators out there are looking at ways to leverage unstructured data in their data centers. Whether the engine of choice will become the open-source Hadoop or some other engine, remains to be seen, but one will emerge, he says, because of all the different protocols in the data center. “If I’m not taking all that data, mashing it up together and looking for that low-hanging fruit all the time (What can I become more efficient with and what's most important to me?) across all of the data, then I’m doing myself a disservice,” he says.

A version of this article appeared in the 29th edition of theDatacenterDynamics FOCUS magazine, out now.