A man in disguise. Armed to the teeth with fake documents, plans to the building, and tablets to help him foam at the mouth and fake a fit in a tight squeeze. This hit has been in the works for weeks. Having called ahead under false pretenses to ensure he’s expected, he walks up the driveway, mentally preparing to break into a high-security data center brimming with critical data and applications… hey is the fire escape door propped open?
While cyber takes the headlines, the physical security of data centers can’t be overlooked. While it may seem simple, the access controls that protect your facilities – and the people that roam the facilities – can easily be compromised if a company is lackadaisical in its approach. So, is the physical security of data centers as good as companies make out? “No,” Andrew Barratt, principal consultant, adversary ops, at security firm Coalfire, says simply. “I could probably count on one hand how many are well thought-out. It gets forgotten because it's a presumed commodity. “Everyone thinks all that stuff just works. And then they don't think about the real-world threats to those physical controls.”
Physical security needs thought
Barratt notes that to do physical security well, companies have to really think designs out well and model threats properly. Otherwise, security controls can often become ‘theater;’ looking like they are doing the job, when they are actually easily circumvented either by attackers or the staff meant to be enforcing them.
“Some of the newer data centers, they'll have things that will look cool from a security perspective, but then you'll see people smoking outside the fire escape and you could have just walked in with a packet of cigarettes.” He notes that the poorest levels of security – where simple confidence tricks of wearing high-vis jackets are most likely to work – are in corporate data center facilities which serve one company and its subsidiaries.
“It's much more common that you've got peripheral defense, a bunch of very rudimentary access controls, or you'll see gates that you could sneeze and fall over to get into a building. I've even seen some environments where the security guards themselves would let you in if you just looked like you were struggling with the key card.
– Sebastian Moss
“In my experience, the ones that have been really well crafted have normally been designed by folks who are ex-military,” he explains. “They’ve been thought out purely from a military perspective for use as critical national infrastructure environments. Generally, the best people who do this professionally are those with the intentions of getting either government, military, or critical infrastructure hosted with them.”
An example of good design might be in the parking. There should be an area to pre-scrutinize and hold visiting cars before they reach the final car park so they can be rejected without creating gridlock. Another would be gates – it should be impossible for someone to tailgate the person ahead of them. However, this is expensive, and corners are often cut for commercial reasons.
Barratt suggests that instead of ‘crappy solutions’ that may be vulnerable or create a false sense of security in a facility’s personnel, companies should sometimes simply accept the risks.
“It's sometimes better to just not do it and know there’s a risk that people have got to be more vigilant towards personally,” he says.
Rishab Verma, of the penetration testing team at Defense.com/Bulletproof, notes that fire exit doors are often neglected and make an excellent point of both entry and exit. “Sometimes people use it, maybe going to lunch and wanting to get out quick, and it's just left open,” he says. “There is no good security, or access control in place for fire exit doors; I can just simply use the fire exit to get out of the building.”
He notes that a lack of logging can make it harder for companies to track personnel. People should be automagically logged for both entry and exit time, and failure to do so – if, for example, a door was held for them at the exit – should be flagged.
The human factor should be a focus
One of the primary routes for a penetration tester to gain access to a data center is through the people; tricking reception and security staff into letting you in directly, or creating scenarios where they can be fooled or distracted long enough to allow an attacker inside.
“I've lost track of the amount of times I've put on a striped suit and walked into a building because people just think you look important,” says Barratt. “The old school confidence tricks are very successful and hard to defend against.”
Equinix CSO Michael Montoya talks to DCD about the company's September 2020 ransomware attack, and how it responded and recovered
Security guards, despite themselves, can often be a weak point in defenses. Often low-paid and outsourced roles, these staff can be over-eager to help for fear of losing their job.
“What really is required is a degree of hostility, in a social environment where people have got a very customer service mindset. The big changing point when it comes to physical security is making them feel like they're actually part of the business and have a valued role,” says Barratt. “You need the CEO to say ‘you can stop me and I'm not going to fire you.’ The CEO even probably needs to make an example of themself occasionally. It requires leadership and good management and actually really good soft skills and team management, so that they don't feel like they can just be bowled over by somebody playing the ‘I'm more important than you’ card.”
Physical security teams operate most effectively when they feel like they're a cohesive part of the overall business and feel empowered. If security guards that are hired help on low pay, and worried about being let go, they aren’t as likely to stay sharp or feel brave enough to challenge people that might be senior to them.
Barratt notes that a CISO he worked with had a portfolio of buildings including data centers, and was concerned about the security guards at his premises.
“On a number of the security tests, the security guards were actually the biggest weakness because they were socially conditioned to be helpful to people; anything that they could do to feel like they were valuable to the business they would try and do.”
What this CISO did was halve his physical security team and triple their pay. They were then split into two teams and made to operate in military-style tactics permanently against one another. They ran a leaderboard and would offer rewards to successful teams.
“It was a really fascinating play because the people were mostly the same but they permanently had a team that was on high alert because they knew their counterparts were always trying to simulate a break in. That level of alert rapidly changed their security team almost overnight to the point they wouldn't trust anybody."
Access controls aren’t impenetrable
Physical access controls such as key cards, biometrics, CCTV, and mantraps can make a facility much harder to break into.
However, many keycard-based systems can be easily circumvented. There are devices that can scan cards in the immediate vicinity, clone them. Some older cards could have their encryption broken to allow attackers to make entirely new profiles for them. Employee mistakes can also make it easier to compromise fixed access controls. Staff should know to keep items such as key cards hidden and secured.
“Depending on the notoriety of the target, users often post pictures online of their key cards on social media,” says Nicky Whiting, director of consultancy at Defense.com. “From there on, it's very simple to create your own key card with a similar layout and print that out on some plastic.”
Eric Florence, cybersecurity consultant at SecurityTech, tells DCD that a small facility he previously worked at conducted regular penetration tests, with one incident involving an attack pickpocketing a security card from an employee.
Keatron Evans, principal security researcher at Infosec Institute, says constantly reminding staff – whether security or more technical roles – of potential security threats is the best way to ensure procedures are followed properly.
“The regularity at which you're reminding them of these things is important,” he says. “When you do a good security awareness campaign, for a month after, the success rate of people blocking attacks and things goes up tremendously. But you go out five to six months later, and it's almost back to where you were in the beginning.”
Evans also points out the importance of screening those given access in the first place to prevent keycards and important information from falling into the wrong hands.
“Companies should do a better job of scrutinizing and background checking because there are cases of people slipping into places through that mechanism. Not just the technical employees, but people like janitors, cleaning crews; anyone that has physical access to the facility.” Sometimes companies make even more basic mistakes that render existing controls moot. Gillian Vanhauwaert, of the penetration testing team at Defense.com/ Bulletproof, notes that one facility had a sign noting gatherings every Wednesday at a certain time. During that gathering, a paper box was put in the door to allow people to walk in and out. “It was advertizing how to get in, so I just had to wait till that time and I just walked in,” she says.