In August 2005, New Orleans was on the receiving end of one of the worst-ever hurricanes to ever hit the United States.
The Category 5 hurricane killed 1,836 people in the country, across seven states and caused an estimated $125 billion in damage - the joint-most in history.
It’s sadly not uncommon for such disasters to hit the US, with hurricanes a regular occurrence each year, though each varies in strength and destruction.
In the event of any natural disaster, be it a hurricane, a tornado, or a wildfire, power will often shut off quickly and mobile networks will go down.
Getting these networks back up and running can be a matter of life and death, and the US carriers recognize this. In February 2023, DCD was invited by US carrier Verizon to view first-hand how telcos prepare for such disasters.
“We see crisis and change as opportunities, not threats. We run to a crisis, not away,” the company often states.
“When natural disasters strike, our team is always ready because we've invested heavily and train relentlessly to be deployed at a moment’s notice to assist with rapid recovery efforts of critical communications infrastructure,” says Julie Slattery, senior vice president of core engineering & operations, Verizon.
Patriot 24 drill
Verizon paired with the US Military to carry out a training exercise in Hattiesburg, Mississippi. The location is pretty poignant, as it’s only a couple of hours away from where Hurricane Katrina caused mass devastation.
Lasting for six days, the training drill saw the Verizon Response Team work alongside the Air Force, Mississippi National Guard, and Mississippi Emergency Management Agency (MEMA).
Dubbed “Patriot 24,” the exercise sought to simulate multiple realistic, high-intensity crisis scenarios, including a Category Four hurricane damaging critical communications at a hospital and a tornado causing a nuclear radiation release with network restoration needed in a hot zone.
The purpose of these drills is for preparation, says Verizon.
“All of these teams do training exercises to sharpen their skills and prepare themselves throughout the year,” adds Mark Paff, director for business continuity & event management at Verizon.
“But this is the first opportunity where we had all of them together coordinating on a joint response. We’re also doing it with the Air and Army National Guard so to the extent that we can train, learn from each other, and be better prepared for when a situation does occur, that’s what makes this training opportunity so special.”
Networks go offline
When natural disasters strike, they usually tend to knock out power, causing telcos to lose their network coverage, thus impacting the customers that rely on these networks.
It means that carriers such as Verizon need to be on high alert when the networks go down.
For example, in 2022, when Hurricane Ian hit Florida and South Carolina, Verizon, along with T-Mobile and AT&T, was on hand to restore telecommunication services to those affected.
During that occasion, Verizon deployed its Wireless Emergency Communication Centers (WECC) in areas of Southwestern Florida that were impacted by the storm. The WECC are generator-powered mobile units that have device charging and computer workstations, along with wireless phones, tablets, and other devices.
On this occasion, the telco had to complete site surveys of impaired cell sites, while moving portable generators, and mobile cell towers where needed.
These sorts of responses are the norm during these outages, but to get to this point, a serious amount of preparation is required. Several teams took part during the drill. This consisted of the carrier’s Major Emergency Response Incident Team (MERIT), Dedicated Impact Response Team (DIRT), Emergency Response Team (ERT), and Verizon’s Frontline team.
MERIT focuses specifically on responding to hazmat conditions, while DIRT takes care of Verizon’s Radio Access Network (RAN) infrastructure. The role of the ERT team is to keep on top of fiber infrastructure, while the Frontline squad works with local and public safety agencies.
The Frontline team is made up of many people who are first responders themselves and is focused on providing solutions to aid first responders at the offset of a disaster. Frontline often provides solutions for critical functions such as search and rescue.
Verizon’s approach to training
During the drill exercise, the various Verizon teams were tasked with a range of drills.
Paff explains that the drills are necessary for the various teams to competently and confidently respond when needed during a crisis.
“We drill all time time,” he tells DCD. “We design our networks for resiliency. A major storm can sometimes have disruptions for that high resiliency, so we train all the time.
“We also do situational awareness. My team does 24/7, 365 monitoring globally of anything in the world that could disrupt communications, we are a global service provider so we have services in all the major countries in the world.
This means we're constantly monitoring for weather threats, natural disasters, or even manmade events, such as civil unrest, and things of that nature.”
When a hurricane looks likely to occur, Paff says that Verizon will usually have advanced warning before it works on its checklists.
“So, we have three sets of checklists that we follow, with the first pre-incident checklist, these are the things that we're doing to prepare all of our response teams in advance of an oncoming threat.
“After this, we have our incident checklist which is after the event occurs, and when we're responding to it, we're making sure that all of the teams are following through with our checklist.”
He notes that during this phase, Verizon needs to ensure it has people with the right skillsets in the right areas carrying out repairs. The final checks that Verizon carries out are post-incident, where the carrier ensures that everything is closed down.
Once the incident is finalized, Verizon will then hold after-action reviews and analyze what worked and what maybe didn’t during its response efforts, adds Paff.
Support from the military
Also on hand at the drill, were the Air Force and the National Guard. The Military plays a pretty important role during natural disasters.
Although it’s the job of carriers such as Verizon to get the network up and running in the wake of a disaster, the Army is also required to lend its support during these humanitarian efforts.
The military supported the rescue efforts during Hurricane Katrina and will attempt to save lives and deliver supplies.
“So within the first 72 hours, we have our National Guard assets, and they can be on scene usually very quickly if they're in the local area,” says Lt Colonel Ashley Nickloes, director of Exercise Patriot.
Nickloes says the National Guard carries out a drill once a year, to prepare troops for a real-life scenario.
It’s no secret that military forces will communicate in a different way to the average person, and this could even be a barrier during a rescue mission, however, Nickloes says that they try to keep it as simple as possible.
“Air Force has a different terminology and acronyms that we use to what the army does,” she says, noting that even this can lead to confusion.
“One of the things we’ve tried to do at Patriot is say, 'hey, we can't use acronyms here,' because we're working with our civilian partners. We want to make sure that when we're talking, everybody's talking on the same page, and from the same script so that we can maximize what we're doing, and that nobody feels intimidated. It's about breaking down those barriers.”
Driving a response
Typically when a powerful storm hits, or a tornado sweeps through a settlement, cellular coverage is lost. This happens regularly during these disasters and was an important part of the simulation held during Patriot 24.
To support the response efforts, Verizon has a substantial fleet of vehicles on hand across the country.
The telco says it has more than 550 of these portable assets scattered nationwide in anticipation of disasters. The vehicles include fully functional, generator-powered cell sites that can replace or enhance network coverage where necessary.
It’s hard to miss the vehicles too, which are heavily branded in red with signage including ‘Verizon Response’ on them. Verizon says the branding is deliberate, as it aims to bring a source of comfort to those in need of support during disasters.
One of Verizon’s advancements in its fleet in recent years is THOR (Tactical Humanitarian Operations Response), which is essentially a custom Ford F650 that can deploy a 5G mast and satellite dish as it responds to emergencies.
THOR runs on Verizon’s ultra-wideband 5G network, which includes mmWave and Edge compute capabilities.
A job for the dog
One particular scenario that played out at Camp Shelby, saw a tornado cause a nuclear radiation release with network restoration needed in a hot zone.
The drill saw Verizon demo its Robotic Emergency Dog, otherwise known more affectionately as RED. But it’s not just any dog, RED can provide a secure and reliable private 5G network for communication.
RED can be controlled by a handler, using a tablet-like device for navigation in between areas. Verizon says it has about seven of these robotic 5G canines.
The purpose of RED is for the robotic dog to go into potentially dangerous areas to scope situations. Verizon says it’s easily deployable and can be used in remote locations and under harsh conditions.
During this particular drill, RED was tasked with finding various sources of radiation in the buildings.
With the chemical leak in full flow, the engineers in full hazmat were called in to carry out their tasks of clearing the area, along with some who would be splicing fiber cables to replace the damaged comms. To splice a fiber cable is a tough task at the best of times, but MERIT teams are trained to do this while wearing a full hazmat suit and thick gloves.
“This particular drill is satisfying some of the hazmat requirements that our MERIT team is required to take. Our MERIT team consists of about 40 engineers and technicians whose day job is to operate and run all aspects of Verizon’s network,” says Paff.
He adds that the MERIT team is also trained hazmat technicians, at the “same level that a fire department hazmat team would have to be.” These members of the team are required to do 80 hours of hazmat training each year, plus one drill.
“They are trained to go into those very dangerous situations, we always say that we run to a crisis, and respond to bring communication services up which are so critical for our customers, communities, and first responders.
“They'll be in the protective gear is a specially designed suit with breathing apparatus that allows them to be solely fully self-contained.
"They can go into those hazardous situations and do the kinds of communicant repairs that they need to bring our communication infrastructures back online, including splicing fiber optic cables that are damaged.
Repairing infrastructure
Unsurprisingly, when a hurricane or bad storm hits, cell towers are caught up in the chaos of it all.
If a cell tower is taken down it can be some time before it’s safe to erect a replacement site or fix an existing damaged cell structure.
During an event, if this is to occur, Verizon’s ERT team is tasked with handling the landline and fiber optic parts of the network. Because fiber optics connect Verizon’s cell towers to the network, if they are damaged during a storm event, it’s the job of the ERT team to restore them.
However, any fixes to the actual towers themselves are carried out by the DIRT team, explains Paff.
“Our DIRT team is focused on the wireless component of our network, and those are all the people that can restore and fix any damaged towers,” he says.
He adds that they’ll inspect the tower and base station on the ground to assess any damage to determine if it’s from a power perspective or a backhaul fiber optic perspective, allowing Verizon to initiate the next step in its restoration process.
“There’s satellite dishes that provide the backhaul antennas that can go up 60 feet, and we have portable towers that we can bring up even higher so they can quickly and temporarily restore communications.”
Paff says that small cells can even be put into its drones so that they can fly up and propagate RF signals to a certain area.
To ensure that Verizon can bring temporary communications to people in need, it has a range of solutions, including its COWs (cells on wheels) and TOWs (towers on wheels).
If power isn’t easily available, Verizon has other reserves in tow, adds Paff.
“We design our networks with multiple sources of power, with battery reserves at all of our sites, so if it loses power, we have generators in almost all of our major sites.
“We have national fueling agreements with major fuel suppliers to make sure that our generators stay fueled during a crisis event. We have a fleet of portable generators. So if everything fails we will strategically place these mobile generators so that we can quickly get out there and restore power.”
Eye in the sky
To get the networks back up, a range of technology is used. Drones are particularly useful, as they can scale heights that are otherwise impossible.
“Drones are a great asset for getting those eyes on the ground where it may be unsafe for people to go in there,” says Paff.
Verizon has previously showcased its usage of drones, doing so during Hurricane Ian, where drones were deployed to provide cell coverage to responders and residents over an approximate five to seven-mile radius.
Drones have also been used to assess wildfires and floods in California and to help spot people trapped in ice in Colorado.
The telco says these drones can fly for up to 1,000 hours and can go as high as 400ft, providing it has the connection.
Significantly, the drones are also 5G-enabled and use Verizon’s ultraband spectrum. The drones are operated by FAA (Federal Aviation Authority) certified handlers too.
“Drones are absolutely mission-critical pieces of equipment,” states Chris Sanders, senior manager, Verizon Frontline crisis response team. “Our drones are able to provide live real-time streaming, both electro-optical and thermal imagery to our public safety agency via our robust network.”
Preparing for the future
The overall purpose of the drills is to prepare for future disasters, but the learnings gained from the events and widespread collaboration are key, says Nickloes.
“The biggest takeaway from these exercises is from meeting partners and building relationships so that we know who we can depend on when we hit the ground. That’s really what the exercise is all about.”
She adds that technological advancements mean more can be learned from the drills.
While US operators typically compete with each other for market share, this mindset is forgotten during times of disaster, adds Paff.
“We have entered into relationships with other carriers so that if they are damaged and need services they can come to us,” he says.
“So, if we've got the capacity and the equipment, we will help them because again, we're all supporting customers and communities and the first responders, so that the whole competitive thing goes by the wayside when you're dealing with a natural disaster or trying to bring communications back up.”
With hurricane season fast approaching in the US, along with the threat of other unpredictable weather, the US carriers will likely put their skills into action once again, to provide the critical connectivity that millions rely on.