When the Covid-19 pandemic hit, healthcare services in New York depended on rapid shifts to hybrid IT, DCD's Fall New York (virtual) event has heard. Meanwhile, sudden booms in demand for videoconferencing and other apps demanded fast innovation for service providers.

"At some point New York was the center of the pandemic," said Bashir Agboola, CTO at the NY Hospital for Special Surgery. As the hospital adapted to changing roles in the crisis, and staff worked from home, he had to upgrade bandwidth 1000 percent, he told the keynote of today's DCD>New York VIRTUAL (Fall Edition). "We had to scale up our infrastructure overnight to upgrade our Internet bandwidth by 1000 percent very rapidly."

The same panel heard how videoconferencing services shifted their underlying cloud resources, while content-delivery infrastructure moved live services to keep customers connected. to support more demand.

Healthcare had to pivot

The Hospital for Special Surgery changed role twice in quick succession, said Agboola. Its normal specialty is orthopedic surgery, "but that is non-essential, so we had to stop doing that and pivot to taking in emergency cases". It then shifted to dealing with Covid-19 patients. At the same time patterns of work changed abruptly. going from one percent working at home, to a sustained level of "30 percent working from home simultaneously, day in and day out".

As well as upgrading its network service, the hospital had to source hundreds of laptops immediately, upgrade its VPN services, and adjust the IT in clinical rooms, to match their change us use. This meant adding webcams so staff could assist without spreading infection. At the same time the hospital had to beef up its support desk: "We took on dedicated level 2 staff, because people working from home tend to have issues."

Like many other users, the hospital increased its reliance on the cloud. "Like many healthcare organizations we have been slow to the cloud game," admitted Agboola. "We have increased our reliance a little more, and seen that this will allow us to respond better to any future crisis." For example, its contact center was previously on premises - but Agboola noted that a cloud-based contact center would have made the move to home working less of a struggle.

"This past few months has lent more ceredence to the need to move more workloads to the cloud," said Agboola. "In the next year as we recover from the economic impact of the pandemic, we will be able to do more."

Friendly sidekick

Of course, that shift in usage stress the infrastructure and application services of the cloud, which had to step up, and make their own changes: "The heroes of the moment were the healthcare workers keeping us safe, and in some ways the Internet was the friendly sidekick," said Nitin Rao, SVP of global infrastructure at content delivery network Cloudflare "Demand increased four times, and we were able to provide the tooling to help other companies work from home. The Internet was built for this."

At the applications level, Verizon's Bluejeans videoconferencing service saw demand go up by 250 percent in the first week of lockdown, according to CTO Aligu Periyannan: "We couldn't have dealt with this just in our own data centers."

At the start of the crisis, 85 percent of Bluejeans' traffic was in its own data centers, said Periyannan. "During the first three this weeks flipped around". It's stayed that way, more or less, he says, with 70 percent of its traffic still on the public cloud: "The hybrid approach really saved us," he said, "Without it we wouldn't have been able to survive and satisfy all our customers."

Bluejeans had to be clever about the cloud services it used. said Periyannan:.the company blended services including the usual AWS resources, alongside expensive on-demand capacity, and experimented with bursting some traffic to "spot" instances which are cheap but can be "yanked back from under you" if others demand them. This worked fine for the first week, but "in the second week we found we were losing massive loads of spot instances, at around 3pm West Coast time."

Periyannan suspects this was because West Coast people start work early and finish early. After the lockdown pattern got established, in the afternoon. Californians would "sign off from work, and sign on to Netflix and gaming services". To increase reliability, Periyannan shifted away from spot instances in AWS' West Coast region in favour of regions like Ohio. "We had to play the game and work closely with our cloud provider to understand these patterns. "

Cloudflare had to be clever to meet the demand, said Rao. It runs all its services on commodity hardware which can handle any task - but increased network traffic demanded increased throughput, and more servers were needed at a time when there was a sudden demand for hardware. The company re-engineered its firewall server software to use 40 percent less resources,. freeing up hardware, and space. "That's an operational benefit, as it bought time for teams supporting increased loads, but it's also an enduring win. We used this as an impetus for problem solving."

Partners

All three said partnerships were key. Agboola praised hardware suppliers: "Our infrastructure scale ups would not have been possible without the active participation of our partners. Everyone else was looking for the same thing as us. We needed hundreds of laptops, and had to leverage our existing equipment partners, at a time when everyone else wanted laptops."

Rao praised the resourcefulness of hardware engineers: "Staff at data centers kept installing hardware as it arrived," he said, and the equipment got through by truck when planes were grounded. Governments made exemptions to allow factories to make those servers, and get them transported during shelter-in-place orders: "That’s been the silver lining in a lot of this."

And Agboola predicts we will see a boom in telehealth now the sector has been pushed into relying on online services. "Telehealth was struggling to take off. Now it is here to stay." Visits to health portals grew from thousands to millions during the months of pandemic, he said.