When Ransomware Cripples a City, Who's to Blame? This IT Chief Is Fighting Back

The former information technology director of Lake City, the northern Florida city that was forced to pay out nearly half a million dollars after a ransomware attack this summer, was blamed for the breach, and for the long time it took to recover. But in a new lawsuit, Hawkins said he had warned the city about its vulnerability long ago — urging the purchase of an expensive, cloud-based backup system that might have averted the need to pay a ransom.

But there was no money. And to those weighing the many competing priorities in the northern Florida city of 12,000 people, purchasing capacity on remote computer servers didn’t seem to rise to the top — at the time. Once the city’s entire computer network crumbled in the space of a few hours, there was an intense round of finger-pointing, and it ended with Hawkins.

“My name has been blasted all over the media and across the country for weeks,” he said in his first interview with the news media since the attack this summer.

The recent cyberattack in Texas, which crippled the computer systems of nearly two dozen cities simultaneously, has served as another reminder of how outgunned most municipalities are against sophisticated hackers. With cities from Florida to Maryland grappling with an onslaught of ransomware attacks that are costing millions, the harsh reality is that it is often one- or two-person information technology offices with meager budgets and strict spending rules that are the main lines of defense.

They are often up against organized criminals and nation-state actors who know how to take advantage of their weaknesses, and who are able to refine their weapons with the hundreds of thousands of dollars in ransoms being paid by vulnerable cities.

The lawsuit Hawkins filed in Columbia County state court Aug. 9 raises the inevitable question of liability: When hackers wipe out a city’s computer system, who is to blame?

“There is a push for accountability, which means firing people. It almost never happens,” said James Lewis, a researcher at the Center for Strategic and International Studies. “A lot of times ransomware exploits a vulnerability that should have been fixed. You need to look: Did somebody slip up on the job?”

Two high-level IT employees were fired after an attack this year in Baltimore, but city officials denied that the dismissals were related, The Baltimore Sun reported. No one in the Texas city of Laredo was disciplined after an attack there. A spokesman for the Texas Department of Information Resources declined to comment, citing the pending investigation.

The troubles in Lake City, about an hour west of Jacksonville, began when several city employees reported that they had fallen for a phishing attack.

Employees at the city clerk’s office, water plant and airport had clicked on an email purportedly from one of their contacts that said something like, “you have an invoice ready.” It was personalized and looked legitimate, but it was really a spear-phishing attack, using what is known as Ryuk “triple threat” ransomware.

One of the emails was cleverly disguised: It even made reference to a prior conversation the city employee had had via email, Hawkins recalled. The email had bypassed spam filters and anti-virus software, which Hawkins said were both up-to-date.

“They were super crafty,” Hawkins said.

Hawkins took the city’s network offline, re-imaged the computers and took other normal precautions. But deep down, he knew that trouble could be looming if anyone else had clicked on the suspicious email without reporting it. The next sign of trouble emerged a few weeks later, on a weekend in early June, when the email system began running slowly.

Nobody works on the weekends at City Hall. So Hawkins waited until Monday morning to tackle the problem, but by then, it was too late. All of the city’s files were encrypted, and a note had been left on the city’s servers that read: “How do you want to open this type of file? Balance of shadow universe.”

Phones were down, email was out of commission, computers did not work, and even the photocopiers were inoperable.

The hackers who had left the note subsequently asked for exorbitant sums of money to release the city’s data.

Even after the city’s insurer paid 42 bitcoins — about $460,000 — for the key to decrypt the files, it took weeks for the city to recover. Some files appear to be still missing, and presumably are lost, said Joseph Helfenberger, the city manager.

Hawkins got a formal letter from Helfenberger on June 21.

“Recent events, including a cyberattack on the City of Lake City and the inability to quickly recover from this attack, including the failure to have in place a reliable and effective backup system,” it said, “have demonstrated significant weaknesses with the city’s IT department under your leadership.”

Hawkins was fired.

Hawkins said that the city could have been able to recover quickly from the attack had it agreed to purchase the off-site, cloud-based backups he had recommended in 2017. City officials balked at the price and went for backups located on the same server, which the hackers sabotaged, he said.

The city did pay for a cloud-based backup for the applications used to run day-to-day business, which was why the city was able to continue offering services.

“It was pretty tough, especially after working so hard toward recovery,” Hawkins said. “Yes, we were affected, yes, we were crippled, but we were still serving the citizens of Lake City the very next day.”

Hawkins filed a public records request for his own hard drive and emails that would prove that he had suggested the extended cloud purchase. His lawsuit this month seeks a court order to disclose the material.

After the lawsuit was filed, the city responded, but said it would be charging about $7,000 to review and redact the records, said Adam Morrison, Hawkins’ lawyer, who said he was also considering filing a defamation lawsuit.

Helfenberger, the city manager, said that because of the lawsuit he was limited in how much he could say on the matter.

“Brian Hawkins might have talked to somebody in 2017 about the need for some improvements, but I did not start working here until August 2018,” Helfenberger said. “I don’t know if he would have put in all the measures we are putting in right now. There are other issues besides this. This was not the only reason he was terminated.”

Hawkins got another job at WatchPoint Data, a firm that has created a tool that helps fight ransomware attacks.

“As soon as I saw the stories breaking that he had been fired, I immediately thought: scapegoat,” said Greg Edwards, WatchPoint’s chief executive. “He was doing the things he knew to do.”

Roy Hadley, Jr., a lawyer who leads the municipal cyber practice for a Georgia firm that represents the city of Atlanta, which was hit by ransomware last year, said incidents like the one in Lake City underscored what cities may come up against: sophisticated hackers, some of whom may have foreign-government backing, whose only job is to launch cyberattacks.

While no government has been accused in the most recent round of municipal cyberattacks, federal authorities identified the digital fingerprints of the Russian military intelligence agency in an intrusion of at least two county election systems in Florida in 2016.

North Korea was accused by several governments of directing the WannaCry ransomware attack that struck computers in 150 countries in 2017.

In the attack on Atlanta, which so far has cost the city about $8.5 million, two Iranian nationals were indicted in December on charges of computer fraud.

“People just need to realize it is a national problem. It is a national problem, and it needs a national response,” Hadley said. “A lot of smaller communities are resource-constrained. If you have $1 million, are you going to fix the potholes constituents have been calling about, open parks and swimming pools for the summer? Or buy new servers and do all the things that are going to make you more secure?”

This article originally appeared in The New York Times.

When Ransomware Cripples a City, Who's to Blame? This IT Chief Is Fighting Back

JOIN OUR PULSE COMMUNITY!