Changing Cybersecurity From An Art To A Science
Paul Rosenzweig was in grad school studying chemical oceanography before he decided to trade a career in science for the law. But there’s still something of the scientist in his approach to cybersecurity. His path to the subject, however, was almost haphazard. It began in 2005, when he was hired by the Department of Homeland Security (DHS). Cybersecurity was only one of the areas he worked on, but it grabbed him and hasn’t let go. He’s been teaching a course on the subject at George Washington University School of Law since 2010. He’s been writing about it from his perch at R Street Institute in Washington, D.C., where he’s a senior fellow. And he’s a senior adviser at The Chertoff Group, while maintaining a solo law practice on the side. He says it’s 50-50: 50 percent of his clients come to him through Chertoff, and the others “come to me because I am a very small and much cheaper version of The Chertoff Group.” He figures about half of his work advances public policy issues, and the balance helps pay the bills. But the enthusiasm with which he talks about issues like trying to measure cybersecurity makes it clear where his heart is.
CyberInsecurity News: What were you doing at DHS when you arrived in 2005?
Paul Rosenzweig: I was asked to help stand up the policy directorate, and I became its first deputy assistant secretary. While I was there, the U.S. government came to realize that there were pervasive vulnerabilities in some of our critical infrastructure—most notably the electric grid—that we had no understanding of and no handle on. So when President Bush directed the government to develop what came to be known as the Comprehensive Cybersecurity Initiative, every department was required to provide staff to that, and at one point I was sort of volunteered—in part because I was one of the law and policy people who had any technical background at all, even though my background was, by then, 20 years old. At least I knew what a computer was. That was 2006, and that was my entry into the field of cybersecurity, pretty much at the beginning of the government’s engagement with the topic in a systematic way.
CIN: Did you know Stewart Baker in your early days there? We did a very long interview with him.
PR: I was his deputy. Stewart was the very first assistant secretary for policy. So not only did I know him, I knew him well.
CIN: Was that an exciting time?
PR: I have really enjoyed all of the work I have done in the cybersecurity realm, because it’s novel. Everything is a new question or a new set of answers, or trying to find a way for old answers to meet new problems. I like to say, “In property law, a new doctrine comes along once every hundred years. In cybersecurity, a new doctrine comes along once every hundred days.”
CIN: Tell us about some of the changes you saw during your four years at DHS.
PR: Cybersecurity at DHS was a startup inside a startup. The Department of Homeland Security itself was a novel beast. When I got there, it was still trying to figure out how to get everybody to wear the same uniform. Or on a single email system. Within that, finding expertise or resources in the department to address cybersecurity was also a novel kind of problem, as was figuring out what the right policy should be. Was this a domain that we were going to have the military defend? Or was it going to be a civilian agency that was responsible? Big, consequential decisions like that. From 2006 to the end of 2008, when I left because President Bush was out of office, we started laying down the very first building blocks of cybersecurity policy for the U.S. government—building blocks that formed the foundation for much of the work that the Obama administration did in the next eight years.
CIN: What about the changes at DHS since you left there? What do you think are the most important developments since 2008?
PR: I think the biggest positive has been the systematization and professionalization of the cybersecurity response apparatus. We’ve developed a whole bunch of capabilities for communicating cyber threat information. DHS has developed a capability to take training and evaluation and combine them into the creation of the Cybersecurity and Infrastructure Security Agency [CISA], which is now an operational agency within DHS—in essence, like Customs, Border Protection or the Coast Guard. It isn’t by any means complete, and I don’t think anyone would say it is, but that’s a big positive. I would say that the biggest negative change has happened in the last couple of years. I don’t mean this to be political, but this administration has prioritized immigration issues at DHS over cybersecurity, and that has, to my perception, slowed some of the forward momentum of the cybersecurity effort.
CIN: There have been a lot of changes in the government’s leadership of our nation’s cybersecurity, and a lot of shuffling. What do you think has been behind all of this?
PR: Any administration can do only two or three things with its attention. President Trump has focused in this sphere on immigration. Other stuff might be health care and tax cuts. I think that the lack of high-level attention to the cybersecurity problem is reflected in the shuffling of portfolios, and the fact that some of them, like the cyber czar at the White House, have effectively been eliminated. Our president is, of course, entitled to set his priorities how he or she sees fit. The lack of consistency in leadership at the department and at the White House on cyber issues is a reflection of the fact that cyber has become a lower priority.
CIN: How secure do you feel we are now?
PR: One answer is that we don’t have any way of measuring that, so I can’t answer. A topic for more discussion. Qualitatively, we’ve invested a lot of time and effort in improving security, and we’ve done a good job in some but not all of the infrastructure domains in eliminating the easy- to-fix risks. We have barely begun a good systematic effort to more proactively think about how to eliminate harder-to-eliminate risks.
CIN: Such as?
PR: Threats from advanced persistent-threat actors—nation-states like China. Criminal gangs. We’ve brought down a lot of risk from drive-by shootings, people who are casual hackers. Most of the electrical grid is now secure again. But we have not developed a strategy for dissuading Russia from targeting the electric grid during times of war. We have no international agreements on norms like that. And we are starting to develop a deterrence model that involves military engagement, but haven’t succeeded yet.
CIN: Is there one agency that ought to be taking the government’s lead on this issue?
PR: I am against a unifying approach. The internet domain is distributed and dynamic. Form should follow function, and it would be wrong to centralize the government’s response in a single space. I think the Department of Defense has interests that are militarily aligned. And regarding the security of its own network, DHS is better positioned to do capacity building and engagement with the private sector. The Department of Justice has an important role in terms of bringing criminal prosecutions. The Department of Commerce needs to help foster innovation. So I think all of the agencies need to play a role. We are missing right now a relatively strong coordinating hand at the White House, which would be valuable. But I would not be creating an über cyber agency to cover everything.
CIN: David Petraeus wrote an article published in Politico arguing that there should be a centralized cybersecurity agency. But it didn’t seem to pick up much support.
PR: Well, I think he was wrong. I don’t say that lightly, because he’s one of the most brilliant military thinkers in recent years.
CIN: One area that DHS is very involved in, and which you already alluded to, is cooperation between the public and private sectors. And that’s what CISA was created to foster and facilitate as far as the infrastructure is concerned. How do you think that’s going?
PR: I think your readers, as consumers, probably have their own opinions about that. I would say B-, if I had to give it a grade. The information-sharing piece of this is improving, but it faces constraints that are impossible to overcome in terms of the incommensurate nature of government information activities and the distributed, insecure information network that we need to feed into. That is never going to be fully successful, but it’s pretty good now. Where we really have fallen down is in finding a way to take the developing set of standards that we have for cybersecurity and turn them into instructions that people will want to implement. Not have to, but want to. We have yet to crack the code of how to make government recommendations that are so attractive that people want to do them.
CIN: If the government was great at that, can you imagine how many people would be lining up to pay taxes?
PR: [laughs] Point well taken.
CIN: To extend the point you were just making, and to return to something you said earlier, you have a special interest in the idea of measuring cybersecurity. And maybe that’s part of the challenge: If we knew what the prescription was for creating good cybersecurity and we could measure it, maybe that would give the government confidence in issuing prescriptions.
PR: And give people an incentive to recognize when the cost-benefit analysis drives a good result.
CIN: Exactly. When did that subject first pique your interest?
PR: I went to law school at the University of Chicago. We were taught the doctrine of law and economics—the idea that most law reflects economic reality, and that it probably should. So I tend to see most of policy issues through the prism of how to incentivize good behavior, or disincentivize bad behavior through economic means. So I’ve been noodling around with this almost since the beginning of my work in the area, back in ’06-’07. But it really didn’t become a research focus of mine until three or four years ago. Every time I went to a private client and made a recommendation, a board member would ask me, “Well, how much does it cost?” And I would say, “That will cost you $5 million.” And they would say, “That’s great. How much better are we going to be if we have this?” And I’d say, “Oh, you’ll be better.” And they would wave at me and say, “No, no, how much?”
I’m obviously caricaturing it, but the inability to express cybersecurity improvements in a quantitative rather than a qualitative means turns out to be key to questions of resource allocation, and oversight, and audit and management by boards. And it’s part of why the government has yet to convince people to do something that might be beneficial to them. Or is it part of the reason? I began exploring the question and found out that people measure security by checklist right now. You’ve complied with certain process-oriented criteria, and you think that helps improve your security profile. And it probably does. But to what degree? And whether or not that’s the right investment of your resources as opposed to buying a new firewall, nobody can say. And that seems to me a bad result. Automotive engineers can tell you exactly how much this new widget costs to add to the car. And they can also tell you, within plus or minus 50, how many lives it will save. And then you can kind of calculate it out. If it’s a $1 million piece and it will save only one life, we don’t do it. If it’s a one-cent piece and it will save a million lives, we do it. And we don’t have anything like that metric system of accounting in cybersecurity.
CIN: This is a big deal that should interest whom?
PR: I think it should interest everybody. It should certainly interest every corporate officer, from the board down to the chief information security officer [CISO], and the lawyers who are doing risk management for their companies. It gives them a way of justifying their decisions. In the end, if something adverse happens, you can look the stockholders in the face and say, “We made this decision on this basis. And here are the numbers that justify it. It was a legitimate risk management choice that we made, and that ought to satisfy everybody.” It should warm the hearts of every chief financial officer who wants to know, if he’s going to invest another $5 million in security, whether he should invest in Product A or Training Program B or Outside Consultant C. It should be useful to the government, which has similar resource allocation and audit and oversight problems. I just had a lengthy discussion with some folks from the Government Accountability Office, who were trying to figure out how to measure cybersecurity inside Department of Defense systems. They have no mechanism that is transparent, auditable and widely accepted. And that’s the goal.
CIN: You warned us that you would have no problem talking the rest of the day on this subject, if we didn’t rein you in. But this subject is worth further discussion. Is your study of this issue under the auspices of the R Street Institute?
PR: It’s an interest of mine that spans all of my jobs, but the principal forum through which I’m doing this is the R Street Institute, which is a relatively new, modest-size center-right think tank here in Washington, D.C., where I am a senior fellow. We have a grant from the Hewlett Foundation to try and do this work, so we’re grateful to them. We’re actively pursuing this question because we have assessed that it’s a strategic gap in America’s ability to be cybersecure. The elevator speech that we gave Hewlett is: We want cybersecurity to be changed from an art to a science.
CIN: How are you going about trying to crack this nut, and what kind of progress are you making?
PR: We broke this work into three phases. Phase one was to do a baseline to try to assess what people think about this and are doing in this space of cyber metrics today. We spent the first six to 12 months canvassing colleagues and the literature. For example, my colleague at R Street, Kathryn Waldron, has published an annotated partial bibliography in which she found all of the different mechanisms that people had tried to use. She’s cataloged them, annotated them, stacked them up in different forms. There are the people who use post-incident reporting as a measure of risk—if you drive your incidents down, they say, maybe you’ve reduced your risk. Either that or the bad guys have figured out a better way to hide from you. But there are lots of different methodologies, and she’s got a nice annotated bibliography of different ways that people think about that. We have likewise conducted informational interviews in which we’ve characterized how people think about it. For example, there’s a strong minority of computer scientists who say that the problem can’t be solved—that cyber risk is too dynamic, that adversaries are too adaptive. You get a number, and it’s out-of-date as soon as you publish it. It might be that that’s where we ultimately come down in the end. But that would be awful depressing, wouldn’t it? That 20 percent of the economy—which is what the government represents—is premised on a domain whose protection level we cannot quantify?
The second phase, which we’re just entering into now, is to take some of the more promising mechanisms that we saw in the first go-round and write about them in nontechnical ways for policy makers and lawyers to come to grips with. We’ll probably have a couple of meetings—off-the-record, Chatham House Rules meetings in which people with different methodologies come together and cross-pollinate. We match them up with legislative people, who tell us what they want to hear—because one of the things I fear is that the measurers are measuring things that the policy makers don’t care about. The goal is to develop some concept of generally accepted metrics systems, maybe more than one, that will be transparent, quantifiable and auditable. And if we can do that, then phase three would be to think about what might be necessary to get those adopted, implemented more widely. That may require funding, or legislative changes, or public education to convince insurance companies to adopt them. Once we figure out what the answer is, or what an answer set might look like, then we would engage in essentially a public education campaign around that, with the hopes of driving a good result in the end.
CIN: We’ve published articles that have focused on cybersecurity’s terms of art. A number of experts have noted that there aren’t very many definitions that are universally agreed upon. Do you concur, and if so, do you think this is a problem?
PR: Yes, I agree. There are very few terms that have generally accepted definitions. Yes, it’s a problem, because we often talk past each other when we talk about concepts that we disagree about. A great example is: What is cyber war? That’s also a problem that’s buried in the metrics discussion we just had. Which is to say, one of the reasons that we don’t have any good definitions is because we don’t know exactly what we’re talking about. When I define what electricity is, and how to measure it, then we have a concrete definition of a watt, and an ampere and a joule. If we wind up having a good metrics system, it’s likely that we’ll go at least part of the way toward a better lexicon for the system.
