Hack the Plant Episode 6: Critical Response for Critical Infrastructure with Megan Samford
“So I think that the private sector has a responsibility to be involved in the conversation in the least. I think that the private sector understands its importance. I go back to the data point around 85% of critical infrastructure is owned and operated by the private sector. Of course, the private sector understands this. And I think that government has shown a willingness to be receptive to what the private sector is saying in terms of, again, you go back to the model of behaviors you want to see more of and behaviors you want to see less of. And DHS, to their credit I believe, has done a pretty good job in this respect, as well as Department of Energy.”
That’s Megan Samford, who serves as a Vice President and Chief Product Security Officer in Energy Management at Schneider Electric. She is the first woman chief product security officer in ICS manufacturing and her background spans the public and private sectors. She was formerly in product safety and security leadership positions at Rockwell Automation and at General Electric. Before that, she served as Emergency Manager in the Governor’s Office in Virginia as Critical Infrastructure Protection Coordinator.
(Subscribe to Hack the Plant on Spotify or Apple, by RSS feed or search for it wherever you listen to podcasts.)
Transcript:
Joshua Corman:
Our dependence on connected technology is growing faster than our ability to secure it, especially in areas affecting public safety and human life.
Bryson Bort:
I’m Bryson Bort. And this is Hack the Plant. Electricity, finance, transportation, our water supply. We take these critical infrastructure systems for granted, but they’re all becoming increasingly dependent on the internet to function. Every day I ask and look for answers to the questions. Does our connectivity leave us more vulnerable to attacks by our enemies? I’m a senior fellow at the R street Institute and the co-founder of the nonprofit ICS Village, educating people on critical infrastructure security with hands-on examples, not just nerd stuff. I founded GRIMM in 2013, a consultancy that works the front lines of these problems every day for clients all over the world.
Bryson Bort:
For today’s episode, I’m joined by the first woman chief product security officer in ICS manufacturing – Megan Samford. Megan is a Vice President and Chief Product Security Officer in Energy Management at Schneider Electric.
Megan’s background spans the public and private sector. She was formerly in product safety and security leadership positions at Rockwell Automation and at General Electric. Before that, she served as Emergency Manager in the Governor’s Office in Virginia as Critical Infrastructure Protection Coordinator.
Today, we’re here to explore security risk management for product manufacturing. What is the threat landscape? How can we build security into our new products? What kind of knowledge sharing is there between the government and private companies?
We also discuss a new initiative called ICS4ICS (Incident Command Systems 4 Industrial Control Systems) that Megan is spearheading – combining Megan’s background in homeland security and emergency management in the Virginia state government with her experience with Industrial Control System security.
Bryson Bort:
Megan, thank you for joining us today on Hack the Plant. Please tell us about your background and current role. You’re the first woman chief product security officer in ICS manufacturing, correct?
Megan Samford:
Correct, with the nuance that I would say there have been chief information security officers that were female that held similar roles in the past when it came to kind of attaching the chief product security officer title to the responsibility and the authority that they already had in these large multinational industrial control systems manufacturing companies.
Megan Samford:
However, in my role, I’d say that I am the first that I’m aware of that has kind of individually climbed the ranks just being within product security as its own domain, separate and unique from what chief information security officers have traditionally dealt with, which is the protection of enterprise networks. Chief product security officers more so deal with the guarantee of safety and security within a product that consumers are purchasing. So it’s that promise, it’s the integrity behind the quality of a product which is integral to a company’s overall reputation and quality with what they’re selling and offering to the market. So it really can impact your bottom line and your reputation if it’s not something that you don’t take seriously and if you don’t do too well.
Bryson Bort:
So part of your path that led here also included a role in government as well, correct?
Megan Samford:
It did. Yes. So I worked for the Commonwealth of Virginia. I served two governors. I served Governor Tim Kaine, as well as Governor Bob McDonnell and their offices of Homeland Security. And I worked for them in a role where I led critical infrastructure protection. So I believe my official title was Critical Infrastructure Protection Coordinator for the Governor’s Office of Virginia. And I also did a little bit of work previously in that office around interoperable communications.
Megan Samford:
But in terms of the critical infrastructure protection role, I absolutely loved it. It’s where I really got my foundations in security risk management. I met tons of people in the private sector. I got to go to some really unique, critical infrastructures around the Commonwealth and work with the Department of Homeland Security kind of in the early days as the department was standing up. We’re talking like circa 2005 through 2010, ’11 timeframe.
Megan Samford:
So super interesting programs, site assessments, lots of work around threat modeling of the critical infrastructures going on, and then of course the work that I came to love, supporting critical infrastructure protection from the emergency management standpoint. So more talking about the interrelation with FEMA and the preparedness cycle and resiliency of the assets, as well as how do the private sector and governments work together when something bad happens? If a natural disaster happens or a terrorist event, what does that coordinated response look like? So again, I can’t tell you how lucky I feel to have some of the early roots of my career stem back to that opportunity.
Bryson Bort:
What do you see as the top cybersecurity challenge in American today as it relates to our critical infrastructure and product manufacturing?
Megan Samford:
Sure. Great question. And I hate to sound pretty lame on the response to this, but, Bryson, I think it’s the same challenge that we’ve had for the past 10,000 years when civilizations have done security. I mean, it’s a general lack of understanding. It’s a lack of calibration against the risk, and it’s the lack of repeatability of process when it comes to security risk management. It’s an old adage that I always say, but we’ve done security the same way pretty much forever, and that in civilizations, we identify those assets, those things that are most significant and important to us as we operate and live together and operate governments and civilizations, and we tend to set security objectives around the protection of those assets.
Megan Samford:
We build up really high fences and walls and protective measures against them, and then we completely ignore likelihood of an event happening. And we kind of see this old adage play out again and again and again. And that, I think, is even more complex in today’s world where 85% of the risk, 85% of critical infrastructures are owned and operated by the private sector.
Megan Samford:
And so, on top of setting security objectives as a civilization, which the government plays a huge role in doing that, obviously, you have the private sector balancing those risks and trying to buy down that risk on top of traditional business risks and acceptance as well. But at the very heart of it, if you wanted to kind of say, “Well, that’s a very complex and loaded response that you just gave, where would you tell someone to start?” I would say that we can’t have a conversation around risk management until we can have a productive conversation around asset management.
Bryson Bort:
Asset management – knowing what you have and controlling the configuration or state of those devices.
Megan Samford:
And that’s where I see people still struggling with what are the top most significant critical infrastructures in the country? What did their supply chains look like, both inbound and outbound? What are the critical components when you decompose the list of assets across the country? What are the critical control points within those assets and how do we buy down risk to them? It’s the same thing and it’s the same cycle that we see over and over because the problem is so complex.
Bryson Bort:
So what are you doing in your role to tackle that challenge and have that conversation leading to that understanding?
Megan Samford:
Sure. So I think it depends on the customer and the audience. So if I’m talking to an internal customer within the company that I work for, I’m talking to them about understanding the threat landscape for the products that we’re making, right? What type of attacks the product would be most susceptible to. And then again, we’re talking about the whole quality issue around secure development life cycle, which is how do we have an accurate and repeatable way of testing the product for potential vulnerabilities and then remediating them prior to the product shipping, as well as the security features that we want to bake into the product to ensure that we’re positioning the product to withstand the potential threats, both known and potentially unknown, 5, 10, 15, 20 years out the entire life cycle of the product. How are we positioning that product to be the most safe, secure product that we can possibly make with the knowledge that we have today?
Megan Samford:
So that’s answering that question part one from my internal customer, which are the product line managers and the folks making the product inside the company, as well as kind of part two of the question, which is potentially my external customer, which could be an asset owner. It could be someone that works to protect the asset, a security manager. It could be a CSO of an organization. It could be a government. Basically any type of end user that would encounter the product or downstream benefit from the product, they’re all my external customers.
Megan Samford:
And I would say that I would talk to them about how to securely utilize the product and work to ensure that if they’re going through an integrator or a third party that maybe they’re purchasing the product from if they don’t purchase it directly or deploy it directly from us, that the product is configured and deployed in a secure way. And that throughout their system, they have proper network segmentation and they have visibility into the assets that they had. They’re performing patching and they’re able to detect if a threat has entered their environment. And importantly, if they do believe that something has occurred in their environment, they clearly know how to reach back to us to notify us to get them started on investigating a potential threat.
Bryson Bort:
So you mentioned product managers. Who are the key stakeholders in securing product manufacturing? Can you give some examples?
Megan Samford:
Sure. And it’s so funny because I draw a lot of parallels from my past life in government that all emergencies begin local and all emergencies end local. And when we talk about product management and product development, the experience of the developer, the person hands on keyboard making the product, everything has to be centered around simplifying and making security requirements and testing easier for them. Because at the end of the day, that’s really the root cause of where problems can stem from. And if we make that experience better for them, we’re more likely to produce a more secure and safe product in a repeatable way, if that makes sense.
Megan Samford:
And we called this a shifting left in the secure development life cycle, but we know that by having the right security products and requirements baked in from the get-go, as well as doing things like threat modeling and static analysis scanning earlier in the development of the product ensures that we’re cycling and we’re finding the issues earlier and we’re fixing them earlier.
Megan Samford:
That’s one example. And again, I know that I tend to center a lot of things around the developer, but product management helps define our requirements for the product. They help analyze the market. They help analyze nuances of different market segments where customers in the water and wastewater sector may be concerned about compliance to this particular standard. Others in power environments may be more concerned with NERC CIP 62443. It tends to be the horizontal standard by which much of the expectation and the bar is set within industry. But everyone essentially has a very important role to play. And when you’re in product security in a centralized office, it’s very important to assure that everyone’s role is respected and they’re allowed to play it.
Bryson Bort:
what kind of products are in your purview, reference critical infrastructure security, at Schneider Electric?
Megan Samford:
Sure. So I work within the energy management business. And so, we are more concerned with electrical distribution equipment. So think more along the lines of panels and protective relays, things that you would expect to see within substations, as well as traditional building automation. We make the electricity flow.
Bryson Bort:
“what is a substation” – the “grid” is made of generation (where it’s made), transmission (from plant to area), and distribution (final stage to individual customers) system. Substations transforms voltage.
Bryson Bort:
What is the relationship between the government and private sector? You spent time on both sides back when you were working for the Commonwealth of Virginia for two different governors, as well now as your private industry roles. Is that common? How does that inform your perspective?
Megan Samford:
Sure. So I don’t know that it’s uncommon. I really haven’t done an analysis to see how many chief product security officers within companies may have had past life or early experience in government. I can say that overall in security, you see a lot of people get their foundations and their early starts within either government federal agencies, some of the state level, and even the local level. So I think that that in and of itself is an interesting part of the question.
Megan Samford:
How does it inform my perspective? I would say that I understand the way that governments work and what their motivations are, as well as I have, I’d say, calibrated expectations on what the government is in a good position to provide me, as well as some challenges or constraints on what maybe they’re not able to provide me that I need to be able to find out for myself.
Megan Samford:
But overall, I’d say that if you take the example of Department of Homeland Security, the government advises on threats and overall risk, as well as indicates generally, at a high level, where they’re looking to see more investment, as well as preparedness, kind of a show of force and campaigns from the private sector if countries are responding to particular threats. So they’re really good at setting high level expectations of the private sector for behaviors they want to see more of, as well as behaviors that they want to see less of.
Megan Samford:
And this can play out in a really cool way, especially if you look at the preparedness cycle, as well as where the government can play a good role in supporting rapid restoration of critical infrastructures during natural disasters and what that process looks like, and how security personnel can hook into the response process with their local and state emergency operation centers.
Bryson Bort:
So what kind of intelligence and knowledge sharing is there?
Megan Samford:
There’s a ton of information sharing. I think that I’m always pretty surprised to see that when you look at government priorities or what the private sector claims they need to see more of, is many people will say, “Well, I need more information.” And it’s been my experience at least in that there’s tons of places to get information. You can almost be inundated with information coming from 20 different sources a day when you look at your inbox,
Megan Samford:
I’m personally a member of the ISA Global Cyber Security Alliance. We’re a newly formed group that’s working across the ICS security space with vendors, cybersecurity companies. We’ve even got some of the big four in there now. And that’s really interesting in that CSOs have had their own networking groups to collaborate with for probably the past 15 or 20 years. And it goes back to kind of the fact that industrial control systems and operational technology, OT security is now ramping up in a similar way. I always joke that we’re probably 15 or 20 years behind the CSOs, but you’re kind of seeing us try to rise to that challenge by forming our own groups to connect and collaborate with them.
Megan Samford:
So that’s one group where I’m talking to my peers with them. Of course there is the Department of Homeland Security Control Systems Working Group. I know that yourself, Bryson, and others are very connected, and with that, that’s another great resource. The Information Sharing and Analysis Centers, the ISACs are pretty well established. They’ve been around for a while. FBI InfraGard is another very good source.
Megan Samford:
At the end of it, I think people have access to information, they just don’t exactly know what to do with it from their vantage point within the world. And so if I go back to my lens, the way that I kind of view the world through my profile in my role, if I am say reading a report that suggests heightened threat activity in one part of the world or another, what action does that provoke within me? Do I then go back and look at products that are being supplied to a particular sector or a particular region of the world and say maybe I should proactively reach out to these customers, or what information, what have we done in the past? In other words, is intelligence kind of the neck that turns the head to help people make decisions?
Megan Samford:
And at the end of the day, if intelligence isn’t provided in an actionable way that is supplied to help inform decision-making, then otherwise, it can just kind of be noise. And I worry that it isn’t a lack of information, but it’s an individual’s ability to understand the intelligence process and understand what actions they may have at their disposal to be helpful and supportive given the information that they’ve been given.
Bryson Bort:
So what kind of responsibility do private companies have when it comes to security? And how does this tie into relationships with the policymakers?
Megan Samford:
Sure. So I think that the private sector has a responsibility to be involved in the conversation in the least. I think that the private sector understands its importance. I go back to the data point around 85% of critical infrastructure is owned and operated by the private sector. Of course, the private sector understands this. And I think that government has shown a willingness to be receptive to what the private sector is saying in terms of, again, you go back to the model of behaviors you want to see more of and behaviors you want to see less of. And DHS, to their credit I believe, has done a pretty good job in this respect, as well as Department of Energy.
Bryson Bort:
You’re spearheading an initiative called ICS for ICS, Incident Command Systems for Industrial Control Systems, blending your background in Homeland Security and emergency management when you worked in the Virginia state government and your experience with industrial control systems. Tell us some more about that.
Megan Samford:
Sure, and thank you for that. So this is an effort that I’m personally, I’m very passionate about because I think that globally, governments and the private sector and cyber first responders, I think that we can benefit from having a common language and a common framework that we can use together to respond to cyber incidents. I believe that every other type of responder in the world, whether you’re a firefighter or a police officer, or a medic, or whomever, in every other response I guess role, there is a framework by which you could literally be picked up an airlifted and dropped into another organization or locality or state or government really, and you would seemingly know how to fall in line with the common framework to respond alongside your peers.
Megan Samford:
But within cyber, it’s very schizophrenic, it’s very disparate, and it’s largely based on the needs of individual companies. And it doesn’t extend much beyond a company’s ability to maybe support its customers or its contracts, but even then, the customer’s incident response plan is going to look very different than the vendors. And then when you get into multi-vendor response or needing to support government, so say you had an attack on an electric grid, organizations are very limited in their ability to have modular frameworks, as well as frameworks at scale.
Megan Samford:
And the other pieces within incident command system, which really grew out of the California wildfires of the 1970s where many different fire fighting localities and groups were having to respond to the same fire, they realized that 10 codes from one locality were very different than 10 coded systems in other locality. And so, it’s really around being able to take the resources that you have at your disposal and being able to type them. And when I say type, generally, across the United States, a firefighter one in one locality in California has to go through the same requirements and training and experience as a firefighter one in Virginia. And so you have that level of assurance in what resources you think need to be brought to bear to respond to a common incident. They are alike in nature, and not only that, the resources are speaking a common language to be able to work together.
Megan Samford:
And what’s interesting is that if we would try to wrap our arms around the number of cyber responders we have within the United States or Great Britain or China or wherever in the world, I don’t think that any country has a good handle on the numbers that they have behind each type of talent, right? We don’t know how many malware reverse engineers we have. We don’t know how many incident coordinators we have on the cyber side, people that know the steps that are required to run a cyber incident from beginning to end. We don’t have a lot of the information that we need to begin that process. And so, Incident Command System for Industrial Control Systems is about copying and pasting what is a very tried and true framework from government that’s used for every other type of responder, and applying that to cyber.
Bryson Bort:
A common theme that we keep hearing in these podcasts is how it takes more than an individual, it takes more than an organization, it really takes a team. And that was something that was common during Reem’s podcast, who’s the CSO for Kuwaiti Oil. What’s something that we can do to make it a lot easier for that kind of collaboration to happen?
Megan Samford:
Sure. I think it goes back to the typing of assets and resources. Today, if we were to look at our workforce capability across cyber which would be needed to foster more collaboration to kind of be singing from the same sheet of music, you would want to know that type two malware reverse engineer has been trained to such and such level, they’ve had at least this baseline of experience. And we just don’t have that today. All we have are people’s resumes, which are great in and of themselves in understanding them as an individual unto themselves, but it’s not helpful in the context of enabling resources to work together as a team.
Megan Samford:
And teams are groups that can come together and rise above individual capabilities and form something much larger. And I would argue that in order for that to happen, you have to, again, it goes back to almost an inventory, you have to know the capabilities and the roles that are out there and what everyone may be able to bring to the table. And again, it’s not just counting the individual unto themselves, but it’s looking across an entire capability set to understand do we need more of this type of resource or are we okay here?
Megan Samford:
It’s just adequate workforce planning. And I would argue that in order to have teams that are able to work across companies or work across localities or states, or even countries being able to support other countries, that it comes down to knowing what you have and being able to utilize them in a more efficient way.
Bryson Bort:
If you could wave a magic non-internet connected wand, what is one thing you would change?
Megan Samford:
Again, gosh, I think it goes back to the cyber resources. I think that we have to view people as our assets in this space going forward and we have to wrap our arms around how big is our community, what resources and capabilities do we need more of. Because it’s an interest to companies, right? So that when events happen, they don’t find themselves quickly overwhelmed and unable to have surge capacity and support themselves and support their customers. But it’s also a national security interest.
Megan Samford:
I mean, if you look at why do disasters become catastrophes, it’s because supply chains break down. And supply chains break down when you’re not able to rapidly restore key critical infrastructures. And I just think in order for this entire thing to work, it’s going to come back to surge capacity and a better understanding of our resources and the enablement of people to work together across frameworks that it can cross over localities, states, and even nations.
Megan Samford:
So that’s what I would say I’d like more of, and I I’d like to build this up as more of a national capability. I mean, if you look at the fact that even today, if we started pumping out folks with college degrees and industrial control systems, cybersecurity, or even more niche areas of cyber incident response, we don’t have the time and we wouldn’t have enough people even if we started doing that in mass scale today. I think that we should look to rely upon existing proven technical schools and trades.
Megan Samford:
If you look at how the International Brotherhood of Electrical Workers works, the IBEW, there’s a technical training school, essentially, a few of them in every major city, as well as different geographic regions outside of cities. So you have local constructed, local driven and operated schools, technical schools, that I think those constructs could be better utilized to produce the cyber workforce that we need tomorrow. And it could be that we are willing to accept and embrace people without college degrees, but are people that are willing to learn cyber as a trade.
Bryson Bort:
You’ve waved your magic wand. Now, looking into the crystal ball for a five-year prediction, one good thing and one bad thing.
Megan Samford:
Sure. So I believe that we’re going to have greater visibility into operational technology environments, meaning, the same way that the chief information security officer may have a pretty good handle over the assets they have on the enterprise side of the house, they have that expectation now within the OT environment. And I believe that industry is going to deliver them asset identification and inventory and capabilities here within the next five years in a pretty stable way, in a meaningful and stable way. But the counter to that is that there is going to be a lot of money spent and not a lot of true risk built down is my fear.
Megan Samford:
So again, it goes back to contextualizing and understanding OT systems and where those architectural control points are on how you can truly buy down that risk, rather than trying to control everything and maybe spreading yourself too thin and spending a lot of time and resources.
Bryson Bort:
Any last things you’d like to add?
Megan Samford:
Just thanks a lot. It’s been a lot of fun. I hope that I answered the questions in a helpful way. And if people have any questions, that they feel comfortable reaching out to me. I’m a very open person. I’m obviously a chatty person. And so I don’t mind talking to folks and figuring out how I can be helpful and support them.
Bryson Bort:
Megan, thank you for joining us on the show.
Megan Samford:
Thank you.
