As we continue our exploration of plant floor cyber security, we need to develop strategies that work for our particular situation and seek guidance from a variety of outside resources. We can then turn discussion into meaningful action. This month’s concluding segment on plant floor cyber security is intended to provide some suggestions to facilitate this process.
It may help to begin by familiarizing ourselves with some new terms. The “actors” are those individuals who take part in creating a cyber security attack or breach. Actors may be nation state agents, organized criminals, lone wolf criminals, bored teenagers, disgruntled employees, or paid informants. Many of these are hackers. There may be “white hat hackers” who seek out “vulnerabilities” in systems for the purpose of protecting against a breach, referred to as ethical hacking. The “black hat hackers” will seek out and exploit vulnerabilities for malicious purposes. “Grey hats” may seek out the same vulnerabilities to sell them to the highest bidder or to claim bragging rights. Each of these groups may be “probing’” our systems and our people as they seek to find and exploit these vulnerabilities.
Not all actors are hackers however. Some may simply be individuals who have obtained legitimate access to systems and use it for malicious purposes. Often such people are not intending to do harm, but are duped by “phishing” or “spear-fishing” attacks where the bad actors seek accomplices who unknowingly provide information that may be used in a more damaging attack. Manufacturers are among the most frequently targeted by spear-fishing attacks. Bad guys go after the weakest link in the chain—the people. Criminals will follow your employees home to steel information that will allow them access to your systems.
It has been said that cyber security requires the integration of psychology and engineering, because understanding the motivation of the people trying to infiltrate our systems is critically important. Too often we make our plans assuming that we operate in an honest and ethical society. When it comes to cyber security, we can no longer assume that. Just because our plant sits in a valley of tranquility, those seeking to do us harm may be anywhere in the world where the values and mores are beyond our understanding. Motives may include terror, espionage (national, industrial, commercial, or private), hactivism (activism motivated by social, political, or ideological beliefs), financial gain, revenge, notoriety, or vandalism.
Our “attack surface” is the amount of area we expose to an actor. The more network connections we have, the more internet connected devices we have, the greater our attack surface and the more likely it is that there will be vulnerabilities. In years past, we could secure all of our assets, both physical and intellectual, by creating a security perimeter around our plant. Only people trusted to enter or leave that perimeter were permitted to do so. We could physically lay eyes on every person, and if we chose, on every document, that crossed the physical perimeter. We could send security personnel to patrol that perimeter and validate its integrity day or night. We could look for holes in the chain link fence or for fire doors left ajar. We could monitor everything with cameras and motion sensors if deemed necessary. Today if we have a network connection, our perimeter is much different. It is not without meaning in this regard that the term “perimeter” implies only two dimensions while the term “surface” implies three.
An “attack vector” is the means or path that an actor uses to gain access to his target. By finding a vulnerability on the attack surface, he exploits that as a means to perpetrate his attack. He might use a receptionist to obtain a legitimate username and password. He (or his robot) might dial thousands of mobile numbers until he finds a broadband modem attached to a piece of factory equipment. He might have an employee attach a cell phone to a network plug that was relocated to the outside of a control panel for safety reasons. He might piggyback on a VPN connection. He might infect a service technician’s USB drive, knowing that it will be plugged into a machine that can later become the attack vector.
Nature of solutions
As was stated earlier, this is not intended as a how-to guide. But in the process of developing this article, some general strategies emerged. Here are some of them.
Peter Holicki of Dow Chemical in an ARC Industry Forum keynote address affirmed that technology requires strategy, business alignment, and business ownership. Dow does not let companies that own the technology control them, DOW controls the technology. This is a tenet that I strongly support, especially as it pertains to the security of our intellectual property, our operations, our people, and our products. Manufacturers should have shop floor technology plans that align with their business, financial, marketing, HR, and security plans.
In that same session, Brigadier General (Ret) Gregory Touhill of the US Department of Homeland Security explained that cyber security is misunderstood as a technology issue for discussion in server rooms when in fact it is a risk management issue for discussion in classrooms, lunchrooms, and boardrooms. It is a matter of risk for everyone in our society.
Cyber security is a team sport. We need our plants to be safe, secure, and resilient. The first thing to do is to put it on the agenda, and keep talking about it until it permeates every part of the company. Help your employees with security not only in the office, but at home. Then discuss with your partners up and down your supply chain.
Cyber security needs to be raised to the level of safety in our plants. A safe work environment is a condition of doing business (a license) in today’s world. A cyber-secure environment should also be a requirement. As we are required to report lost time accidents to OSHA, we should be required to report cyber security incidents as the Germans are already doing. One CPG representative told me that they were treating cyber security like safety and like sexual harassment awareness, where every employee is required to attend training and retraining. We need to create a cyber security aware culture in our plants. This is probably one of the most important steps to be taken.
Realize the impossibility of protecting all of your information to the same level. Moltke the Elder taught that in warfare, he who defends everything defends nothing. Identify where the really important data is (maybe in the process control system, not the office) and apply more resources there.
Realize that you can’t harden everything. There are still tens of thousands of systems in the plants running unsecure-by-design systems such as Windows 98 and XP. These aren’t going away anytime soon. Think of a turtle. These soft structures can be surrounded by a hardened shell of hardware and software that monitors all of the assets and controls any information flow to or from them. While you cannot hope to keep software up to date on all of the connected devices, you can apply daily patches to the system comprising the shell to keep it as secure possible. This will require dedicated staff who understand both IT and control systems. And, this is not the long term solution. In parallel with this approach, we need to take a ‘secure by design’ approach for new systems.
Planning should be multi-dimensional including plans for protection, prevention, mitigation, response, and recovery. Our systems must be both safe and, when things to go wrong, resilient.
If you allow external connections, make everyone come through a common and closely managed access point. It is like having only one entrance to your plant.
Establish, communicate, and enforce strict policies regarding who can authorize the addition of ANY device onto a network or the addition of any communications access to a machine. Is your landlord or your building management department making connections that your process control or IT departments don’t know about? Is the cafeteria or the lab having their equipment monitored remotely? If so, chances are good that there are cross connections to your internal networks.
Establish, communicate, and enforce policies regarding visitors, especially service technicians bringing computer technology into your plants and attaching it to their equipment in your systems. Keep in mind that big corporate equipment suppliers may resist allowing you to scan their laptops or USB drives just as much as you may insist, resulting in a standoff while production is down. Plan and agree in advance.
Remove and prohibit vulnerable technologies unless you can prove your system keeps them secure. These would include DHCP, dial-up modems, broadband cellular modems, tablets, and smart phones. In municipal systems such as water and wastewater, the concept of bring your own device has emerged whereby plant operators use their own cell phone as an operator interface. What a vulnerability that makes!
You will need to know about every digital device in your plant and have up to date network and data flow diagrams. In 1999 you probably had these things in preparation for Y2k, but that inventory has long since gone out of date. When you complete this one, establish procedures to keep it current. Learn from other mistakes of Y2k.
Think about secure-by-design, but realize that every design will eventually be compromised. Security needs to be part of every design going forward.
Cyber security activities need to be both measured and tested. Have cyber security key performance indicators (KPIs) as part of your plant and corporate balanced scorecard.
Use industry and government standards and practices as part of your solution, but don’t mix up minimum recommended practices with what you really need to do.
Resources
There are a great many public and private resources available to help you get started on the journey of protecting your factory floor assets from malicious cyber activity. Presidential Policy Directive 21 issued February 12, 2013 addresses Critical Infrastructure Security and Resilience and clarified the roles and responsibilities of cabinet level departments with respect to physical and cyber security. The key areas of responsibility include overall coordination by Department of Homeland Security (DHS), national defense by the Department of Defense (DOD), enforcement by the Department of Justice (DOJ) and the FBI, and research and development of tools for improving security by the Department of Commerce (DOC). Other departments such as the NRC, FCC, FDA, GSA, etc. have specific responsibilities within their sectors. All of these departments have established teams who support cyber security efforts.
The FBI has cyber security squads, referred to as GeekSwats, in each of its 56 field offices that work within 16 identified segments including critical manufacturing and food & agriculture. The FBI has established a partnership with the private sector called InfraGard for the sharing of information and intelligence to prevent cyber crime. There are 80 chapters of InfraGard that meet across the United States with 350 of the Fortune 500 represented.
DHS operates the National Cyber Security and Communications Integration Center (NCCIC), the U.S. Computer Emergency Readiness Team (US-CERT), and the Industrial Control Systems Computer Emergency Response Team (ICS-CERT). Each of these agencies has extensive information, tools, and resources available on their websites with the ICS-CERT focusing specifically on the topic of factory floor security. They provide alerts, advisories, assessment, training, standards, conferences, and a host of tools, case studies, and best practices. Among interesting documents offered by ICS-CERT is one entitled Cybersecurity Questions for CEOs.
The National Institute of Standards and Technology (NIST) falls under the Department of Commerce. Last year NIST issued the document Framework for Improving Critical Infrastructure Cybersecurity to guide businesses in applying a systematic process for identifying, assessing, and managing cybersecurity risk. NIST operates 60 Manufacturing Extension Partnerships across the US that can make resources available to apply this framework, especially for smaller manufacturers.
Professional and trade organizations such as the International Society of Automation (ISA) provide tools, assessment, training, and certifications in cyber security. ISA focuses on factory floor systems, and has created a series of ANSI and ISA consensus standards on Security for Industrial Automation and Control Systems.There are 13 parts envisioned in the series, which has been under development for 13 years by groups of volunteers. Find information at isa99.isa.org .
Industries closely related to manufacturing, especially those that are being driven to implement protective measures by regulation, have developed much useful information. For example, the North American Electric Reliability Corporation (NERC) has developed 81 Critical Infrastructure Protection Standards, the so-called NERC-CIP Standards. Unlike ISA standards that are copyrighted and available for a fee, the NERC-CIP’s are available at no charge at www.nerc.com.
Educational institutions are gearing up to prepare cyber security professionals. Gary Beach in an article in the Wall Street Journal made the claim that lack of talent is America’s most challenging cybersecurity challenge. One step in addressing this was a $23.2 million Department of Labor grant to establish the National Consortium for Mission Critical Operations (NCMCO), a group of community colleges partnering to create programs and curriculum to address the needs for a skilled workforce that can anticipate, prevent, mitigate and respond to cyber security breaches. Strategies being used in this effort are well aligned with the strategies outlined in The Manufacturing Workforce Development Playbook available at www.packworld.com/workforce. These strategies have also been extensively used to build capabilities for industrial maintenance and mechatronics. Universities are also engaging in the cyber fight with, for example, Carnegie Mellon (CMU) establishing the CyLab partnership with industry and the CERT Division, which is part of the Software Engineering Institute at CMU.
Communications and software companies with a vested interest in cybersecurity collect information and provide reports, training, and information for the public. Verizon and McAfee publish annual reports such as the Verizon 2015 Data Breach Investigations Report or the McAfee Labs quarterly Threats Report.
Hardware, software, and service suppliers have made a great deal of information on cyber security available. These include white papers and blogs such as those offered by Tofino Security. New security hardened products are being offered by vendors. Despite the ongoing shakeout of PLC platforms, even a completely new security and electro-magnetic pulse hardened PLC platform has been introduced by Bedrock Automation. Five years ago, I think no one would have imagined a new entry into the PLC marketplace, but security is deemed to be that big of a deal that entrepreneurs thought it worthwhile.
In preparing this article, I made use of Chantal Polsonetti’s LinkedIn discussion group Industrial Internet of Things where I posted the question, “Is anyone concerned about the security aspects of having our factories connected as part of the IIoT?” This resulted in a number of thoughtful responses, as have many of the other threads in the group. You may join this group on LinkedIn.
These are by no means all of the resources that are available or that will become available as the battle continues. The FDA has taken limited steps in its areas of control, focusing on medical devices and healthcare facilities. Other food and pharmaceutical entities need direction just as do the larger CPG and hybrid industries and their equipment suppliers. It would be worthwhile for manufacturers in these spaces to encourage their industry associations to help them wade through the vast quantities of information that are available and to help develop guidelines for their particular segments. In the long run, this could arguably obviate the need for forced government regulation and produce superior results. NAM prefers a voluntary system, while others point to the safety success of the nuclear industry as an example of forced regulation that works. In my experience, hybrid manufacturers have not heretofore shown enthusiasm for participating in the development of standards, bringing them to a timely conclusion or adopting them in a timely fashion. But in this case, it seems to me, there are only three viable choices: 1) everyone takes on this gigantic task on their own; 2) manufacturers work together to create and adopt robust standards; or eventually 3) government attention will turn to these additional industry segments and force regulation upon them.
It’s a new world
It is a new world, in which whether we like it or not, cyber security is a real threat. It’s not just the financial sector’s problem or just a problem for nuclear plants, pipelines, and defense contractors—they are the tip of the iceberg and the areas that need to be addressed first. A broader manufacturing industry undergirds our society and our economy and cannot be allowed to become the soft underbelly to be attacked by cyber criminals. And within our manufacturing enterprises, our factory control systems may hold the most confidential of our confidential information. They are critical to process, people, and product safety.
Our factory systems contain the widest variety of digital systems, in age, source, and function, making it the hardest part of our infrastructure to secure. We should not put our businesses at risk for loss of confidential information, loss of product integrity, loss of availability, or loss from civil claims if our systems or employees become the vector used to attack a customer or supplier. We need to discuss this at the highest levels of our companies, plan for it, fund it, and create a security culture that encircles the threat.
As CPGs and hybrid manufacturers, we should band together through appropriate associations to assure that we aren’t left behind as government focuses on process and discrete “critical” industries, and to obviate the eventual transfer of regulations created for those industries onto us. We need to support education and employee development and cross-pollination with other industries. And we need to plan carefully and act prudently as the Industrial Internet of Things comes upon us, to be sure that we balance security, productivity, risk, and revenue.
