Assessing risk of integrating AI into your equipment
In this semi-occasional column we explore the risks associated with integrating generative AI into your equipment and how to mitigate them. Included is a link to a database of all known risks associated with AI.
Welcome back to a semi-occasional column on incorporating AI into your packaging and processing equipment. In my previous two columns I wrote about why it’s important to explore the opportunity to build AI into packaging equipment, making it potentially much easier for operators to operate and troubleshoot. Natural language communication with your machine, in multiple languages, can become a game changer for your customers, given declines in workforce quality and availability. I also walked through a series of small language models that can be embedded in a PC right on your machine, such as the HMI or even a small Raspberry Pi.
With generative AI embedded in your machine, what could possibly go wrong? When it comes to generative AI, two main risks come to mind, starting with the greatest risk of all.
Causing physical harm to an operator or technician
The only way for generative AI to cause physical harm is if it’s in charge of the machine’s operation, i.e., it’s part of the machine control architecture, and it gives an “order” to the machine to operate in a way that is unsafe (or counter-productive).
The solution is to simply separate your generative AI architecture from your controls architecture. One approach is to completely firewall the design so that the generative AI component has zero visibility into the controls architecture from a read or write perspective.
But such a draconian separation denies the opportunity to access real-time machine data to diagnose problems. Ideally you’d architect your generative AI as a “read-only” platform so that it could read machine states and production data being collected, but have zero ability to change those states or change any machine logic. In a future column I’ll tick through some possibilities of how generative AI might be able to help if it had read-only access to real-time data.
Bottom line, with zero access to the PLC code that runs the machine, it’s not possible for the generative AI to cause the machine’s operation to change by itself.
The only other way generative AI could cause physical harm is if it just plain gives bad advice, which leads to the other major risk factor with incorporating generative AI into your equipment.
Hallucinations and bad advice
You’ve gone through the effort of building a small language model into your machine, you’ve trained it on all your support content, and you’re ready to test it. You type in (or speak) a question, and the answer you get back is flat out incorrect, suggesting an action to the machine that is counter-productive, or worse, unsafe.
Unfortunately, this is clear and present danger with language models as they exist today, in the year 2024. But before you throw the baby out with the bathwater and forsake generative AI for all eternity, take a breath for a moment.
Let’s start by breaking down the phenomenon known as “hallucinations”, when a language model returns an answer that is clearly wrong. You’ve probably heard about this and you may have even experienced it yourself.
What causes AI hallucinations? Many reasons, but one of the big culprits boils down to data and training issues.
When language models generate an answer, they are calculating what is the most likely to be the right answer (and really, the right words) based on the information that was fed into the model during its training. However what is likely to be the right answer is not the same as the actual right answer.
Dave Newcorn, guest author of this column, owing to the flimsy confluence of having dabbled in arcane-and-useless programming languages 40 years ago and having covered the packaging machinery industry for 30+ years. Day job: PMMI Media Group president.To illustrate my point, let’s say you were to ask a popular generative AI model what is Dave Newcorn’s background that qualifies him to write a column on AI in a publication for packaging and processing machinery builders. It’s conceivable the model might say that Dave Newcorn came from a multi-generation packaging equipment family and also obtained a journalism degree from Northwestern University (which is in the greater Chicago area, where I currently live). Both sound entirely plausible, but in fact none of those things are true. (Though I did partake in NU’s summer journalism program for high school students in the 80’s! And though I was neither born nor raised here, I’ve called Chicago my home since the late 1980s.) The point is, depending on what information exists about Dave Newcorn online, these statements are likely to be true, the way that language models are designed and operate.
The big issue with off-the-shelf LLMs like ChatGPT is that while they are trained on vast amounts of textual content, that content is produced by humans, and humans are known for creating content that has gaps, biases and inaccuracies.
However, when training a small language model on your own content, you have 100% control over what information the model is ingesting. (I’ll write a future column on the nitty gritty of language model training.) In other words, with a small language model, you greatly sidestep much (but not all) of the potential for hallucinations related to training on inaccurate or incomplete information on your machine (as long as what you’re giving the language model is accurate).
A second huge issue with receiving inaccurate information (whether a hallucination or not) from a language model is the quality of the prompt or input. Remember the old “Garbage in, garbage out?” Well, that’s true here too. If an operator or technician types in something that’s excessively vague (“can’t run machine”) or even non-sensical, then the AI will be forced to guess not just what the answer is, but what is the actual question. (By the way, we see this in search queries for the PACK EXPO exhibitor search as well as PMMI ProSource. You wouldn’t believe how simplistic some of the searches are. No AI in the world can guess what some of these people are asking for.)
Nobody likes AI models that hallucinate, and the AI companies are working hard on this problem. (AI researchers are also testing new ways to address the problem including “challenger/verifier” models where AI’s challenge the answers produced by other AI’s.) There is support for the belief that AI hallucinations are declining as newer and more capable AI models are being released.
Another way to reduce the risk of generative AI providing incorrect information is to take Apple’s approach, which is to lock down the inputs and the outputs. Apple plans to limit AI to specific tasks and areas where it looks for certain inputs and conditions, and provides a set of limited and constrained output choices. (Users will still be able to type in free-form queries but only in a window or construct that is clearly labeled as a third-party service, such as via OpenAI.)
By the way, Apple is also said to be baking into its AI certain core instructions such as “Do not hallucinate”, “Do not make up factual information” and even, reportedly, “Do not generate content that is religious, political, harmful, violent, sexual, filthy, or in any way negative, sad or provocative.” Wild stuff.
Testing, testing, testing
Although there are risks, the payoff of a building machine that allows natural language communication with operators is too great to ignore. Consider a “crawl, walk, run” approach to testing this capability, first internally only (and extensively).
Then consider select trials with trusted customers, with the output clearly labeled as experimental. Consider adding buttons that say “accurate” and “inaccurate”, logging the output into a file for subsequent analysis to see where it gets it right and wrong.
We are truly in the early days and advances in Generative AI are hurtling down the pike at enormous speeds. Things are changing daily, and unless you are actively experimenting with these technologies, you’re at risk of falling behind. Perhaps the risk you have to assess first is whether the risk of falling behind outweighs the risks inherent with those pursuing the first-mover advantage with a new technology like generative AI.
OEM Magazine is pleased to inaugurate this semi-occasional column tracking the rapid advances in AI and how packaging and processing machine builders can leverage them to build next-generation equipment. Reach out to Dave at [email protected] and let him know what you think or what you’re working on when it comes to AI. Alternatively swing by the PMMI booth at PACK EXPO and let Dave know what you’re working on.
Comprehensive framework for evaluating AI risk
This month, a pair of researchers at MIT published a thoughtfully conceived “meta” framework for all the different ways AI can go wrong. To compile this meta framework, the researchers studied 43 other frameworks and identified nearly 800 specific risks. (The researchers pledge to regularly update this framework, or database, as they’re calling it.) While there are some scary things listed, most of them come into play with general-purpose AIs that we experience as consumers. Few of the risks seem to apply with very specialized, controlled applications, such as embedding in packaging and processing machinery. Nonetheless, from a risk assessment perspective, it’s absolutely worth studying this framework, which is presented as a comprehensive, multi-tab spreadsheet.
One of the tabs in the spreadsheet (labeled “Domain Taxonomy statistics”) distills the nearly 800 risks into a series of seven broad categories and a handful of subcategories per broad category. I have reproduced the summary below, but a quick perusal reveals that most of these risk areas pertain to broad societal and political impacts (unnerving nonetheless, but beyond the scope of this column). Very few of them are applicable to packaging or processing machine applications. I’ve included the link to the full database above so that you can do your own evaluation and draw your own conclusions.
Source: AI researchers at MIT. Full database of AI risks can be accessed at https://airisk.mit.edu
Source: AI researchers at MIT. Full database of AI risks can be accessed at https://airisk.mit.edu
