How a startup uses AI to put worker safety first

Unpredictable spikes and drops in demand combined with chronic supply chain and labor shortages are accelerating the pace of digital transformation in manufacturing, starting with worker safety. Forty-eight percent of manufacturers say their progress on digital transformation initiatives has accelerated so much that it’s years ahead of what was originally anticipated, according to a KPMG study. Keeping workers safe and connected is the primary goal of most digital transformation and hiring plans, with on-site distancing & workplace safety listed as the two highest priorities.

Everguard.ai, a startup based in Irvine, California, combines AI, computer vision, and sensor fusion to reduce the risk of injuries and accidents by preventing them before they happen. The company’s SENTRI360 platform proves effective in preventing workplace injuries and operational downtimes at several steel-heavy manufacturing companies, including Zekelman Industries and SeAH Besteel.

Worker safety is the future of manufacturing

From redesigning shop floors, to meeting social distancing guidelines, and doubling their investment in training and development, worker safety now dominates manufacturing — even more so due to the pandemic. Frontline workers saved many manufacturing companies from going out of business by applying their expertise and insights in real-time, enabling entire plants to pivot and produce new products at record speed. Continued trade tensions, tariffs, and supplier shortages put more pressure on manufacturers to reshore production and have worker safety programs in place now. As manufacturing returns to the U.S., AI and computer vision are stepping up to improve worker safety.

Frontline workers reconfigured entire production lines and machines, and also learned new work instructions to produce much-needed Personal Protective Equipment (PPE), medical supplies, devices, and products — in some cases overnight. What began as an emergency response to the world’s PPE and medical products’ shortage quickly turned into a validating event that proved that protecting and connecting workers is the future of manufacturing. Gartner says that manufacturers who prioritize worker safety, training, and development create a strong foundation for the future of a connected workforce. Healthcare-optimized devices, wristbands, and lone worker protection are on the slope of enlightenment, according to Gartner’s research.

Above: Worker collaboration, safety, and security technologies are delivering results across distribution and manufacturing enterprises today, driven by the combination of advances in supervised and unsupervised machine learning algorithms and computer vision.

Image Credit: Gartner Hype Cycle for Frontline Worker Technologies, 2021.

Improving working safety with AI and computer vision  

Computer vision has progressed from an experimental technology to one that can interpret patterns in images and classify them using machine learning algorithms to scale. Advances in deep learning and neural networks enable computer vision uses to increase for enterprises, improving worker safety in the process. Computer vision techniques to reduce worker injuries and improve in-plant safety are based on unsupervised machine learning algorithms that excel at identifying patterns and anomalies in images. Computer vision platforms, including Everguard’s SENTRI360, rely on convolutional neural networks to categorize images and industrial workflows at scale.

The quality of the datasets used to train supervised and unsupervised machine learning algorithms determines their accuracy. Convolutional neural networks also require large amounts of data to improve their precision in predicting events, fine-tuned through iterative cycles of machine learning models. Each iteration of a machine learning model then extracts specific attributes of an image and, over time, classifies attributes. Everguard uses real-time video feeds from production plants combined with Industrial Internet of Things (IIoT) sensor data to create the data that convolutional neural networks needed to improve their predictive accuracy of potential accidents and safety incidents. The greater the volume and quality of data provided to machine learning models, the higher the predictive accuracy and the more effective prescriptive analytics will become.

The SENTRI360 platform is differentiated from pure computer vision-based systems because it relies on a proprietary sensor fusion approach. Sensor fusion leverages various sensors fused together at the edge to help contextualize the workers’ harsh environment more completely than any single-sensor approach ever could.

Above: AI and computer vision-based platforms are evolving from providing baseline descriptive analytics that are often lagging indicators of safety events to more predictive and prescriptive analytics that prove successful in averting accidents and injury. Combining AI and computer vision is to control and eliminate workplace injury risks.

Everguard’s SENTRI360 platform relies on these techniques to generate leading indicators and produce real-time prescriptive metrics and interventions to protect workers’ safety. Their goal is to provide real-time, predictive analytics-based alerts to reduce worker injury risks and improve shop floor productivity.

Advanced analytics techniques have been used for years to provide descriptive, after-the-fact metrics on worker safety. What Everguard.ai’s says makes its approach unique is providing actionable alerts before a potential event occurs, combining AI and sensor fusion to provide a more practical approach to averting accidents and injury. Like many companies whose core technology is AI-based predictive analytics and outcomes, Everguard.ai relies on synthetic data and simulations of potential accidents and injuries to fine-tune predictive and prescriptive analytics.

Manufacturers are redesigning shop floors, re-routing workflows, and modifying work cells to ensure worker safety. Protecting their workers from COVID-19 and assuring every plant is safe is the highest priority they’re pursuing today. Computer vision identifies which workers have PPE equipment in compliance with OSHA guidelines. Real-time locating systems (RTLS) identify a worker, provided that they’ve opted to participate. Everguard.ai’s Sensor Fusion technology merges computer vision and RTLS to provide a real-time safety assessment of a given plant and provides alerts back to employees via wearable devices. Audible, haptic, LED, and text-based messaging keep workers informed of potential risks or dangerous conditions in real time. The wearable was designed by Everguard and can also sense biometrics, such as dehydration.

Above: Sensor fusion combines computer vision and Real-Time Locating Systems (RTLS) to produce alerts of potentially dangerous conditions for workers. The above picture is from the SeAH Besteel steel mill in South Korea. Everguard.ai’s computer vision models have proven effective in detecting different human postures and looking for unsafe activity, including repetitive motion, unsafe load pick-up posture, improper hand on load handling, and worker orientation relative to heavy equipment (if worker facing oncoming crane or vehicle load)

Prioritizing  workers’ privacy

Everguard’s a CEO, Sandeep Pandya shared details about workers’ privacy, given the massive amount of data it captures and analyzes at client sites. “The most important thing is to give shop floor workers and their leaders [the] complete visibility into how the data collected is used. Our implementation teams work with them and provide complete access to our systems, how data is anonymized for specific tasks, and how we are careful to protect each workers’ identity,” Sandeep said.

“All effective change management starts on the shop floor. Our goal is to be transparent with the workers there because their choice to own the system will mean the difference in it succeeding or not,” he said.

Sandeep told VentureBeat that “workers can choose to wear the device that can alert them of a safety issue to any data being captured or not. We advise clients to have the systems be 100% opt-in to improve adoption rates and protect workers’ privacy.”

Note that some workplace safety platforms, including Everguard, don’t disclose how they trained their computer vision algorithms or whether they retain any recordings of workers. In lieu of this information, how — or whether — these companies ensure data remains anonymous is an open question, as is whether they require their customers to alert employees that their movements are being analyzed.

Forcing someone to wear a sensor for biometric data is a sure way to lose valuable employees. Production workers in its client’s plants are so valuable that Sandeep said, “our clients are doing all they can to retain them. Talented manufacturing workers are in high demand and nearly impossible to replace today.” Instead of using the data to rank employees’ productivity or threatening employees to produce more or be let go, Everguard says its system’s primary function is accident prevention, and the data is used for coaching workers, so they stay safe.