Using the Power of AI to Make Factories Safer

As industrial automation increases, safety becomes a greater challenge and top priority for enterprises. Safety encompasses multiple aspects: System…

As industrial automation increases, safety becomes a greater challenge and top priority for enterprises.

Safety encompasses multiple aspects:

System safety: A rational pursuit of acceptable mishap risk within a systems perspective.

Safety of the intended functionality: The absence of unreasonable risk due to hazards resulting from functional insufficiencies of the intended functionality or by reasonably foreseeable misuse by persons.

Functional safety: Part of the overall safety that depends on the correct functioning of the safety-related systems and external risk reduction facilities.

Worker safety: Providing a safe working environment for employees by incorporating safe equipment and safe procedures at the workplace.

And other, more general types of safety.

The same technological solution that’s driving automation can be used to also address safety: artificial intelligence.

AI-powered stationary outside-in safety platforms, which monitor activity across many distributed machines or robots, can predictively and proactively orchestrate consistent safety policies. Machines and robots equipped with inside-out reactive safety can detect any specific interaction within their workspace and take appropriate measures.

Figure 1. Inside-out and outside-in safety

IEC 61508 is an international standard published by the International Electrotechnical Commission (IEC). It consists of methods to apply, design, deploy, and maintain safety-related systems.

Using IEC 61508 general architecture, AI can either be used to implement the safety function or to assist functional safety to provide additional risk reduction measures.

Figure 2. Reactive and proactive safety

The use of AI in safety-related systems is being analyzed in upcoming international standardization activities such as ISO/IEC TR 5469 and ISO/IEC AWI TS 22440 being developed by ISO/IEC joint working group JWG 4.

The examples in Table 1 describe some typical safety use cases that can benefit from AI.

Use caseSurveilled machinesDetecting position and movement of humans toward a machine safety zone.conveyor belt, high-speed door, collaborative robot, autonomous mobile robot, forkliftDetecting machines moving in the direction of a forbidden area or in the direction of other machines; machines moving at excessive speed or excessive strength.autonomous mobile robot, forklift Detecting defects or material that could cause safety hazards based on norms such as ANSI/RIA R15.08, and so on.autonomous mobile robot, train, airplane Detecting and classifying worker or object poses and movement.Factory, warehouses Table 1. Use cases and machines that can benefit from AI proactive safety functions

The following examples of responses to those proactive safety functions are ordered by increasing level of intervention:

Updating statistics to later optimize worker-machine or machine-machine interactions, change the factory floor plan, or improve worker training.

Informing a safety supervisor.

Generating visible and audible alerts to workers, through machine local alerts or personal devices.

Slowing down or stopping a machine (acting as a speed or force limiter).

Changing the position or trajectory of the machine.

Preventing the start of the equipment mission.

Activating hazard protection devices.

NVIDIA IGX Orin combines industrial-grade hardware with enterprise-level software and support to implement both inside-out reactive and outside-in proactive safety in the same platform:

AI safety framework

Framework to provide proactive safety through AI.

Safety Extension Package (SEP)

Safety services for fault avoidance, detection, and control leveraging hardware safety extensions

Edge Safety Link, a safety communication protocol.

Hardware safety extensions:

Hundreds of diagnostic mechanisms implemented in NVIDIA Orin SoC elements (CPU, GPU, accelerators, interfaces, ECC, and so on), In-System-Test, clock, temperature and voltage monitors, and so on.

Functional Safety Island (FSI): An independent, redundant set of processors with onboard memory that is used as a monitoring and safety processor working beside other SoC engines.

Safety MCU: A third-party, safety-certified microcontroller, collaborating with FSI and Orin SoC, and controlling board-level voltage and temperature monitors.

Safety documentation

Set of documents (safety application note, safety manual, and FMEDA) to support customer safety cases.

Safety in action example

Here’s an example of proactive and reactive safety in action, implemented in collaboration with FORT Robotics and Protex.AI.

Figure 3. Proactive and reactive safety in action

In the example, an HD IP camera connected with Ethernet to NVIDIA IGX running Protex AI stack detects a worker moving toward an industrial mobile machine.

Figure 4. Overall setting with camera and safety zones

If a worker crosses the warning safety zone, IGX commands switch on a visible alarm (proactive safety).

If the worker continues or crosses the stop safety zone, the FORT safety stack on IGX commands (WiFi) the FORT endpoint (embedded in the machine) to activate the emergency stop (reactive safety).

Figure 5. Setting safety zones in Protex.AI user interface

The Protex edge computer vision app uses NVIDIA DeepStream 6.2 and the Protex model has been trained with NVIDIA TAO Toolkit and quantized with NVIDIA TensorRT.

Figure 6. Proactive safety stack and the camera-person-zone-event chain