INSIGHT

Detecting humans, part I – The 3 most promising industrial applications

Object detection is a rising method in the field of computer vision whereby different objects – such as humans, industrial machinery, or components – are detected from still and video images (which are basically just still images one after another). The past years have marked huge advances in object detection, largely thanks to the development of deep learning techniques.

Detecting people and their behaviour in images is now one of the most promising use cases of object detection. Human detection in turn has vast applications in areas such as safety, people flow, and security surveillance. 

How can we ensure our employees are using required safety equipment? How can we optimise people flow in huge production sites? How can we detect unauthorised people in high-security areas?

In this blog series, we will provide an overview of the most promising industrial applications of human detection, the features that constitute a real-world, viable human detection system, and the most exciting technological breakthroughs in the field.

First, we need to distinguish between human detection and recognition. Detection aims  to identify humans in given images on an abstract level – i.e. without connecting these detections to any personally identifiable information. Recognition on the other hand aims to identify humans and understand who they are. While recognition is a groundbreaking method with various use cases from smart device access control to advanced security surveillance, it brings about all sorts of privacy issues (worth another article altogether). 

Luckily, in many business cases, detecting humans is enough – and much lighter on privacy and regulatory aspects. Some of the most promising applications of human detection already in use fall under three categories:

Personal protective equipment surveillance

Today, workplace safety is closely monitored and a key concern, especially in heavy industries and construction. Personal protective equipment (PPE) such as helmets, protective shoes and high-visibility clothes are required for people working at construction sites and different production facilities. For long, PPE compliance has been monitored by either physical inspections to sites or through CCTV cameras – both of which can be very laborious. 

PPE surveillance is a textbook use case for human detection  – we have images of people who we want to detect, and we want to understand if they are wearing something. Combining object detection, object tracking and other methods, computer vision based PPE surveillance systems can, for example, count the number of humans on a video image and indicate whether they are wearing required PPE or not. This can be used to automate safety compliance reporting from sites, lessening the requirement for physical inspection visits. In addition, PPE compliance analytics can be derived from these kinds of features.

Human detection can also be used to generate real-time alerts of safety compliance issues. Responsible personnel for safety can be notified about these alerts, helping them to react to potential safety threats as quickly as possible, but without the need for constant manual surveillance.

Once an algorithm has been implemented for one purpose, it can easily be updated and re-trained to detect other objects as well. For example, you can first implement a helmet surveillance system, and later expand it to detect protective glasses, gloves, or something completely different, like face masks to meet new COVID-19 regulations.

People and customer flow analysis

While human detection for PPE surveillance focuses on detecting individuals and understanding their appearance, people flow analysis focuses more on detecting masses of humans and understanding their size or movement patterns. This has various use cases such as optimising and analysing workflows in production facilities or understanding how people move inside public spaces such as stores or shopping malls.

Consider inspection tours that are conducted at many factories: it is essential to inspect certain components periodically to ensure that they are operating as they should. Human detection can be used to monitor employee movements inside these facilities, keeping track that all relevant locations are inspected. Alerts can be created if a certain location is not visited on the inspection tour for a given period.

In a more commercial setting, layout design is key for optimising both customer experience and revenues from physical stores, and automated people flow analysis can help understand customer behaviour in more depth. Are customers often drawn to certain product areas? How do customer movements change during different times of the day? How does customer behaviour differ between different demographic groups? Human detection can also be used to construct 2D maps of people movement within the spaces, in addition to displaying results in the camera feed.

Coupling human detection with features like action detection (are people standing, walking, running, etc.), we can determine how people move in different physical spaces. Take a supermarket for example – with human detection, we can detect when long queues start to form at counters, and create an automatic alert of that to the personnel. With similar logic, we can detect if a certain mall entrance is always crowded at certain times of the day, or if there are apparent bottlenecks in employee movement at an assembly line in a production facility.

Security surveillance

Security surveillance is perhaps the most intuitive and common use case for human detection, as cameras and video are plentiful. From regular offices to stores and high-security government facilities, video surveillance is widely used to detect people in unauthorized areas. 

Thanks to recent technological breakthroughs, human detection can be enriched with a variety of features: for instance, it can be configured that certain areas are unauthorized only at certain times. Or, the detection system can be coupled with access control information – understanding the difference between authorized and unauthorized entries to a certain space. Furthermore, many visual characteristics can be inferred from the detected people to understand the difference between employees and external (unauthorized) people – in many production facilities, areas with sensitive information or higher safety requirements may be limited only to authorized personnel.

All of these examples use human detection algorithms coupled with many other features – in most cases, it is simply not enough to just detect humans in images, but other insights are needed as well. In part II of this blog series, we will dive deeper into the features and technical requirements that constitute these types of human detection systems.

Kalle Kyyrö

COO

kalle@smartbi.fi

+358 50 304 7409