Door Safety Function for Elevators Using Video Analysis

This paper was presented at    Berlin 2018, the International Congress on Vertical Transportation Technologies, and first published in IAEE book Elevator Technology 22, edited by A. Lustig. It is a reprint with permission from the International Association of Elevator Engineers (www.elevcon.com).

by Shuhei Noda, Kentaro Yokoi, Hiroshi Sukegawa, Teh KoK Long and Sayumi Kimura

We have developed a camera-based door sensor that detects people approaching an elevator and prevents a collision between a passenger and the closing door. The new sensor has two advantages over conventional photoelectric sensors: a wider detection area and precise motion detection. It can determine the difference between a person approaching the elevator car and one passing by the door at the entrance area. We have implemented the algorithm on an embedded system and run it in real time.

Introduction

Toshiba has been selling photoelectric sensors for preventing people from being caught between or pulled by doors. Figure 1 schematically depicts the detection range of a photoelectric sensor for detecting a person who is about to get on an elevator. When a person who is about to get on the elevator is detected, the doors are reopened to prevent the person from being caught between them. However, this photoelectric sensor has such a narrow a detection range and is so close to the doors that it has only a limited effect in terms of safety.

In view of this circumstance, we have developed a new door sensor using a camera (camera-based door sensor).[1] Figure 2 depicts the detection range of the new door sensor. It is safer than conventional sensors because it has a wider detection range. Furthermore, it can predict when a person is approaching the elevator by using a newly developed video-analysis technology. With this feature, the new door sensor will not open or close the doors unnecessarily, because it does not respond to a person simply passing in front of the doors or who has stopped.

In 2016, Toshiba released SMART DOOR™ for controlling the opening and closing of doors according to the detection result of the new door sensor. This paper describes the configuration of the new door sensor with the employed video analysis technology and provides results of performance verification tests.

Composition of Camera-Based Door Sensor

The new door sensor is composed of a camera, on which a wide-angle lens is mounted, and an image-control processing board (Figure 3). The camera is mounted on a frieze board installed in the upper part of the elevator car (Figure 4) and captures an image with a field angle overlooking the elevator entrance area. Because the sensor is installed in the elevator car, there is no need to install a camera on each floor, and, thus, the SMART DOOR system can be configured with one sensor per elevator.

Figure 5 is one example of an image that is captured when a person approaches the elevator. The camera captures video images of the elevator entrance area and a part of the interior of the elevator car, including the border between the elevator car and entrance area. The video images are processed by the image control processing board, and the system determines if a person is approaching the elevator. If it is determined that a person is approaching the elevator, the doors are re-opened or are kept open.

The video-analysis technology that runs on the image- recognition processing board will be described in detail in the following section.

Video Analysis Technology

Figure 6 shows the flow of the video-analysis method that has been developed for the new door sensor. This section describes the video-analysis method according to this flow.

Dividing an Image Into Arrayed Blocks

The image captured by the camera is divided into an array of segments (Figure 7). The minimum unit formed by division is referred to as a block. Dividing the image into blocks is advantageous in reducing the amount of image-analysis processing.

Detecting the Motion in Each Block

The mean image luminance value is observed in a time series for each block to detect changes in the luminance value. If the luminance value changes in a time series, it is considered that there is motion (Figure 8).

Detecting Foot Position

The position of the block closest to the doors from among the blocks in which a motion is detected is considered to be the foot position (Figure 9).

Determining if the Foot Position is Approaching the Doors

The distance from the doors to the foot position is observed in a time series. If the distance becomes short over time, it is determined that the person is approaching the doors (Figure 10). If the distance does not become short, it is determined that the person is not approaching the doors and has no intention to get on the elevator (Figure 11).

With the above-described method, a person who intends to get on the elevator can be detected within a wide range at the entrance area. Meanwhile, if a person is just passing in front of the elevator or is just standing, the sensor does not respond because he/she is not approaching the doors; therefore, the doors can be closed. In this way, elevator passengers are not bothered, but safety is still ensured. Introducing this new door sensor can provide a comfortable elevator usage environment.

Because the new door sensor is based on motion detection, it will detect not only a human body, but also a person in a wheelchair or a stroller being pushed by a person. For this reason, safety can be enhanced at various locations, such as dwellings, commercial facilities and office buildings.

In addition, the new door sensor has an approach detection area very close to the elevator doors. It is determined that a person is approaching the doors merely by detecting motion (Figure 12). We have succeeded in further enhancing safety by providing this detection area.

We implemented the video-analysis method described above into an embedded system. Like a general embedded system, the new door sensor does not have a lot of computational resources. There are severe constraints on the CPU performance for computational processing and the storage capacity of the memory for captured video images and calculation results. This meant that to implement the video-analysis method without decreasing the detection performance, we needed to reduce the amount of memory and calculations. This was achieved by dividing the image into blocks as described in (1) of the image analysis method.

Results of Performance Verifications

To verify the performance of the new door sensor, we captured video images with the camera using an actual elevator. Assuming actual operation of the elevator, images were captured under various conditions, including different environments and different motions of a person, thus generating a database of video images for evaluation. Figure 13 shows examples of evaluation video images.

We verified the performance of the captured evaluation video images offline. The result was that all motions of the person approaching the doors could be detected.

We also verified the entire system, including control of the opening and closing of the elevator doors. Assuming actual operation of the elevator, the motion of getting on and off the elevator, the motion of walking and standing in front of the doors, and so forth were repeated many times. As a result, it was confirmed that the opening/closing of the elevator doors was controlled properly based on the detection performed by the new door sensor.

Conclusions

The new door sensor we have developed can determine whether a person is approaching the elevator on the basis of the video-analysis technology. When it is determined that a person is approaching the elevator, the doors are reopened or kept open. If there is a person passing or standing in front of the elevator, the doors can be closed without responding to that person. Thus, a safe and comfortable elevator usage environment can be provided.

In 2016, Toshiba released SMART DOOR for controlling the opening and closing of elevator doors on the basis of a sensor detection result. This smart elevator is widely on sale now.

Reference
[1] Kawaguchi Kenichi, Nakanishi Kota, Onoda Hiroshi and Nishimura Hidekazu. “Elevators Offering Comfortable Ride by Application of Universal Design,” Toshiba Review, Vol. 72, No. 2, p. 32-35(2017).