Emergency Communication for All

Enhancing Elevator Emergency Communication Systems by Incorporating AI-Powered Language Translation and Accessibility Technologies

by Kurt Schieszer

Abstract

The 2018 International Building Code (IBC) introduced the integration of video, voice and text into elevator emergency-communication systems, adopted into ASME A17.1/CSA B44 2019, to enhance passenger safety. This paper proposes further advancing these systems by incorporating AI-driven language interpretation technologies, enabling real-time translation and communication for non-native speakers and individuals with language barriers. It suggests leveraging AI and Machine Learning (ML) for automatic speech pattern analysis, speech-to- text and text-to-speech capabilities, ensuring accessibility for all. The implementation calls for collaboration between elevator manufacturers, software developers and regulators to ensure reliable, multilingual support, ultimately improving emergency response efficiency and inclusivity.

Introduction

The 2018 IBC set the groundwork for ASME A17.1/CSA B44 2019. The 2018 IBC introduced a few outlines of how video, voice and text (VVT) should operate. ASME A17.1 introduced more details around the significant improvements to elevator passenger safety and quality through an enhanced emergency-communication system. By mandating the integration of VVT communication capabilities, the code laid the foundation for better emergency management for passengers, building management and emergency response teams. However, as technology rapidly evolves, opportunities arise to enhance these systems further to meet the growing needs of diverse populations, including non-native speakers, individuals with hearing impairments and people with a wide range of disabilities.

This paper explores the potential benefits of integrating simple AI and ML technologies into elevator emergency-communication systems, focusing on automated real-time language translation and accessibility enhancements that can be utilized with or without internet access. By analyzing the current landscape of elevator safety, user demographics and the projected financial costs of implementing these systems, this research aims to propose critical updates to the ASME A17.1/CSA B44 2019 code that will make elevator emergency communication more inclusive and effective.

Background: The Evolution Of Elevator Safety And Communication Systems

Elevator safety has been a cornerstone of building design for decades with continuous updates to codes and standards aiming to minimize the risks associated with entrapment and mechanical failures. The adoption of two-way communication systems into elevator cars was a major milestone, allowing trapped passengers to communicate with authorized personnel during emergencies. Then, the addition of a hands-free phone further improved usability of the system. The introduction of the IBC 2018 mandates for VVT-based communication utilizes a video and screen for remote access, while ASME 17.1 also requires an on-premises system for buildings that have an elevator with more than 60 ft of travel.

The addition of this code has undoubtedly caused a major disruption in the industry, but with the code being widely adopted now, further changes will have little impact on the cost or implementation. The benefits of expanding the offerings to include AI tools can better serve buildings and help emergency responders when working with entrapped passengers.

Despite these advancements, critical gaps remain in the inclusivity and efficiency of current systems. According to U.S. Census Bureau data from 2020, nearly 22% of the U.S. population speaks a language other than English at home, and more than 40 million Americans are classified as having some form of disability, including hearing and speech impairments. These statistics underscore the importance of designing emergency-communication systems that can cater to diverse populations, particularly in high-density urban settings where language barriers and disabilities may impede emergency responses.

Proposal: Ai-Driven Language Translation And Accessibility Features

What if we can adapt and tailor the emergency response to those in need?

Incorporating AI technologies into elevator communication systems presents an opportunity to address the limitations of current designs. By integrating automated language translation and speech-to-text (STT) and text-to-speech (TTS) technologies, elevators can provide a seamless communication experience for all passengers, regardless of their language proficiency, physical abilities or mental disabilities.

AI-driven algorithms can automatically detect and interpret multiple languages in real-time, providing instant translation between passengers and emergency personnel. When a passenger speaks, the system can recognize the language, convert it into English (or another target language) and display or vocalize the message to authorized personnel. Similarly, responses from emergency personnel can be translated back into the passenger’s language via text or voice output.

This multilingual support is especially crucial in large metropolitan areas or popular tourist locations. For example, NYC alone has more than 3 million foreign-born residents with more than 200 languages spoken, according to nyc.gov. Nearly one-half of all New Yorkers speak a language other than English at home, and almost 25%, or 1.8 million persons, are not English proficient. Ensuring that emergency-communication systems in such areas can accommodate this diversity is essential for maintaining passenger safety and well-being.

Beyond these capabilities, analyzing the conversation in real time can provide emergency personnel with suggestions on how to proceed with assisting the entrapped passenger. AI tools can monitor the tone and panic in the voice or track health information that is disclosed (e.g., pregnant, diabetic, high anxiety) and then provide information to the response team on considerations of how to proceed.

In the event that emergency communication is not being actively monitored, AI can step in to act as the failsafe and provide automated communication rather than the phone going unanswered. Protocols can be set up to provide real-time communication updates via text to building management or emergency call centers. The conversation can then be accessed on-site by emergency personnel when they arrive.

Stt and Tts Integration

For passengers with hearing or speech impairments, traditional two-way communication systems may not suffice. By incorporating STT and TTS technologies, elevators can bridge this gap. STT converts spoken words into written text, allowing emergency personnel to receive messages from passengers who may not be able to speak clearly or at all. Similarly, TTS technology reads out text-based messages sent by emergency personnel, ensuring that passengers who cannot read the displayed text still receive instructions.

This dual capability not only enhances inclusivity but also improves the efficiency of emergency responses. Passengers can communicate more clearly, and emergency personnel can quickly assess the situation based on accurate, real-time information. Additionally, this technology does not require significant processing power or an internet connection; it can be installed in the hardware.

Impact On User Safety

The primary motivation behind integrating AI-powered communication systems is the potential to significantly improve user safety. According to the National Elevator Industry, Inc. (NEII), more than 17 billion elevator trips are taken annually in the U.S. With the diverse demographic landscape, particularly in cities with large immigrant populations, the chances of language barriers or communication difficulties during emergencies are high.

By implementing multilingual translation, STT and TTS technologies, elevators can provide real-time, accessible communication to passengers who might otherwise struggle to receive or convey critical information during emergencies. For example:

• Non-native speakers: In an emergency, panic and stress can further impede communication for passengers who do not speak English fluently. AI-driven language translation can instantly detect their native language and facilitate clear communication.
• Hearing or speech-impaired passengers: Traditional voice-based communication systems may leave passengers with these disabilities unable to seek help. STT allows them to type or speak in fragmented words, while TTS reads out messages from emergency personnel, ensuring no one is left without assistance.
• Other disabilities: Where the passenger has a disability such as dyslexia or autism, the functionality can be adapted to best suit those entrapped while also notifying the emergency personnel, providing the opportunity to reduce panic and more accurately address the situation. Where a loud tone would benefit someone who is hard of hearing, it could create a higher panic situation for one with autism.

The U.S. Census Bureau reports that nearly 14 million people in the U.S. identify as having hearing impairments. Additionally, more than 40 million people, or about 12% of the U.S. population, are non-native English speakers. Given these numbers, incorporating AI-powered systems that cater to these groups can prevent a significant number of emergency situations from escalating due to communication barriers.

Recommended Changes

To create a safer environment and to ensure that these advanced communication systems become standard in future projects, the offerings provided by manufacturers must be expanded and further developed. Updating the code for IBC 3001, ASME A17.1, CSA B44, and EN 81 would mandate these upgrades offered by AI and machine learning technologies if the industry is not able to get there on its own. The following changes are recommended:

1. Mandate for Automatic Language Interpretation:
   • Section 2.27.1 should be updated to require all newly installed elevators to include AI-driven language interpretation capabilities, supporting at least 10 of the most commonly spoken languages in the region.
2. Inclusion of SST and TTS Technologies:
   • Section 3001.2 in the IBC should mandate the inclusion of STT and TTS systems in all elevators, ensuring that passengers with hearing and speech impairments can communicate effectively during emergencies.
   • Emergency personnel should also be required to receive training on interacting with these systems to ensure smooth communication during critical situations. 
   • There should be no more than a 5-s delay on the translations.
3. Inclusion of On-Premises Devices:
   • Currently there is insufficient language in the Code to require a complete system in the car and on-premises devices for buildings with 60 ft of rise.
   • The Code needs to require a complete system that includes the VVT equipment as fully integrated.
4. Required Touchscreen & Buttons:
   • IBC 3001.2 and ASME A17.1, 2.27.1 needs to require a touchscreen so the user can select, or confirm, the language or assistance needed.
   • Additionally, to simplify the responses, “yes” and “no” buttons should be required in lieu of allowing for alternative buttons to be used.
5. Standardization of Emergency Communication Testing:
   • Both the IBC and ASME Code should introduce regular testing requirements for AI- powered communication systems, ensuring that they remain functional with or without internet connection.
   • Annual inspections should include assessments of the accuracy and speed of language interpretation and the reliability of STT and TTS functionalities.
6. Collaboration with AI Developers and Manufacturers:
   • NEII, the National Association of Elevator Contractors and the International Elevator & Escalator Symposium should encourage collaboration between elevator manufacturers, AI developers and software vendors to establish clear guidelines on the integration of these technologies.
   • This initiative should provide detailed specifications on the hardware and software components required to meet these new standards.

Technical Implementation And Feasibility

1. Software and Hardware Requirements:
   • Overview of the necessary components to support AI-driven language interpretation in elevators
   • Analysis of current market availability and reliability of such technologies
2. Offline Capabilities:
   • Language Models and Datasets: The AI models, including language models and translation datasets, need to be stored locally. These can be updated periodically through manual updates rather than continuous internet connectivity.
   • Real-Time Processing: The local system should be capable of real-time processing, meaning it must handle speech recognition, language detection, translation and TTS with minimal delay. This requires optimized algorithms and efficient use of hardware resources.
3. Examples of On-Premises AI Solutions:
   • NVIDIA Jetson Series: These are powerful embedded AI systems designed for real-time processing and can be used for offline AI tasks, including language processing.
   • Intel Movidius Neural Compute Stick: This USB stick provides AI processing power to devices and can be used for local AI inference tasks.
   • Custom AI Appliances: Companies like Dell and HPE offer specialized AI appliances designed for on-premises deployment, capable of running complex AI models offline.

Conclusion

Updating elevator emergency-communication systems to incorporate AI-powered multilingual interpretation and accessibility technologies represents a critical step toward ensuring the safety and security of all passengers. By enhancing existing communication systems with STT and TTS capabilities, we can provide more inclusive and efficient emergency response frameworks.

While the financial costs of implementing these systems may seem high, the potential benefits in terms of user safety, reduced liability and improved emergency response times justify the investment. By updating the ASME A17.1/CSA B44:19 Code to reflect these advancements, we can ensure that elevators in new and existing buildings are equipped to meet the communication needs of diverse populations, ultimately making building infrastructure more resilient and responsive to emergencies.

References

[1] census.gov/content/dam/Census/library/stories/2022/12/languages-we-speak-in-united- states-table.jpg 

[2] nyc.gov/site/planning/about/language-access.page

With more than 3 million foreign-born residents from more than 200 different countries, New York is home to one of the most diverse populations in the world. New Yorkers come from every corner of the globe and speak more than 200 different languages.

Nearly one-half of all New Yorkers speak a language other than English at home, and almost 25%, or 1.8 million persons, are not English Proficient.

Appendix
A Code Changes
IBC 2018

3001.2 Emergency elevator-communication systems for the deaf, hard of hearing and speech impaired. An emergency two-way communication system shall be provided that:

1) Is a two-way visual and text-based and video-based 24/7 live interactive system.

2) Is fully accessible by the deaf, hard of hearing and speech impaired, and shall include voice- only options for hearing individualsvoice and text communication for all individuals.

3) Has the ability to communicate with emergency personnel utilizing existing video conferencing technology, chat/text software or other approved technology.

4) Has the ability to automatically detect and physically select the preferred language to communicate in for the individual in the elevator. Automatic translation in text-to-voice and voice-to-text shall be provided.

ASME A17.1-2019

2.27.1.1 Emergency communications. The two-way voice‡ communications shall conform to Sections 2.27.1.1.1 through 2.27.1.1.6.

2.27.1.1.1 A communications means between the car and a location staffed by authorized personnel who can take appropriate action shall be provided.

2.27.1.1.2

(a) Two-way voice communication shall be provided between the elevator car and elevator machine room, and/or control room.

(b) If the call is not acknowledged [Subsection 2.27.1.1.3(c)] within 4530 s, the call shall be automatically directed to an alternate on- or off-site location.

(c) If the alternate call is not acknowledged within 30 s, the call shall be responded to by an AI answering device. The transcription of the conversation shall be relayed to the primary, alternate, and emergency personnel via voice to text or text to voice means.

2.27.1.1.3 The permanent two-way communication means within the car shall comply with the following requirements:

(a) ICC/ANSI A117.1.

(b) A push button to actuate the two-way communication means shall be provided in or adjacent to a car operating panel. The push button shall be visible and permanently identified with the “PHONE” symbol (see Section 2.26.12.1). The identification shall be on or adjacent to the “PHONE” push button. The communications means shall be initiated when the push button is actuated.

(c) On the same panel as the “PHONE” push button, a message shall be displayed that is activated by authorized personnel to acknowledge that communications are established. The message shall be permitted to be extinguished where necessary to display a new message [see Subsections 2.27.1.1.3(d) and 2.27.1.1.3(e)] or when the communications are terminated.

(d) On the same panel as the “PHONE” push button, messages shall be displayed which permit authorized personnel to communicate with and obtain responses from a trapped passenger(s) including a passenger(s) who cannot verbally communicate or cannot hear.

(e) On the same panel as the “PHONE” push button, a message shall be displayed that is activated by authorized personnel to indicate when help is on the way. The message shall continue to be displayed until a new message is displayed [see Subsection 2.27.1.1.4(c)] or the communication is terminated.

(f) The communications means shall provide on demand to authorized personnel, information that identifies the building location and elevator number.

(g) The communications, once established, shall be disconnected only when authorized personnel terminate the call or a timed termination occurs. A timed termination by the communications means in the elevator, with the ability to extend the call by authorized personnel, is permitted if voice notification is sent by the communications means to authorized personnel a minimum of 3 min after communication has been established. Upon notification, authorized personnel shall have the ability to extend the call; automatic disconnection shall be permitted if the means to extend are not enacted within 20 s of the voice notification.

(h) The communications means shall not use a handset in the car.

(i) The communications shall not be transmitted to an automated answering system unless primary and alternative answering systems were not connected. The call shall be answered by authorized personnel.

(j) Operating instructions shall be incorporated with or adjacent to the “PHONE” push button, the touchscreen located in the main car operating panel.

(k) A means to display video to observe passengers at any location on the car floor, to authorized personnel for entrapment assessment, shall be provided.

(l) A means to automatically dictate the language being spoken of the entrapped passenger shall be provided.

(m) A means to automatically dictate and, if necessary, translate voice to text shall be provided. The text shall be made visible inside the elevator cab on the touchscreen in the main car operating panel.

2.27.1.1.4 Where the elevator rise is 18 m (60 ft) or more, a permanent two-way voice communication means within the building accessible to emergency personnel shall be provided at the designated level, and shall comply with the following requirements:

(a) The means shall enable emergency personnel within the building to establish communications to each car individually. The communications shall be established without any intentional delay and shall not require intervention by a person within the car. The means shall override voice communications to outside of the building.

(b) The communications, once established, shall be disconnected only when emergency personnel outside the car terminates the call or a timed termination occurs. A timed termination by the communications means in the elevator, with the ability to extend the call by emergency personnel, is permitted if voice notification is sent by the communications means to emergency personnel a minimum of 3 min after communication has been established. Upon notification, emergency personnel shall have the ability to extend the call; automatic disconnection shall be permitted if the means to extend are not enacted within 20 s of the voice notification.

(c) Once the communications have been established, a message shall be displayed on the same panel as the “PHONE” push button, which is activated by emergency personnel, to indicate that help is on-site. The message shall be permitted to be extinguished where necessary to display a new message [see Subsection 2.27.1.1.4(e)] or when the communications are terminated.

(d) Operating instructions shall be incorporated with or adjacent to the communications outside the car. Instructions shall conform to Section 2.27.7.3.

(e) On the same panel as the “PHONE” push button, messages shall be displayed that permit emergency personnel to communicate with and obtain responses from a trapped passenger including a passenger who cannot verbally communicate or cannot hear.

(f) A means to display video to observe passengers at any location on the car floor to emergency personnel for entrapment assessment shall be provided. The communications system shall be located within the fire command center where one is provided.

(g) A means to automatically translate text to and from English, Spanish, Chinese, Tagalog, Vietnamese, Portuguese, French, German, Arabic and Italian shall be provided. The primary language shall be English and shall always be visible in addition to the language identified or selected by the person in the elevator.

(h) A means to adjust the size of the text and volume shall be provided and can be activated by the touchscreen or voice control. Size of text shall not be less than 1/4-in. high and not more than 1-in. high. Volume of communications shall be 10 dBA minimum above ambient, but shall not exceed 80 dBA measured at 12-inches in any direction from the “PHONE” push button.