Eyes on the Future

thyssenkrupp unveils “mixed-reality” preventative-maintenance technology Microsoft HoloLens at 1 WTC in NYC.

Scientists studying 3D images of the surface of Mars. Medical students learning about human anatomy from digital (rather than actual) cadavers at Case Western University in Cleveland. And, now, elevator technicians training, preparing for jobs and receiving expert guidance using the same futuristic technology. Worn as a headset, it is called Microsoft HoloLens, described by Microsoft as “the world’s first self-contained holographic computer” and originally used by video gamers.

thyssenkrupp plans to roll out HoloLens in coming months to support approximately 24,000 of its technicians.

The elevator-specific HoloLens was unveiled by thyssenkrupp and Microsoft to approximately 40 journalists and interested parties during an event on the 63rd floor of One World Trade Center (1 WTC) in New York City (NYC) on September 15. After being ushered into an expansive sky lobby with amazing 360° views of the city and treated to hors d’oeuvres, attendees listened to thyssenkrupp CEO Andreas Schierenbeck, thyssenkrupp Elevator Corporate Communications Director Luis Ramos, Microsoft Azure Internet of Things (IoT) Partner Director Sam George and Microsoft HoloLens General Manager Scott Erickson describe the technology before being divided into small groups for one-on-one demonstrations in the next room. With colorful HoloLens images projected onto the walls, the room held five tables, each with one HoloLens and an expert user to guide people through the experience.

Starting at 8 a.m., four approximately hour-long sessions took place throughout the day. By the end of it, thyssenkrupp Corporate Communications team members said they were still getting a kick out of it. “It’s wonderful to watch people have that ‘Aha!’ moment when they realize what it’s able to do,” Kellie Harris, director of Media and Communications for thyssenkrupp North America, said. A little fun was interspersed, such as letting attendees see not only adjustable views of an elevator machine and various forms, but also a hologram of a monkey eating pizza and a dancing ballerina.

The fun and games belied the seriousness and complexity of the undertaking, however. thyssenkrupp is known for its willingness to invest — big — in its ideas. It is also very thorough in its thought process.

“Do you feel that?” asked George, referring to the slight jiggle of the cab as the high-speed thyssenkrupp elevator ascended to the 63rd floor of 1 WTC.

“thyssenkrupp intended it that way. They wanted these elevators to have that old-school, natural elevator feel.” It is not at all an uncomfortable sensation and points to the attention to detail that thyssenkrupp put into the 1 WTC system, which features some of the fastest elevators in the world in the tallest building in the Western Hemisphere.

thyssenkrupp chose to showcase HoloLens at 1 WTC because the structure, and NYC itself, perfectly illustrate global trends, such as urbanization, that are driving the need for technology such as TWIN (two elevators in one shaft), MULTI (the ropeless elevator system), preventative-maintenance technology MAX and, now, HoloLens. Schierenbeck observed:

“In terms of physical size, Manhattan is not growing, so we have to build up, to heights we have never experienced before. Today, buildings that are 1 km tall are becoming a reality. Since 2000, buildings taller than 200 m have tripled. Elevators made modern life possible in these structures, and increasing urbanization has led to an installed base of 12 million elevators worldwide that travel day and night. Every day, as an industry, we transport approximately 1 billion people in a very safe manner.”

Vertical transportation is an extremely complex, multifaceted industry and an individual unit within it — like an automobile — provides a service that is taken for granted until issues arise. Issues arise too frequently, and Schierenbeck believes the industry can do a better job of addressing them. That is why the company invested in preventative maintenance technology such as MAX and HoloLens.

To illustrate how such technology makes repair and maintenance more efficient, Schierenbeck observed the system in 1 WTC consists of components from all over the world — Germany, of course, but also Brazil, Canada, South Korea and locations throughout the U.S. Using HoloLens, a technician can initiate contact with experts in any of these locations to guide them, hands-free using holograms, through a problem-solving process.

Using HoloLens, the amount of time it takes to replace a part has been reduced from approximately 2.5 hr. to 25 min., Schierenbeck said. It is far better than the traditional way to troubleshoot, which involves looking at drawings and trying to visualize them in one’s head. “Now,” he said, “we can make 3D holograms and videos that show how systems work. Training is much more efficient because what technicians see is much easier to digest and remember.”

A Warm and Exciting Welcome

The HoloLens unveiling was preceded by a welcome dinner on the night of September 14 hosted by thyssenkrupp at the celebrity hotspot Tao Downton in Chelsea, where guests were treated to a lavish six-course dinner featuring sushi, lobster wontons, seared tuna and steak. Set in a dimly lit, cavernous space with a 20-ft.-tall statue of a multi-armed Buddhist “Celestial,” a pulsating light show and loud dance music spun by a DJ (in his own carved-wood DJ booth), the evening was one guests are unlikely to soon forget.

Guests came from all over the world, and included everyone from freelancers to writers for daily newspapers and technology magazines and websites. As they enjoyed the eclectic cuisine, diners networked and shared information about where they are from and their jobs. Most guests stayed at the 56-story Millennium Hilton, which afforded amazing views of 1 WTC, the under-construction 3 WTC and the Santiago Calatrava-designed Port Authority Trans Hudson (PATH) transportation hub, which, in itself, is a marvel of vertical transportation.

The Unveiling

The dinner set the stage for the unveiling of HoloLens, which thyssenkrupp is confident will “transform the global elevator service industry.” Prior to guests trying out HoloLens for themselves, George described it thus:

“It’s a full computer, just like any laptop you have, but you wear it. You look through the lenses, and you can see the world around you, while you have holographic overlays. It uses a technology we call ‘mixed reality,’ because it’s not virtual reality, and it’s not augmented reality. It takes elements of both and allows you to combine the physical world around you with digital information.”

Eyeglasses are not an issue, since the HoloLens lenses, which are attached to an adjustable headpiece, fit over them. A user’s eyes serve as a cursor, and finger taps and hand motions enable him to adjust the size of what he is looking at. A video shown prior to the live demonstrations showed how someone at a remote location can use HoloLens to receive training or get questions answered via Skype.

HoloLens augments MAX, where technicians access elevator data collected into a cloud via their laptops, phones and mobile devices. MAX is connected with thousands of units in the pilot countries of the U.S., Germany and Spain, and “is on track to be connected with 180,000 units by the end of 2017.”

HoloLens is a significant investment, pointing to thyssenkrupp’s conviction that trends such as urbanization are locked in and will continue to pick up steam. This will have a direct impact on the elevator industry, adding to workload and job complexity.

According to the publication Alphr, a single HoloLens device cost approximately US$3,000 when it debuted earlier this year. Acknowledging that HoloLens is an expensive piece of equipment, Schierenbeck stated, “It’s not cheap — not yet — but the potential is huge. It’s an amazing technology, and maybe we have not discovered all the things that we’re going to do with it yet.”