In the world of first responders, time is everything. How quickly a fire department or emergency medical team can get to a scene, assess the situation, plan, and act can be a matter of life and death.
This makes training essential to their effectiveness – the more realistic, the better.
And outside the real world, there’s nothing more realistic than a digital twin, according to Bill Gregory, CEO of RSET, Inc., a company that creates virtualizations for fire, EMS, police, and military groups to conduct core skills and training.
“We’ve had some pushback, but many people get it and say, ‘No, this building is very specific to our neighborhood and what we would be dealing with, and the exterior of this building and how we get to it is very different from another building,’” Gregory said. “So those that get it say that this is exactly what we want – to do a virtual training in that scenario by using a digital twin.”
Gregory has been using a Leica BLK2GO handheld imaging laser scanner to capture digital reality data of real-world structures in minutes and Hexagon’s Reality Cloud Studio, powered by HxDR, to get a complete deliverable within hours. Reality Cloud Studio, a new product from Hexagon, is a powerful cloud application for reality capture data visualization, collaboration, and storage that offers a suite of essential tools including automated registration and meshing of scans as well as measuring, annotation, and dynamic virtual tour creation.
After scanning, Gregory uses the field-to-cloud workflow available in the BLK2GO Live app to upload those scans to the cloud. By simply pressing a button in the app, all the scan data, including images, gets uploaded to the cloud; then, using the automated point cloud registration and meshing in Reality Cloud Studio, he turns his scans into digital twins compatible with popular 3D engines. Finally, RSET uses those digital twins in their own modeling and visualization platform, built on the Unity engine, to create virtual spaces for first responders to train in.
Fire departments, for example, often do training drills in homes and other buildings slated for demolition. That means there’s a limited window that they can be used – unless they can be captured digitally.
“We used the BLK2GO to walk the premises [of a fire training drill site], and soon we’ve got the terrain map, the exteriors of the building, the trees and locations and everything,” Gregory said.
RSET’s team would then merge multiple data sources, from Reality Cloud Studio-processed BLK2GO meshes to UAV photogrammetry, into a continuous model for the fire department.
“With us, they would do one exercise physically while the building was still up, but then they would have that exercise afterward to do many times over, virtually,” he said.
Those exercises inform everything from the placement of fire ladders to the location of the hose trucks and how to deploy human resources.
“When a fire occurs, the commander is the first to rush there if he can,” Gregory explained. “He calls what trucks he needs, what resources, and if they do that wrong, it creates a problem for fighting that fire.”
“For us to be able to have our data uploaded and frequently create digital twins and go straight into the virtual training process is huge,” he said. “Because it’s just user-friendly. There’s no massive training on how to get through the software and clean files up with Reality Cloud Studio. Just upload data and download meshes. And you can’t get much easier than that.”
But that’s been the goal of RSET from the get-go. The company was initially funded by a federal Small Business Innovation Research grant and started with scanning military installations like Camp LeJeune, Camp Roberts, and Quantico — the latter of which RSET scanned during a live training exercise.
“We had to be geared up in protective gear,” Gregory said. “We literally had a 15-minute window to do a scan, and I could pick up the BLK2GO and walk through that 15 minutes and come back out. Then they went back to combat practice.”
Reality Cloud Studio automation makes digital twins accessible to all audiences
RSET says its goals aren’t to do the scans themselves but to empower first responders to use 3D reality capture technology, with RSET creating the virtual training modules for them. That’s a real possibility with a handheld scanner like the BLK2GO and automated processing through Reality Cloud Studio.
“I’ve advocated that our clients get their own scanners,” Gregory explained. “Because it’s not our model to be scanners for everyone. We want people to do their own scanning — we’re just the software to help you train in it.”
“Our stuff was built for the Marines initially,” he continued. “We were charged to build something for a non-technical marine to use. They’ve not had any training in 3D modeling of any kind. But they can play Call of Duty, and that’s exactly what we built. You load in the map, and then you can drop in characters, and you can drop in elements, and we can drop in anything that’s another OBJ file.”
Currently, RSET still does much of the scanning and processing. Still, with each trade show and demonstration, they get closer to convincing otherwise stalwart institutions that scanning and meshing could be within reach of a layperson.
“We’ve got the process to within a couple of hours,” Gregory said, “where we’ve scanned a location and come back, uploaded it, gotten it back as a mesh, and then we were immediately training it and showing people.”
“They said, ‘You guys just got here this morning, and we’re doing your demo with it right now!’” he said. “That’s always a jaw-dropper to a lot of people.”
