When you read about spatial computing, it’s extremely common to hear from early-adopters who are already knee-deep in the technology: VPs at 3D visualization start ups, technologists, manufacturers, mixed-reality consultants, and so on.
But if you’re a potential end user in the field, or you’re working in a more technologically conservative role or industry, you probably want to hear more from people like you. How are they planning to use spatial computing? What do they think of hyped technologies like AR and VR, or the metaverse? What does their road to adoption look like, and what can you learn from it?
At Spatial Computing, we want to know those answers as well. So when EveryPoint’s marketing director Jonathan Stephens attended EquipmentSHIFT—a recent conference organized by the Association of Equipment Management Professionals—I interviewed him to learn more about the ways this oft-overlooked industry is moving into the realm of spatial computing.
We chatted about equipment managers getting used to the idea of spatial computing, and how they want to use it for predictive analytics, better supply-chain management, faster and more sophisticated communication with stakeholders, and even attracting younger professionals to the job.
Sean Higgins: Can you start by talking a bit about EquipmentSHIFT? What is it?
The basic idea was for equipment managers to talk about pretty much all aspects of managing equipment. That’s heavy equipment that you would see at a construction or mining site, like a bulldozer, a front loader, or even your rolling fleet like trucks, pickup trucks, and so on. You might also be talking about the equipment at an asphalt plant. Basically, you’re talking about machines that require lots of maintenance, all the time.
So the attendees are not necessarily technologists, analysts, or back-office guys. They’re in the field.
Right. Over 50% of the people of the attendees were actual end users. They’re the managers who own and use the equipment, or maybe rent it. They’re the ones who decide what to purchase and how to maintain it. Then there was a small percentage of attendees that were equipment dealers, OEMS such as Caterpillar or Komatsu, or parts dealers.
So you had—as it as one presenter called it—the three legged stool of the end user, the dealer, and the OEMs. They all have to work together in this industry. The end user gives data to the OEM to help improve their machinery, and then it trickles down to the dealer. And the dealer is usually working directly with the end user to make sure they have their needs met.
This seems like a great crowd for thinking about spatial computing in real-world business usage, and in a less hyped context. So what does spatial computing actually mean to the people you spoke to? Did anyone mention it?
The phrase “spatial computing” never came up.
Where are they at, technologically speaking? What’s their level of sophistication right now?
I’ll tell a story to explain. I was talking to one gentleman who was head of equipment management for quite a large construction company. He said about five or six years ago, the hot word at this conference was telematics. What is it? How do I use it? How do I get it installed? What are the benefits? That means, very recently, technology wasn’t prevalent across the equipment industry.
And they found that telematics can be very helpful. With telematics they know, for instance, where all their fleet is, or how many hours their machines are running for. Or, let’s say they have 15 operators of equipment on your site, and they’re managing all the equipment for it. Before telematics, they’d have to rely on them to say, I noticed this check engine light has turned on, or I’m hearing this weird noise, or I’ve found some overheating issues.
With telematics, they don’t need to rely on the operators. The technology can alert the management or technicians so they can catch things that used to go unnoticed much longer, and just have better data.
It sounds like they’re doing spatial computing without realizing it.
Think of it as like Google Maps-level 2D spatial computing at this point. They can see where their fleet is. They are getting feedback that allows them to ask, Why is that truck idling? Why is a piece of equipment sitting there for that long, why has it not moved?
They’ve learned that this little bit of real-time data has already made a huge difference. And they know that that technology is coming for them quick, so they’re definitely embracing it as a group. Your larger construction companies, regional companies, and so on, they see the huge value in in technology, helping them stay ahead.
And how did these equipment managers feel about 3D spatial data? You were there to talk about 3D capture, right?
Exactly, that was my presentation. I walked them through the ease of capturing 3D data and its value.
I explained that 1.4 trillion pictures are going to be taken this year, mostly with smartphones, and asked how many of them have taken a picture at this conference. Everyone raised their hand. I said, How many of you guys have uploaded social media? Most of them raised their hands.
So I told them—You guys are already taking pictures, you’re just taking pictures of the wrong things for what you do. What happens if you take pictures, like you do for social media, but for your equipment? Then I showed them how they can take pictures of their equipment, like a dozer, and turn them into a 3D model. A lot of people were there with mouths open. “All of this just from some pictures?”
This industry, which has only just recently started to adopt telematics, immediately saw a use for 3D capture and spatial computing.
It really connected with them. They saw that they can do a simple capture, and then use these 3D models to do remote human inspection. With a 3D model, they recognized they’d have all these reference pictures that they can zoom in on and get different angles.
I think it really connected with a lot of people that visual intelligence is very important, because they’re going to identify a lot of issues with their equipment just by seeing things.
So, the idea of having a 3D model was kind of big for them. They learned that when they have a problem, they could use this 3D data to annotate it—to say, watch out for this area, this is becoming an issue. They could monitor tread health, prevent the blowout of a tire by marking when the sidewall has structural issues, or get ahead of it when the teeth need to be replaced on their excavator’s bucket.
Are there any other uses for 3D spatial computing data that they locked onto?
We talked a bit about inspection, too. I showed them a confined spot in a cement plant and said, if you just took a little video, you could turn it into a 3D model and then you could inspect this area remotely, when you’re not in a confined space. Any uses like that, which promote safety and promote proactiveness, were big for them. And since they all have smartphones with great cameras, it’s an easy sell.
How do they feel about technology more advanced than smartphone photogrammetry? Did they have any sense about tools like wearables?
I had these Ray-Ban wearable cameras on, and they were a big hit. They loved the idea that you could have a camera in your safety glasses that would capture imagery or video automatically as you walk around inspecting things. At that point, the data collection is just happening. You don’t even need to hold out your phone.
Basically, they were eating this up because they figured out telematics and felt like well okay, this visual intelligence, these 3D models, are the next big thing.
And spatial computing, with 3D models and all that, fits in neatly with their current use of telematics?
It’s part of a progression.
Maybe these equipment managers started with paper checklists and spreadsheets. Then they started using telematics to get more real-time data. Now they’re using predictive analytics, which gives them some guidance what to do, but they’re still doing everything manually. By using spatial computing, they can move to the next stage, they can use these 3D models to feed machine learning algorithms that make decisions automatically.
Now, maybe they can have cameras set up at a bay where they park their machines, and the cameras just scan them when they get parked. Then all this data is stored in the cloud, so they can share the 3D data across the organization, and then integrate it into a bunch of different automated workflows.
Using this process, they could probably send the data to their dealer or OEM directly, right?
Right, if you bring in your dealer and your OEM, now there’s an ecosystem industrial cloud where you could be sharing this data. And this is where—at least at EveryPoint—we believe this kind of enterprise 3D tech could go.
We believe that OEMs will have an app for their machines. The equipment manager scans their machine, the OEM gets the data. Now they can help the manager determine what needs to be fixed. At the same time, they get all that great wear data to feed their machine learning algorithms, which can help them improve the machines in the future.
And this app can even connect with the dealer. So, if the app “sees” that something is becoming an issue with a machine, it can tell the manager, hey, this part is going to need replacement within the next month or two months. And then it can notify the dealer, who can determine if they have the replacement part already. If not, the dealer can order the parts from the OEM.
This is where the equipment managers thought the technology was really interesting, because knowing the part is going to fail in advance helps them to play supply chain game and get further in front of it, giving them a longer lead time to source replacement parts. That’s important right now when everything is out of stock due to supply chain problems.
And maybe the manager can talk to the OEM to figure out what kind of maintenance they can do in the interim.
This sounds like a win for everyone involved—more data means better efficiency.
And this enables OEMs to get a data point that they love. These equipment managers can use spatial computing to monitor their machines in real time, to see how often they’re working. Not just how long the machine is on, but how many hours it’s being used. That’s important because, if someone is operating the machine for 100 hours moving loads, it’s not moving loads for 100 hours. There is idle time in between.
The more granular the managers can get on that actual usage data, the better the usage data they can send the OEM, who can better predict when a part will need to be replaced. They could say, OK, you have 10,000 scoops on that bucket, so our machine learning model says it’s probably time to replate it, based on average use.
That’s big. It’s encouraging to see that people who are in less technologically glamorous positions are getting ahold of spatial computing technology, and that they see the value in it. I wonder, though, do they have any interest in the metaverse, or AR/VR/MR?
They see the value in tech, but they don’t care about virtual environments. They are not going to be spending time there. They literally just want to know how quickly they can scan it, and how they will get notified when they need to take action, or when they need to know a part’s going out.
And maybe they are looking at 2D or 3D images to help diagnose a problem, but that can easily be on a browser, they don’t need a 3D headset. The technology is simply a tool for a remote person to get a better look at something and give an expert opinion on an issue. A dealer, for instance, can easily diagnose problems just through an iPad or website.
They sound ready to use spatial computing. But are there any remaining challenges that stand between them and adopting it?
They’re waiting for someone to actually build an app or a solution that makes sense specifically for their industry. And I don’t think that will come from a technology company. That’ll come from an equipment manufacturer. That’s who could work hand in hand with the telematics software company to make something useful. Because it’s going to take an expert to realize what the telematics company, and the end users care about.
I think the technology is already there. It’s not like it’s not like the metaverse where we’re still trying to figure out how to miniaturize augmented reality headsets that people will wear more than an hour. That’s not a problem. The question is, who’s going to build the software?
Anything else? It’s usually not just one issue.
This technology is new, so someone’s got to go first prove that it will make a difference. People in this industry are not early adopters for the most part. So, it’s going to be a hard sell for some of these companies.
I also think the best solution is going to be as invisible as possible. It’s going to be something like putting these wearable cameras on or sending robotic dogs to walk around the site and scan the machinery for them. The more that this work is hands off, and the more this spatial computing just happens, the better it’ll be for the industry. Because they already have a lot going on.
We need to make it as simple for an operator as walking around the machine with their glasses on. That might mean making a checklist that pops up on the HUD that says, go look at this part, go look at that part. As they go around, they’re collecting all the imagery at the same time through their smart safety glasses. And the system automates the rest.
As we finish up here, I want to ask—Did you have any other takeaways, thoughts, comments, any, any important feelings or any things you noticed at the conference there that we didn’t cover?
One thing I thought was very interesting is that they dedicated the closing keynote to figuring out how are we going to attract young talented people into this industry? People are not going into construction; `people are not going into equipment maintenance. It’s not the hot new place to be right now.
And when we split into small groups, my group talked about how crazy that is. More and more people want to go into tech, and make a lot of money in tech, but what people aren’t realizing is that construction and technology have collided completely.
Construction companies need savvy technical people who will use technology in amazing ways. If you’re interested in technology, you could be a technologist within a construction company, or a mining company. They need data analytics experts, they need telematics experts, they need scanning experts, they need all these people who will lead technological adoption in the company.
It’s one of the last industries to be hit by technology full force. And they need young people to be the hands-on technologists, using tools like 3D capture and machine learning. Spatial computing is coming at the construction industry, and if you want to be a leader in it, you should jump in.