Image: A smart perception system that assists blind and visually impaired people in a variety of indoor and outdoor environments.
A lego block for spatial computing
OAK-D is an integrated unit that combines the technology you need to start building spatial AI products. (For the uninitiated, spatial AI refers to the use of AI to to tell you what a physical object is, and where it is in 3D space, in real time.)
Brandon Gilles dreamed up this $299 hardware unit and now uses it extensively in the products offered by his spatial AI company Luxonis. He explains that he wanted to “make something that’s the equivalent of the website design experience you can have in 2021, instead of the one you had in the 1990s.”
Why it matters
Spatial AI is currently very common in mobile devices, where developers and businesses already have access to processing power, multiple cameras, and maybe even a lidar unit. OAK-D puts this hardware in a small, affordable package—and adds the tech you need to run deep learning models on board. As a result, it is poised to bring the technology into previously impossible applications.
It’s already making waves
A recent Bloomberg article explains that Gilles developed OAK-D while building a tool to alerts cyclists when a car comes too close. Since then, the kit has been used by ” a company that makes an autonomous submarine that can detect underwater pipeline leaks, an entrepreneur building an intelligent robotic dog assistant, and DARPA.”
It has also played a central part in an Open-CV competition for solutions to real-world problems using spatial AI.
Here are the proofs of concept developed by the winners of last year’s competition. Check out the announcement page if you want to see a full video for any of these applications.
- Backpack-based vision system that helps the visually impaired. It uses spatial AI to identify traffic signs, hanging obstacles, crosswalks, moving obstacles, and elevation changes like staircases and curbs.
- Universal hand control. This allows people to use their hands to remotely control computers, or connected devices like smart speakers and TVs.
- Automatic parcel classification and dimensioning. This technology could help remove the bottleneck in international cargo logistics caused by the inability to measure cargo accurately.
- A solution that enhances the real-time perception of autonomous forklifts with advanced sensors, vision and geo guidance technology. This will help improve the performance of these forklifts, which warehouses already rely on to “load, unload and transport goods within the warehouse area, by connecting to one another and forming flexible conveyor belts.”
- At-home workout assistant. This solution judges the pose and location of at-home gym users to advise on the correct way to perform exercises and avoid injury.
- Automatic lawnmower navigation. This solution makes autonomous lawnmowers smarter. It removes the need for boundary wires and prevents the robots from killing small animals, hurting children and housepets, and driving into molehills or other “unwired” obstacles.