Imagine knocking over a box full of Lego bricks on a table. Now jump in with me, put on your imaginary augmented reality glasses. The camera in the AR glasses will immediately start cataloging all the different types of bricks in front of you, from different shapes to colors, offering suggestions on what models you can build with the pieces you have. But wait, someone is at the door. You will check and come back. Luckily, your glasses don’t need to rescan all of those parts. AR knows they are sitting on the table where you left them.
This ability to permanently remember real objects that have been scanned is the main argument behind a new AR software platform called Perceptus from Search Singles. Perceptus can retain these objects in memory even if the camera is no longer looking directly at the scene. As you headed for the door, the Perceptus platform kept thinking about what else you could build with the pieces on the table. It didn’t stop working just because you weren’t looking at the coins anymore.
“When we’re in an AR space, we don’t look at the whole room at once, we only look at part of it,” says Brad Quinton, CEO of Singulos Research. “As human beings, we have no problem with the idea that there are things that we cannot see at the moment because we have seen them before and remember them. Once you have an AR that can understand what’s around you, it can kick in and proactively do things for you.”
At least, that’s the idea. Perceptus acts as a layer on top of existing AR technologies like Apple’s ARKit or that of Google ARCore, which developers use today to create AR applications. But a lot has to happen behind the scenes before it can work on your smartphone or tablet.
The app developer provides Singulos Research with 3D models of the Lego bricks – or any object – they want Perceptus to detect. The platform then uses a type of machine learning process where it studies all the different ways it can expect to see the object in the real world, with different lighting conditions, on different surfaces, etc. . Perceptus is then overlaid on the developer’s application, allowing them to use this new object understanding. It’s the developer’s job to make sure the app actually gives you things to do with the objects, like how our imaginary Lego app might suggest things you can build using the bricks it identifies .
Scanning and identifying objects are still very manual processes. To get started, app developers who license the Perceptus platform will need to provide computer-aided design models of the objects they wish to memorize. But these CAD models will be added to the Singulos library, and future developers will be able to browse the digital stacks to find the objects they need faster. Soon, Quinton expects Perceptus to be able to identify a host of common elements, especially since there are already “a large number of very accurate 3D models available” from video game manufacturers.