Augmented reality is a technology more commonly associated with Pokemon Go and Snapchat filters, but Edgybees has found numerous real-life use cases for the technology - including search and rescue operations, military missions and broadcast. CEO Adam Kaplan explains how it works to Dan Robinson
Edgybees' AR software helps in search and rescue missions
Until now, augmented reality has been associated with video games like Pokémon Go – but Edgybees has found a way of using the technology to save lives.
The San Diego start-up works with emergency services and defence organisations to deploy AR in operations for search and rescue, military, fire response and criminal suspect pursuits.
By overlaying data such as street names, individual locations and points of interest on real-time video, it provides a helping hand to people working in those fields using geolocation services.
CEO Adam Kaplan – who co-founded the company in 2016 with Amazon alumni Menashe Haskin, described as Edgybees’ “brains” after helping to develop Alexa and working on creating a delivery drone for the e-retail giant, and Nitay Megides – says the company has come a long way since it started out by creating an AR video game that enabled users to pretend they were flying Star Wars spacecraft rather than off-the-shelf drones.
“The market liked what we were doing and we had tens of thousands of downloads but we realised the technology had more potential for businesses and government departments.
“There’s cameras all over the place today and we’re able to help them visualise any type of data over a real-time video.
“Images could come from CCTV, a body camera, drone or even smartphone. We’ve implemented software that allows you to take any data feed and use computer algorithms to map the information over the video.”
During a time when there’s been growing uncertainty about the future of AR as pioneers like UK unicorn Blippar have collapsed, Adam says the tech’s uses in public safety and industry – broadcast media, automotive and agriculture are among other adopters – prove its here to stay.
How Edgybees helps save lives by using augmented reality in search and rescue missions
Edgybees worked with emergency services during Hurricane Irma in September 2017 and Hurricane Florence in September last year to provide flooded streets with geolocations.
It also assisted in the emergency response to the Californian wildfires last year.
Adam says: “We gave the ability to overlay the street names over flooded areas so rescue personnel could understand what they were looking at and identify where victims were located.
“The issue is that they get real-time video coming through from an aerial camera but they couldn’t understand which streets they were looking at.
“It becomes like a Google Maps from the air. A lot of the time when you get to your destination using that app, you’re trying to look at the map to understand if it’s the right building – and we’re overlaying this information so people know exactly where they need to go.”
As well as natural disasters, the tech has been used to help firefighters, ambulances and police in states including California, Texas and Florida during everyday emergencies to find the points or people of interest quicker than they would without AR.
“We can overlay the information to show the optimal path a car or helicopter might take to reach their destination quickest.
“We can also drop a pin for where first responders need to go to find a victim, as instructed by someone in a command centre.
“The victim can also be sent information through a text message. They’ll open a link that includes a map showing where the people who are coming to help them are located.
“A lot of people might think they’re in building number 13 but they’re actually in number 15, so it gives them a situational awareness.”
Augmented reality in military operations helps to avoid friendly fire
Perhaps the most crucial part of any military strategy is knowing where people are in the field.
This can be difficult in dim lighting and unfamiliar terrain, but Edgybees assists missions by highlighting soldiers’ locations and points of interest.
Adam says: “It’s important to give situational awareness so we can keep soldiers safe and ensure there’s no friendly fire.
“We’re working with some international defence organisations to visualise their planned mission over video footage – which might come from drones, helicopters, planes or surveillance footage – and understand if they’re in the right place.
“It’s similar to search and rescue and, instead of them looking at 30 screens in the command centre, they can look at one screen containing that information.”
Among Edgybees’ military clients are the US Air Force and Adam is keen to stress his company only works with western governments, and its technology is used to keep soldiers out of harm’s way rather than attacking targets.
Edgybees uses augmented reality in sports broadcasting to add another dimension for viewers
TV viewers of the PGA Championship 2018 golf tournament, held in Missouri last August and won by Brooks Koepka, were able to follow the trajectory of the ball when driven 300 yards down fairways by the world’s top pros using Edgybees’ AR software.
Adam says it’s a similar concept to the post-match analysis provided by TV pundits in football matches, which can follow a shot, cross or pass, but presented in real-time.
“When Tiger Woods is hitting the ball, you can now see where the golf ball is going as it’s in the air,” he says.
“Previously, broadcasters have shown us the perspective of the golfer and where the ball will land, but it’s only been a simulation and not in real-time.”
One of the main clients in this field is WarnerMedia-owned US broadcaster Turner Sports, while sailing is among the other sports that use AR to offer more value for TV viewers.
Broadcasters use wide-angle camera shots that make it difficult to identify teams and a racing course – merely showing a number of boats in a body of water – but Edgybees has been able to add visual aids to show teams, their positions in the race and a finishing line.
Gamification of driving and next-generation navigation in driving through augmented reality
With cars now fitted with numerous sensors providing an unprecedented amount of data, it gives the chance to add intelligence to the driving experience.
Edgybees could provide a simple visual layer of information on windscreens to offer instant clarity in complex or dynamic environments – helping with navigation and assistance for drivers.
“If you’re driving to a restaurant, a satellite navigation system might just take you to the general address but this software can overlay information to help you understand exactly which building it is when you arrive,” says Adam.
AR also has entertainment value through the “gamification” of driving.
Using the numerous cameras fitted around the vehicle and AR tech, child passengers can play app-based video games in the style of Mario Kart that mimic the car’s movements – but add extra interactive digital content such as coins to collect in the road and banana skins to throw in front of other virtual racing cars.
Going forward, there is also the potential for using AR in autonomous driving.
Augmented reality in enterprise: Edgybees growing into new industries
Other traditional industries could find a use for AR, too, and Edgybees’ portfolio also includes work with agricultural and energy companies to help them visualise infrastructure such as underground or underwater oil and gas pipes from the air.
Construction is another that could be explored in the future, Adam says.
He adds: “We’ve created a platform that’s able to geolocate data so the ability to have a visual technology like this has many use cases.
“Rather than just cool demos, we’re providing real value that customers are prepared to pay real money for.”
Blippar collapse doesn’t have to spell death knell for AR – as Edgybees proves business uses
Augmented reality’s entrance to the mainstream has been very recent and confined to a couple of use cases – namely the Pokémon Go video game, where players can “catch” franchise characters popping up in real-world environments with their smartphone, and the selfie filters for everything from puppy ears to demons on Snapchat.
But the tech was dealt a sizeable blow in December, when London tech unicorn Blippar collapsed into administration after a funding dispute.
The company, set up in 2011 by Ambarish Mitra and Omar Tayeb, had created an AR app that adds interactive digital content to everyday objects ranging from sports tickets to Heinz ketchup bottles – receiving more than $130m (£102m) in investment.
Some industry commentators suggested it was a huge blow for AR as investors began to view it as a “gimmick”, but Edgybees’ Adam believes his start-up has proved the enterprise potential for the tech.
The company employs about 25 staff and such is the growth in clients that he believes headcount could double by the end of the year.
This has been aided by raising $5.5m (£4.3m) from investors such as Motorola Solutions Venture Capital and Verizon Ventures in February last year, while there are plans to seek at least another $10m (£7.9m) this year.
Adam believes the development of AR glasses by companies such as US start-up Magic Leap will help move the technology forward in a similar way to how VR has been pushed on by devices like the Oculus VR headsets – although he doesn’t expect “the entire world to be wearing them – just certain segments”.
He adds: “We started in the video game space and to now be assisting first responders where we’re helping to save lives shows the technology doesn’t have to be a novel concept.
“It’s not just about Pokémon Go or putting filters on images – AR can really make a difference in real-world scenarios.”