An introduction to Augmented Reality (AR) including: key industry players; latest developments; and the AR ecosystem. The key strategic question is what role should telcos play? Three possible roles are outlined. (February 2011, Executive Briefing Service)
This Analyst Note covers:
- Augmented Reality Today
- What is AR?
- Why AR Now?
- AR Ecosystem Roles
- AR Browsers
- AR Application Examples
- Telcos’ “toes in the water”
- Three Alternate Visions of Telcos’ Role in AR
- Conclusions and Next Steps
Augmented Reality Today
Augmented Reality (AR) is a hot topic right now, attracting much of the hype that was reserved for apps a few years ago. But what’s beyond the hype? Is AR simply the next stage of development for existing value propositions, or will it bring with it entirely new propositions that offer new revenue streams for the Telecoms ecosystem?
In this Analyst Note, we explain what Augmented Reality is, why it is developing so quickly now and lay out three possible roles that telcos could develop in order to monetise the AR explosion in ways they haven’t with apps in general.
This analysis was written in collaboration with Christine Perey, Perey Research and Consulting, a world expert in the AR field, who is working in association with Telco 2.0 at our forthcoming AMERICAS, EMEA and APAC Executive Brainstorms and on further research publications.
What is AR?
In simple terms, ‘Augmented Reality’ applications and technologies bring users information that exists in the digital world and presents it automatically and intuitively in association with things in the real, or physical, world. Often, but not always, this information is from the web.
AR is about creating, making explicit and displaying the relationships between the real and virtual worlds.
At the highest level, AR can be seen as the latest evolution in information search, viewing and, ultimately, to its manipulation by the user. It’s part of an evolution: the Web made Internet-based information accessible to audiences through websites and, when there proved to be too many web sites, with search engines. Search brought information access into the reach of everyone, everyday on a separate screen. With AR, digital information can be automatically ‘connected’, in context, to real world objects.
This can take many different structures but is best described as a three-stage process.
- Sensors in the user’s device (such as camera, GPS and/or compass, microphone, even a thermometer) detect a ‘condition’ in the local environment – say the visual image of a recognisable geographic feature like a mountain or famous building.
- The application or ‘system’ finds the digital data (any text, image, 3D model, video, URL, sound, etc.) that has previously been associated with the specific condition in the local environment identified by the sensor.
- The digital information is presented visually (or aurally) to the user in such a way that it is ‘synchronized’ with the real world. It is only presented as long as the condition remains the same. When the user’s condition changes, e.g. they move away from the object, the digital information disappears or is replaced with new information if the sensors are triggered by another condition.
This is illustrated in the example below.
Figure 1 – Sensors triggered by the environment deliver relevant data
Source: PEREY Research and Consulting
For a service to be seen as Augmented Reality, all three of these steps have to take place. To illustrate some differentiating features of AR services, let’s compare two services: first without AR, then with AR.
- Non-AR Location Based Applications Example. A basic GPS location service detects where you are. You then search your smartphone for information about restaurants nearby and these appear as points on a map around you. These are augmentations – virtual information with the real data that you input. The processes are neither automatic nor seamless..
- AR Location Based Applications Example. The sensors of the Wikitude Drive AR service detect your location and direction of travel. To direct you, arrows are automatically generated and ‘registered’ on a live in-car video of the road ahead of you (not on a map view). Instead of a manual search, the ‘Yelp’ mobile AR application takes location information from the GPS and compass, identifies the local businesses in its database which are nearby, and returns local information that is overlaid on the live video where the business is visible.
‘Augmented Reality’ can be seen as sitting on a continuum with ‘Reality’ at one end and ‘Virtual Reality’ on the other. Augmentation can be used both ways – to augment the virtual world with real objects and to augment the real world with virtual data. An example of ‘Augmented Virtuality’ is a soldier training environment in which the “world”, be it urban or forest, is synthesised and virtual, and where there are other real, ‘flesh and blood’ people operating at the same time. The soldier sees both the virtual world and the real people in one view. To help bring this to life we have annotated Milgram’s Continuum with some examples below.
Figure 2 – Milgram’s Continuum from real to virtual
Why AR Now?
AR is not a new development. In fact, it’s been studied in one dimension or another in labs for over 20 years. However, the opportunities that emerged around mid-2009 have given the subject appeal and many practical, real-world applications are now possible. The most important developments have been the release of new sensor-laden and processor rich smartphones with touch interfaces connected to cloud services by faster networks.
In addition to enhancements in computing power, devices are increasingly pre-packed with GPS, compass, accelerometers and gyroscopes, adding to the now ubiquitous cameras and microphones. Thermometers, RFID and other wireless sensors are also appearing. At the same time a critical amount of information is available in digital format, meaning the potential for bringing the real and virtual worlds together through a mobile device is huge.
- Other important developments adding momentum to Augmented Reality include:
- Improved computer vision algorithms increasing the reliability with which the feed from a camera can be matched with images in a database;
- Real time information is increasingly becoming the default, especially “local” (LBS) when the user is in transit and social information being updated in real time;
- Contextual services are better understood as having value to end users (e.g., get a coupon when you enter a store, not after you have left);
- The emergence of 3D (on the web, in cinema and on TV) is improving 3D technology that lends itself well to AR for visualization of information over 3D objects.
Combined, these trends have led to an upsurge in AR activity including by mobile operators. However, to date, this has been piecemeal and predominantly focused on customer acquisition and promotional activities. For example, Orange UK launched a free iPhone app and AR service for the Glastonbury Festival and others have released similar apps around special events.
So, what kinds of companies are the most active in Augmented Reality?
AR Ecosystem Roles
The Mobile AR Ecosystem is composed of companies each occupying one or more of five segments:
- Content Publishers;
- Enabling Technology Providers;
- Packaging Companies;
- and Distribution/Discovery Providers.
Each of these segments contributes a valuable role in the process of connecting the physical world and the digital world. As shown in the figure 3, we also see that the user’s own content, as it is published to social networks, could be inserted into the system as well, provided that the platform of an application developer can receive the User Generated Content (UGC) through an open API.
Figure 3 – Mobile AR Ecosystem
Figure 4 provides a summary of the AR market segments, their business models, and illustrative examples.
Figure 4 – Mobile AR Ecosystem – Examples of Players
| Role in mobile AR value chain
| Business Model(s) for the player
| Example companies
|Paying for advertising to promote engagement with customer, prolong/raise brand awareness and increase sales.||H&M, McDonalds, Stella Artois, Starbucks, Volkswagen, Kia, Coca Cola, Walt Disney Co and dozens of others.
|Content – providers and publishers
Creators and publishers of original digital content used to augment a real image e.g. Travel guides, educational publications, listings
|Service revenue share or licence fee for content; paying for advertising.
|Lonely Planet, Esquire Magazine, Yell.com (Yell has also launched its own AR browser building on its role as a content aggregator), newspaper publishers, Twitter (TwittARound, etc)
|Enablers – technologies that enable AR at either the server or device end
Hardware: cameras, accelerometers, GPS, gyroscopes, solid state compasses, RFID, wireless sensors, head mounted displays; handheld displays; Location technologies – GPS, A-GPS;
Software: image registration technologies, recognition technologies,
|Product/service sale or end service revenue share.
Hardware-based – component sales, promotion of mobile device sales.
Software: free, licensing or revenue share.
|Hardware: Device manufacturers – Nokia, Sony Ericsson, Apple, Motorola; Sensors – Aero Electronics Operations
Specialized hardware (graphics processors, sensors): Qualcomm, Intel
Software: Recognition technologies – Google – Goggles; Polar Rose (facial); SREnginez (architecture), Software Development Kits for specialized silicon (Qualcomm AR SDK).
Alcatel-Lucent has invested in mobile AR technologies in visual recognition (see its AR vision video here), and spun-off its project a year ago. The company which they acquired called Mobile News Channel (MNC) now has a division called dekaps which contracts with operators for custom projects using AR.
UI – ‘TAT’ The Astonishing Tribe – acquired by RIM.
|Packaging – providing the publishing environments to app developers and communities who are then putting the AR app together with content
App Developers are also part of the packaging segment of the ecosystem
|Service/advertising revenue share;
Charging developers for placements;
Charge to develop, for the end service or ad funded.
|Platforms: Layar; Mobilizy; metaio, Total Immersion.
There are thousands of app developers who could be using AR platforms and enablers.
|Discover and Delivery – promoting AR to end users, reducing barriers to the discovery (e.g., pre-loading apps), distributing AR applications through app stores
|Revenue sharing, driving data revenues, sales of more powerful handsets in bundles with data.
|Mobile Network operators: Orange, NTT DoCoMo, KDDI.
Handset manufacturers: Nokia, Samsung, HTC.
AppStores: Apple, Android Marketplace.
AR browser and platform providers are the leaders in the ‘packaging’ segment of Figure 3, both aggregating AR content and displaying it in a way that makes it usable for the mass market. Their key role is to mediate the interface between the user’s context and the digital data available.
Since 2009, a number of AR browsers have appeared and there are now more than six to choose from, though only three have significant market share. These AR browsers are in and of themselves applications designed with unique user experiences and into which others, the content providers, can publish their data in targeted “layers” or “channels”. End users choose the layer or channel they wish to see.
In the case of Layar and Wikitude, the platform is also available to those who seek to develop custom, dedicated apps. The content appears in an AR browser or embedded in a standalone application.
The leading AR platform and AR browser is provided by Layar, a young Dutch company that launched in June 2009. Layar’s free AR browser is the most popular with over 3.5 million users as of January 2011, of which a million are active monthly. In addition, Layar’s AR browser has been pre-loaded on over 10 M devices to date, making it easy for potentially more subscribers to discover it quickly. The company raised $14 million in its latest round of funding in November 2010.
Another of the top 3 providers, and the oldest, in terms of conceiving of a browser for AR, Mobilizy, provides its Wikitude browser. Launched in the winter of 2008-2009, Wikitude has roughly a million users, but it does not publish monthly active figures. The company has also received outside investment in the 4th quarter 2010, and while it is behind in the AR browser race, the company has taken a different strategy. The Mobilizy team has invested heavily in better understanding and moving its platform to meet the needs of those who will use AR for navigation. The Wikitude Drive application and service for smartphones uses Navteq data to not only show the arrows overlaid on the real world, but also to speak/talk to the user who is driving. Wikitude has also begun, in early 2011, to solicit more developers for its platform, playing catch up with respect to both Layar and metaio in this respect.
The third popular AR browser, Junaio, has around 500,000 users. The Junaio Glue platform is provided by metaio, one of the original AR platform companies that pre-exists the mobile AR frenzy. Metaio, a Munich-based company with offices in the US and Korea, has the distinct advantage of combining both geo-location as a source of context for the user, but also visual recognition. Metaio’s image recognition algorithms make its browser suitable for more indoor applications and services than those browsers which rely solely on GPS. Further, metaio’s combination of technologies make it attractive for those companies which wish to use mobile AR for interaction with packaging and printed media campaigns without having a dedicated application. Junaio is open and its developer community is growing rapidly.
The key to success for all of these companies is their ability to attract and support developer communities. As of October 2010, Layar had over 5,000 developers, some of which are partners and promoted as such to content owners looking for developers to create layers for them if they can’t do it for themselves. Layers can be free or paid for, providing content owners and developers with the chance to monetise their developments.
These aggregators are growing fast and the mobile AR ecosystem is building up around them and appstores, not telcos, so even at this early stage, the potential for telcos to take a complete hold of the platform is limited, unless they outright acquire the companies.
AR Application Examples
Creating AR applications is a specialised activity and requires special tools and skills. Some of the tools are open source, others available to certified developers under a wide range of conditions.
Today, the majority of AR experiences on the desktop are provided to consumers via specialized companies who design special experiences for brands to attract consumer attention. There are also innovative developers who already own databases and choose to use AR as an alternative way to “view” their data. A handful of companies are dedicated to specific AR application segments such as games, tourism services and shopping. And, finally, some creative artists use AR to entertain, provoke and make a name for themselves.
The majority of current mobile AR applications follow the standard free, freemium and occasionally paid for models of the app marketplaces they are being distributed through.
The following are indicative of the types of services that are available for end users to experience AR today.
Figure 5 – Examples of AR Apps
|Service Name||Differentiation|| Requirements
|Takes the concept of location and store offers to a new level and removes the intrusion of text alerts from shops as you walk by. Using the camera as the sensor trigger, it displays discounts available in surrounding stores, restaurants etc.
|Android app available in the UK.
|Nesquik Factory AR Game
|Combines AR and gesture recognition in a promotional game for Nesquik. Part of a new generation of advertising.
|Flash-based and requires a camera.
|H&M Promotional Vouchers
‘GoldRun’- – see here for more
|AR game in which users search for and, using their camera and the application, collect virtual objects in order to access discount vouchers.
|Currently available in the US and targeted at New York with an Android app expected in 2011.
|Coke Print Advertisement
|Using a mobile phone camera as the trigger, a print ad becomes live and provides more information, offers etc. IT doesn’t just link to a website but provides relevant and real-time information. See here.
|iPhone and Android ‘Channel’ available in Germany with Metaio’s Junaio AR browser.
|mtrip City Guides
|AR city guides that identifies points of interest using the camera of a mobile phone as the sensor trigger (see here).
|iPhone and access to iTunes AppStore.
|WordLens by Visual Quest
|Overlays the translation of the English text in Spanish and vice versa.
|iPhone and access to iTunes AppStore.
In terms of the telecom industry’s position in the market, device and chipset manufacturers are currently most active, seeing AR as a means to drive differentiation and demand for even more powerful processors and devices.
Qualcomm is the most recognisable of the active players, having acquired a small Austrian development company in early 2010 and launched its own AR SDK in Q3 2010. Its model is to use AR to bolster the sale of its chipsets and although it supports new ways to monetise AR, this is not its major driver. Intel Capital has made a $13.4M investment in Layar as it also perceives AR as driving the requirement for faster processors.
Deutsche Telekom has also spun out a separate business unit to develop heads up displays for AR to be introduced later this year, not something that telcos are likely to do.
Telcos’ “toes in the water”
Where some telcos are entering the market, they are doing so as application or software developers, or promoting the services developed by a third party. For example, Bouygues Telecom in France released the first in-house operator-developed mobile AR look up service <Ici Infos> in November 2009 with over 900,000 unique points of interest (POI), while Telefonica has a group in Barcelona R&D which is working on its own visual search technology. NTT DoCoMo offers its smartphone subscribers the intuitive navigation services “chokkan nabi” developed under contract for DoCoMo devices.
However, just as with the broader applications market, third parties are often better placed to create the apps themselves.
In general, AR is moving more quickly than most operators can track. At the AR research community’s annual event, the International Symposium for Mixed and Augmented Reality (ISMAR) in October 2010, the announcements and proposals included:
- A system for multi-viewer 3D visualization that is extremely easy and inexpensive to implement using a simple monitor;
- Developments on how to track the user’s gaze and resolve the true shape of non-rigid objects (e.g., a paper map surface);
- Developments in the area of real time sensor fusion that increase the reliability of trigger detection;
- The ability to “cover” or disguise the existence of a marker in a video image in real time (this is extremely useful for reducing the end user aversion to markers and increasing the reliability of tracking).
We would argue that telcos are unlikely to corner the market by seeking to develop a key technology. Instead, they should look to leverage their position as the discovery and delivery segment, an essential part of the ecosystem for subscribers to discover many AR experiences. However, while this differentiates telcos from other market segments and generates greater sales of higher margin devices, it doesn’t offer great monetization directly from AR.