Augmented Reality (AR) in Financial Services


Who would believe that Steven Spielberg’s high-tech futuristic shopping mall concept in the movie Minority Report, where Tom Cruise’s character is confronted with digital signs that call out his name when he passes them, would one day be brought to the real world by NEC and IBM, who are separately developing facial recognition advertising billboards. Recently, Google’s project Glass, which is developing an augmented reality head-mounted device (HMD), has made the tech world sit up and take notice. the retail industry has seen a spurt of development around augmented reality with Tesco being at the forefront. The real estate and the advertising industries have been quick to embrace augmented reality applications to improve sales and customer outreach. In financial services, augmented reality has not been able to make much headway but the time is ripe for companies to leverage augmented reality technologies to deliver cutting-edge solutions to customers.


Augmented reality, in simple terms, is the use of diverse to enhance and improve our view of the real world. Augmented reality is a direct or indirect representation of physical, real-world surroundings whose components are enlarged by system-generated sound, video, graphics, etc. For example, telecasting a cricket match with live scores, images etc. on the screen. this is actually considered to be the opposite of virtual reality, wherein people involve themselves to an extent that they exceed the boundaries of physical reality and have got nothing to do with real world events. The basic scheme of augmented reality is to overlay graphics, audio and other sense enhancements over a real-world environment in real-time.

What benefits can financial institutions derive from adopting AR?

Augmented reality can benefit financial services which are embracing the mobile and social media revolution. Financial transactions are increasingly being executed through mobile devices. There is an ever-growing user base for location-based services and presents a huge opportunity for financial institutions. With mobiles becoming advanced, sensors through AR applications, financial firms can generate direct benefits for consumers, businesses as well as workers.

Technology used in AR

Augmented reality technology with smartphones is drawing a lot of attention these days By using specialized software, users can turn their iPhone, Android or other smartphones into a virtual heads up display.

All a user has to do is point his phone’s camera at any point/location/building and the relevant information is displayed on the phone’s screen as graphics. Most of the developers of mobile AR applications are expecting AR to become the next “Big thing” in the market.

AR apps, like World Surfer (for smartphones like the iPhone and high-end Android-based devices) provided by GeoVector and Mobilizy’s Wikitude World Browse, provide users with a better way to interact with different places in their surroundings.

How does an AR app work?

  1. AR app reads the GPS data on the phone to find the current location device
  2. The app then governs the phone’s positioning from its electronic compass to decide the direction it is pointing at
  3. The app then looks into its database for text, hyperlinks, pictures, etc. that have been tagged to that location in the compass direction from the current location
  4. If it finds anything relevant, then it positions them over the image of the object on the phone’s screen. For example, If it is a restaurant, the app displays it’s operating time, menu, directions, etc.)
  5. The AR app usually first displays the required nearest place towards the pointed, followed by other places that are at a distance

For all this to happen, a high-end phone with good battery backup is required because the graphics and GPS technology used in these phones consume lots of battery charge.

This technology can be applied in any situation where the user needs to locate a place in his surroundings. Users trying to locate banking channels can benefit a lot by using augmented reality applications. are now looking forward to use this technology to enhance their experience with the users and to pull in more customers to their door step. this technology can be best utilized by banks in their consumer, commercial banking and Capital Markets businesses.

Strategy to implement AR applications

Anticipate – instead of responding to customer needs, financial firms need to anticipate them. Firms need to understand the customer’s behavior and complexity and understand if the AR strategy is aligned with business goals.

Build – build the product with a team of developers and business managers. Augmented reality ideas can make good business sense if a proper analysis is done.

Launch pilot – the AR application should be launched on a pilot-basis initially for testing purposes.

Get feedback – get feedback from customers as well as the sales force. Do a cost-benefit analysis.

Broader deployment – go for a broad deployment with a marketing push. Customers need to be made aware of how to use the application.

Application of AR in financial institutions

Customers using web and mobile applications expect a compelling user experience that is usable, useful and attractive. Augmented reality applications can be leveraged to improve the existing business scenario and even create new business cases.

The various scenarios where augmented reality can be applied in the financial services industry:


How can AR benefit your financial institution?

Better cross-sell of opportunities

Ability to identify and blend real-world into cyberspace can help track your bank cross-sell various credit products in a highly targeted fashion. For example, a customer waiting at an airport to check-in for a flight can quickly point their phone at the nearest image of their bank to buy a travel insurance policy.

Reduced customer support costs

With augmented reality, customers can easily locate and self-serve traditional low-value transactions or non-transactions without calling customer support. For example, locating the nearest branch or ATM, determining the working hours of a bank, etc.

Personalized service

At a time when banking services are considered exchangeable/replaceable, augmented reality can help a bank significantly personalize a customer’s experience, like showing the nearest suppliers to a business partner, etc.

Increased retention and potential for lead generation

Augmented reality offers a highly effective way to capture the customer’s attention, resulting in greater dwell at one place.

AR Initiatives by financial institutions


PayPal AR

PayPal has just been granted a patent for displaying information on products through augmented reality. The patent, titled Augmented Reality View of Product Instructions, was filed last January and would enable users to see relevant information about a product using an AR headset or device.

The patent asserts that part of the goal is to help people figure out how to assemble and control the (increasingly complex) products that are purchased. 

“Today’s products are increasingly complex to setup and operate. Often, the products are accompanied with detailed instructions that may be daunting for a user to follow. In other cases, a user may discard instructions after an initial set up of a product.”

Thus, the goal is to offer people assistance so that they can actually enjoy the products that they get, as opposed to fight with them for hours:

“In each of these cases, the user’s ability to operate the product may be hampered by a lack of information about the product or an inability to use the product. This in turn may affect the user’s experience with the product and the user’s perception of the manufacturer of the product.”

The information includes product reviews, instructions, and recommended accessories.

With this patent, PayPal is looking to help make purchasing flows easier for consumers. The technology would presumably be able to identify objects and pull all pertinent information from the web to overlay onto the AR display.

Patent Preparation

The patent also hints at this being just the beginning. The technology has the potential to identify any physical object, not just those purchased with PayPal, and immediately get information on it (including how to purchase it online, most likely through PayPal).

For now, that’s just a pipe dream, seeing as this is only a patent and patents are filed all the time for technology that might never see the light of day.

One major hurdle in the way to making this product possible is the current limitations in computer vision. The ability to identify and recognize objects to get information on them is still being developed, but it is making huge leaps. So who knows what the (augmented) future will really look like.


Augmented reality is a technology ready to explode and banks and insurance companies need to adopt it early. Financial institutions need to have a strong creative angle to apply augmented reality concepts to customer-centric applications.  Prototypes should be tested in select markets before a wider launch. Mobile banking and social media had their critics but they did not take long to become a part of most people’s lives. Augmented reality is not very different and needs to be prioritized and brought in-sync with the business and technology strategy of financial institutions. With smartphones acting as a catalyst it presents an opportunity like never before.