3D and interactive knowledge articles, Virtual manuals for the Field Service Agents on field for quickly identifying the model - type - other configurations of the installed product that helps in finding and fixing the problem more easily, Equip the Health Cloud with Augmented Reality to empower the physicians with real-time analysis of three dimensional reports/anatomy, Creative 3D view and marketing of your real-estates for a realtime experience to your customers etc. These are just the tip of an iceberg for what can be achieved by Augmented Reality and Salesforce. So let us understand what is Augmented Reality a.k.a AR and how can it be made possible within Salesforce.

Augmented 3D Virtual manuals can empower the Field Service Agents on field to quickly identify the model - type - other configurations of the installed product that helps in finding and fixing the problem quickly
A physician can quickly pull 3D reports of a patient, do real time analysis and also explain the patient about the problems and the required treatment/surgery
 Sales Reps can give 3D and virtual tour of the real estate property to their customers guiding them for a better decision making. All these even before the projects completion.
Sales Reps can give 3D and virtual tour of the real estate property to their customers guiding them for a better decision making. All these even before the projects completion. 

So What is AR?

Augmented Reality is a technology which augments layers of two or more objects (2D/3D) on top of the existing reality (as seen through a camera of device mostly mobile, tablet etc) that gives an impression as if its real and as it is actually happening. There are SDKs available to develop AR apps and these SDKs take care of rendering the objects on your camera screen by making necessary communications with the hardware of the device based on the operating system - Android, iOS etc.

Besides the in-house AR SDKs from Apple and Google  below are some of the top SDKs and libraries available for AR. A detailed comparison of some these SDKs can be found here - an article by Elina Bessarabova

Unity - Unity
Unity is the ultimate game development platform. Use Unity to build high-quality 3D and 2D games, deploy them across mobile, desktop, VR/AR, consoles or the Web, and connect with loyal and enthusiastic players and customers.
Unreal Engine | The most powerful real-time 3D creation platform
Unreal Engine is the world’s most open and advanced real-time 3D creation platform for photoreal visuals and immersive experiences.
Vuforia: Market-Leading Enterprise AR | PTC
Learn why more market leaders trust the expansive capabilities of Vuforia’s Enterprise Suite to build AR solutions—and transform their business.
Wikitude Store: Find Best Pricing for your Augmented Reality Experiences.
Wikitude Store has purchase and pricing information for Augmented reality SDK. Licenses available on yearly or one-time fee. Enterprise packages on request.
EasyAR-Best engine for developing Augmented Reality
World-leading AR developing platform for its high efficiency, powerfulnessand easy to use. EasyAR Pro released in July offers improved API and workflow for Basic, improves compatibility, provides developers with desired AR underlying technology for meeting different needs
Kudan
XLsoft: Kudan AR SDK is a professional AR SDK for iOS and Android that can support marker or markerless tracking and location requirements.

Adding AR to Salesforce can be done in 2 ways.

  1. Utilize one of the above SDKs along with Salesforce Mobile SDK and develop a custom app that can combine the power of Salesforce features and AR
  2. The one I have tried; utilize a framework that harness the combination of WebGL and WebRTC technology that adds native support to browsers for realizing AR. Before we look at sample VF code that uses similar framework - jsartoolkit, let us see a quick demo.

Demo

In case the above Video isn't working in your place below is a short GIF demo

ARToolKit

ARToolKit is a software library for building AR applications

One of the key difficulties in developing any AR applications is the problem of tracking the users viewpoint that actually defines the perspective of the 3D object. In order to find this perspective, the application needs to know where the user is looking in the real world through the camera.

The ARToolKit video tracking libraries uses computer vision algorithms to calculate the real camera position and orientation relative to the physical markers in real time.


JSARToolKit

To avoid the dependency of installing a library for building AR applications I have used JSARToolKit which is a Javascript wrapper around ARToolKit. This library is also Marker based for the reasons explained above. Hence in the sample demo shown above, a Hiro card (which is nothing but a predefined marker image associated with a bitmap pattern) is to be shown to the Camera on top of which different 3D objects are rendered. The 3D objects are created using Three.js library - a sample of which we have seen in one of our previous article.

We can create our own custom markers with the image of our choice but more on that in the next article.

Visualforce Code

Code referred from example created by Lee Stemkoski.  More details on Marker and a walk-through of code in upcoming article.

<apex:page showHeader="false" sidebar="false">
    <html>
        <head>
            <meta name="viewport" content="width=device-width, user-scalable=no, minimum-scale=1.0, maximum-scale=1.0"/>
            <title>My First AR - inteygrate</title>
            <!-- include three.js library -->
            <script src='https://cdn.rawgit.com/stemkoski/AR-Examples/master/js/three.js'></script>
            <!-- include jsartookit -->
            <script src="https://cdn.rawgit.com/stemkoski/AR-Examples/master/jsartoolkit5/artoolkit.min.js"></script>
            <script src="https://cdn.rawgit.com/stemkoski/AR-Examples/master/jsartoolkit5/artoolkit.api.js"></script>
            <!-- include threex.artoolkit -->
            <script src="https://cdn.rawgit.com/stemkoski/AR-Examples/master/threex/threex-artoolkitsource.js"></script>
            <script src="https://cdn.rawgit.com/stemkoski/AR-Examples/master/threex/threex-artoolkitcontext.js"></script>
            <script src="https://cdn.rawgit.com/stemkoski/AR-Examples/master/threex/threex-arbasecontrols.js"></script>
            <script src="https://cdn.rawgit.com/stemkoski/AR-Examples/master/threex/threex-armarkercontrols.js"></script>
        </head>

        <body style='margin : 0px; overflow: hidden; font-family: Monospace;'>

            <script>

                var scene, camera, renderer, clock, deltaTime, totalTime;

                var arToolkitSource, arToolkitContext;

                var markerRoot1, markerRoot2;

                var mesh1;

                initialize();
                animate();

                function initialize()
                {
                    scene = new THREE.Scene();

                    let ambientLight = new THREE.AmbientLight( 0xcccccc, 0.5 );
                    scene.add( ambientLight );

                    camera = new THREE.Camera();
                    scene.add(camera);

                    renderer = new THREE.WebGLRenderer({
                        antialias : true,
                        alpha: true
                    });
                    renderer.setClearColor(new THREE.Color('lightgrey'), 0)
                    renderer.setSize( 640, 480 );
                    renderer.domElement.style.position = 'absolute'
                    renderer.domElement.style.top = '0px'
                    renderer.domElement.style.left = '0px'
                    document.body.appendChild( renderer.domElement );

                    clock = new THREE.Clock();
                    deltaTime = 0;
                    totalTime = 0;

                    ////////////////////////////////////////////////////////////
                    // setup arToolkitSource
                    ////////////////////////////////////////////////////////////

                    arToolkitSource = new THREEx.ArToolkitSource({
                        sourceType : 'webcam',
                    });

                    function onResize()
                    {
                        arToolkitSource.onResize()  
                        arToolkitSource.copySizeTo(renderer.domElement) 
                        if ( arToolkitContext.arController !== null )
                        {
                            arToolkitSource.copySizeTo(arToolkitContext.arController.canvas)    
                        }   
                    }

                    arToolkitSource.init(function onReady(){
                        onResize()
                    });

                    // handle resize event
                    window.addEventListener('resize', function(){
                        onResize()
                    });

                    ////////////////////////////////////////////////////////////
                    // setup arToolkitContext
                    ////////////////////////////////////////////////////////////    

                    // create atToolkitContext
                    arToolkitContext = new THREEx.ArToolkitContext({
                        cameraParametersUrl: 'https://cdn.rawgit.com/stemkoski/AR-Examples/master/data/camera_para.dat',
                        detectionMode: 'mono'
                    });

                    // copy projection matrix to camera when initialization complete
                    arToolkitContext.init( function onCompleted(){
                        camera.projectionMatrix.copy( arToolkitContext.getProjectionMatrix() );
                    });

                    ////////////////////////////////////////////////////////////
                    // setup markerRoots
                    ////////////////////////////////////////////////////////////

                    // build markerControls
                    markerRoot1 = new THREE.Group();
                    scene.add(markerRoot1);
                    let markerControls1 = new THREEx.ArMarkerControls(arToolkitContext, markerRoot1, {
                        type: 'pattern', patternUrl: "https://cdn.rawgit.com/stemkoski/AR-Examples/master/data/hiro.patt",
                    })

                    let geometry1   = new THREE.CubeGeometry(1,1,1);
                    let material1   = new THREE.MeshNormalMaterial({
                        transparent: true,
                        opacity: 0.5,
                        side: THREE.DoubleSide
                    }); 

                    mesh1 = new THREE.Mesh( geometry1, material1 );
                    mesh1.position.y = 0.5;

                    markerRoot1.add( mesh1 );
                }


                function update()
                {
                    // update artoolkit on every frame
                    if ( arToolkitSource.ready !== false )
                        arToolkitContext.update( arToolkitSource.domElement );
                }


                function render()
                {
                    renderer.render( scene, camera );
                }


                function animate()
                {
                    requestAnimationFrame(animate);
                    deltaTime = clock.getDelta();
                    totalTime += deltaTime;
                    update();
                    render();
                }

            </script>

        </body>
    </html>
</apex:page>

Augmented Reality =/= Virtual Reality

The below graphics tries to create the line of difference between AR and VR. Despite of this difference most AR libraries can also be used to develop VR applications because the actual difference is just about whether a virtual object is felt in a real environment (AR) or a real person is felt in a virtual environment (VR).

Image Credit https://blog.vakoms.com/
You've successfully subscribed to inteygrate
Welcome back! You've successfully signed in.
Great! You've successfully signed up.
Success! Your account is fully activated, you now have access to all content.