In this tutorial, we’ll be discussing Augmented Reality.

Overview

According to Wikipedia, ARCore is a software development kit developed by Google that allows for augmented reality applications to be built.

ARCore uses three key technologies to integrate virtual content with the real environment:

Motion Tracking: it allows the phone to understand its position relative to the world.
Environmental understanding: This allows the phone to detect the size and location of all type of surfaces, vertical, horizontal and angled.
Light Estimation: it allows the phone to estimate the environment’s current lighting conditions.

ARCore is a Google platform that enables your applications to “see” and understand the physical world, via your device’s camera.

In our app/build.gradle file, we can add the dependency for Sceneform to update the project. Although we’ll be writing this tutorial in Kotlin, Sceneform uses some language constructs from Java 8, so we’ll need to explicitly add support since our minimum API is less than 26.

   implementation “com.google.ar.sceneform.ux:sceneform-ux:1.4.0”

1	implementation “com.google.ar.sceneform.ux:sceneform-ux:1.4.0”

add project lable dependency

classpath 'com.google.ar.sceneform:plugin:1.0.1'

1	classpath 'com.google.ar.sceneform:plugin:1.0.1'

you need to set the ar plugin in your app’s Gradle file as well.
Add the following below the dependencies:

apply plugin: 'com.google.ar.sceneform.plugin'

1	apply plugin: 'com.google.ar.sceneform.plugin'

We need to permission for camera in Android Manifest.

<uses-permission android:name="android.permission.CAMERA" />
    <uses-feature
        android:name="android.hardware.camera.ar"
        android:required="true" />

<uses-permission android:name="android.permission.CAMERA" />

<uses-feature

android:name="android.hardware.camera.ar"

android:required="true" />

Adding our Model

To make life easier, and to allow us to import our own 3D assets, we’ll also include the Sceneform plugin for Android Studio. You can install this by selecting preferences, plugins, browse repositories, and search for Google Sceneform Tools (Beta)

We can download the Google Sceneform Tools plugin in our Android Studio to view and render the 3d models.

You can go to https://poly.google.com/ and download a sample model. Don’t forget to credit the creator!

Typically, the OBJ and GLTX formats are used for rendering augmented images.

Now let’s build our first AR application where we’ll use the above 3D model as our AR image.

Now right click on app and create simple directory.

copy and paste all files that you download.

Then right click on obj file and click on “import sceneform assets”

Let’s code

<?xml version="1.0" encoding="utf-8"?>
<android.support.constraint.ConstraintLayout xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    xmlns:tools="http://schemas.android.com/tools"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    tools:context=".MainActivity">

    <fragment
        android:id="@+id/sceneform_fragment"
        android:name="com.google.ar.sceneform.ux.ArFragment"
        android:layout_width="match_parent"
        android:layout_height="match_parent"
        app:layout_constraintBottom_toBottomOf="parent"
        app:layout_constraintEnd_toEndOf="parent"
        app:layout_constraintStart_toStartOf="parent"
        app:layout_constraintTop_toTopOf="parent" />

    <android.support.design.widget.FloatingActionButton
        android:id="@+id/floatingActionButton"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:layout_marginBottom="8dp"
        android:layout_marginEnd="8dp"
        android:layout_marginStart="8dp"
        app:layout_constraintBottom_toBottomOf="@+id/sceneform_fragment"
        app:layout_constraintEnd_toEndOf="parent"
        app:layout_constraintStart_toStartOf="parent"
        app:srcCompat="@drawable/ic_explore_white_24dp" />

</android.support.constraint.ConstraintLayout>

<?xml version="1.0" encoding="utf-8"?>

<android.support.constraint.ConstraintLayout xmlns:android="http://schemas.android.com/apk/res/android"

xmlns:app="http://schemas.android.com/apk/res-auto"

xmlns:tools="http://schemas.android.com/tools"

android:layout_width="match_parent"

android:layout_height="match_parent"

tools:context=".MainActivity">

<fragment

android:id="@+id/sceneform_fragment"

android:name="com.google.ar.sceneform.ux.ArFragment"

android:layout_width="match_parent"

android:layout_height="match_parent"

app:layout_constraintBottom_toBottomOf="parent"

app:layout_constraintEnd_toEndOf="parent"

app:layout_constraintStart_toStartOf="parent"

app:layout_constraintTop_toTopOf="parent" />

<android.support.design.widget.FloatingActionButton

android:id="@+id/floatingActionButton"

android:layout_width="wrap_content"

android:layout_height="wrap_content"

android:layout_marginBottom="8dp"

android:layout_marginEnd="8dp"

android:layout_marginStart="8dp"

app:layout_constraintBottom_toBottomOf="@+id/sceneform_fragment"

app:layout_constraintEnd_toEndOf="parent"

app:layout_constraintStart_toStartOf="parent"

app:srcCompat="@drawable/ic_explore_white_24dp" />

</android.support.constraint.ConstraintLayout>

MainActivity class

class MainActivity : AppCompatActivity() {

    private lateinit var arFragment: ArFragment

    private var isTracking: Boolean = false
    private var isHitting: Boolean = false

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(R.layout.activity_main)

        arFragment = sceneform_fragment as ArFragme
        arFragment.arSceneView.scene.addOnUpdateListener { frameTime ->
            arFragment.onUpdate(frameTime)
            onUpdate()
        }

        floatingActionButton.setOnClickListener { addObject(Uri.parse("NOVELO_EARTH.sfb")) }
        showFab(false)

    }

    private fun showFab(enabled: Boolean) {
        if (enabled) {
            floatingActionButton.isEnabled = true
            floatingActionButton.visibility = View.VISIBLE
        } else {
            floatingActionButton.isEnabled = false
            floatingActionButton.visibility = View.GONE
        }
    }

    // Updates the tracking state
    private fun onUpdate() {
        updateTracking()
        // Check if the devices gaze is hitting a plane detected by ARCore
        if (isTracking) {
            val hitTestChanged = updateHitTest()
            if (hitTestChanged) {
                showFab(isHitting)
            }
        }

    }

    // Performs frame.HitTest and returns if a hit is detected
    private fun updateHitTest(): Boolean {
        val frame = arFragment.arSceneView.arFrame
        val point = getScreenCenter()
        val hits: List<HitResult>
        val wasHitting = isHitting
        isHitting = false
        if (frame != null) {
            hits = frame.hitTest(point.x.toFloat(), point.y.toFloat())
            for (hit in hits) {
                val trackable = hit.trackable
                if (trackable is Plane && trackable.isPoseInPolygon(hit.hitPose)) {
                    isHitting = true
                    break
                }
            }
        }
        return wasHitting != isHitting
    }

    // Makes use of ARCore's camera state and returns true if the tracking state has changed
    private fun updateTracking(): Boolean {
        val frame = arFragment.arSceneView.arFrame
        val wasTracking = isTracking
        isTracking = frame.camera.trackingState == TrackingState.TRACKING
        return isTracking != wasTracking
    }

    // Simply returns the center of the screen
    private fun getScreenCenter(): Point {
        val view = findViewById<View>(android.R.id.content)
        return Point(view.width / 2, view.height / 2)
    }

    private fun addObject(model: Uri) {
        val frame = arFragment.arSceneView.arFrame
        val point = getScreenCenter()
        if (frame != null) {
            val hits = frame.hitTest(point.x.toFloat(), point.y.toFloat())
            for (hit in hits) {
                val trackable = hit.trackable
                if (trackable is Plane && trackable.isPoseInPolygon(hit.hitPose)) {
                    placeObject(arFragment, hit.createAnchor(), model)
                    break
                }
            }
        }
    }

 
    private fun placeObject(fragment: ArFragment, anchor: Anchor, model: Uri) {
        ModelRenderable.builder()
                .setSource(fragment.context, model)
                .build()
                .thenAccept {
                    addNodeToScene(fragment, anchor, it)
                }
                .exceptionally {
                    Toast.makeText(this@MainActivity, "Error", Toast.LENGTH_SHORT).show()
                    return@exceptionally null
                }
    }

  
    private fun addNodeToScene(fragment: ArFragment, anchor: Anchor, renderable: ModelRenderable) {
        val anchorNode = AnchorNode(anchor)
        // TransformableNode means the user to move, scale and rotate the model
        val transformableNode = TransformableNode(fragment.transformationSystem)
        transformableNode.renderable = renderable
        transformableNode.setParent(anchorNode)
        fragment.arSceneView.scene.addChild(anchorNode)
        transformableNode.select()
    }
}

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

class MainActivity : AppCompatActivity() {

private lateinit var arFragment: ArFragment

private var isTracking: Boolean = false

private var isHitting: Boolean = false

override fun onCreate(savedInstanceState: Bundle?) {

super.onCreate(savedInstanceState)

setContentView(R.layout.activity_main)

arFragment = sceneform_fragment as ArFragme

arFragment.arSceneView.scene.addOnUpdateListener { frameTime ->

arFragment.onUpdate(frameTime)

onUpdate()

}

floatingActionButton.setOnClickListener { addObject(Uri.parse("NOVELO_EARTH.sfb")) }

showFab(false)

}

private fun showFab(enabled: Boolean) {

if (enabled) {

floatingActionButton.isEnabled = true

floatingActionButton.visibility = View.VISIBLE

} else {

floatingActionButton.isEnabled = false

floatingActionButton.visibility = View.GONE

}

// Updates the tracking state

private fun onUpdate() {

updateTracking()

// Check if the devices gaze is hitting a plane detected by ARCore

if (isTracking) {

val hitTestChanged = updateHitTest()

if (hitTestChanged) {

showFab(isHitting)

}

// Performs frame.HitTest and returns if a hit is detected

private fun updateHitTest(): Boolean {

val frame = arFragment.arSceneView.arFrame

val point = getScreenCenter()

val hits: List<HitResult>

val wasHitting = isHitting

isHitting = false

if (frame != null) {

hits = frame.hitTest(point.x.toFloat(), point.y.toFloat())

for (hit in hits) {

val trackable = hit.trackable

if (trackable is Plane && trackable.isPoseInPolygon(hit.hitPose)) {

isHitting = true

break

}

return wasHitting != isHitting

}

// Makes use of ARCore's camera state and returns true if the tracking state has changed

private fun updateTracking(): Boolean {

val frame = arFragment.arSceneView.arFrame

val wasTracking = isTracking

isTracking = frame.camera.trackingState == TrackingState.TRACKING

return isTracking != wasTracking

}

// Simply returns the center of the screen

private fun getScreenCenter(): Point {

val view = findViewById<View>(android.R.id.content)

return Point(view.width / 2, view.height / 2)

}

private fun addObject(model: Uri) {

val frame = arFragment.arSceneView.arFrame

val point = getScreenCenter()

if (frame != null) {

val hits = frame.hitTest(point.x.toFloat(), point.y.toFloat())

for (hit in hits) {

val trackable = hit.trackable

if (trackable is Plane && trackable.isPoseInPolygon(hit.hitPose)) {

placeObject(arFragment, hit.createAnchor(), model)

break

}

private fun placeObject(fragment: ArFragment, anchor: Anchor, model: Uri) {

ModelRenderable.builder()

.setSource(fragment.context, model)

.build()

.thenAccept {

addNodeToScene(fragment, anchor, it)

}

.exceptionally {

Toast.makeText(this@MainActivity, "Error", Toast.LENGTH_SHORT).show()

return@exceptionally null

}

private fun addNodeToScene(fragment: ArFragment, anchor: Anchor, renderable: ModelRenderable) {

val anchorNode = AnchorNode(anchor)

// TransformableNode means the user to move, scale and rotate the model

val transformableNode = TransformableNode(fragment.transformationSystem)

transformableNode.renderable = renderable

transformableNode.setParent(anchorNode)

fragment.arSceneView.scene.addChild(anchorNode)

transformableNode.select()

}

Ashwani Yadav 2 December 2020

. . .

What is Ar Model in Android

Overview

Adding our Model

Leave a Comment Cancel Reply

What is Ar Model in Android

Overview

Adding our Model

Leave a Comment Cancel Reply

Message Sent!