AR core with glb/gltf model using Sceneform

The 3D model can in the form of GLB or GLTF format and these 3D models can be loaded at runtime without conversion. Which is an improvement in the flexibility of the models rendered in the application.

In this blog, we will be using the GLB format because the GLTF model is dependent on there .bin file which has to be saved with its corresponding file but a GLB file contains a 3D model saved in the GL Transmission Format (glTF). It stores information about a 3D model, such as node hierarchy, cameras, and materials, animations, and meshes in binary format. GLB files are the binary version of .gltf files.

Some important links for the AR model

1:Conversion from .gltf to .glb online :

Copy all the files from the model(.gltf and .bin file) and drag on this link: Click here

2: Conversion from .obj to .glb offline using node.js: Click here

3: To get the sample models: Click here

Let’s get it into code

1. In your project build.gradle dependencies:

classpath 'com.android.tools.build:gradle:3.6.3'

1	classpath 'com.android.tools.build:gradle:3.6.3'

2. In your app build.gradle dependencies:

implementation 'com.google.ar.sceneform.ux:sceneform-ux:1.15.0'
implementation 'com.google.ar.sceneform:assets:1.15.0'

1 2	implementation 'com.google.ar.sceneform.ux:sceneform-ux:1.15.0' implementation 'com.google.ar.sceneform:assets:1.15.0'

3. In the Android Manifest.xml file:

    <!-- Features -->
    <uses-feature 
        android:name="android.hardware.camera"
        android:required="false"  />
    <uses-feature
        android:name="android.hardware.camera.autofocus"
        android:required="false" />

    <uses-feature 
        android:name="android.hardware.camera.ar"
        android:required="false"  />



<application
  android:icon="@mipmap/ic_launcher"
        android:label="@string/app_name"
>
-------


  <!-- AR Core -->
        <meta-data
            android:name="com.google.ar.core"
            android:value="optional" />

  </application>

<uses-feature

android:name="android.hardware.camera"

android:required="false" />

<uses-feature

android:name="android.hardware.camera.autofocus"

android:required="false" />

<uses-feature

android:name="android.hardware.camera.ar"

android:required="false" />

<application

android:icon="@mipmap/ic_launcher"

android:label="@string/app_name"

-------

<meta-data

android:name="com.google.ar.core"

android:value="optional" />

</application>

3. Adding ar code on the MainActivity.kt file:

class MainActivity : AppCompatActivity() {

    private lateinit var mContentViewBinding: ActivityArBinding

    private var arFragment: ArFragment? = null
    private var objectRenderable: ModelRenderable? = null
    private var arModel: String? = null
    private var mModelStateSnackBar: Snackbar? = null
    private lateinit var mModelCompletableFuture: CompletableFuture<Void>
    var anchorNode: AnchorNode? = null
    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)

        arModel = "AR_MODEL_URL"
        if (checkIsSupportedDeviceOrFinish(this)) {
            mContentViewBinding = DataBindingUtil.setContentView(this, R.layout.activity_ar)
            startInitialization()
        } else {
            ToastHelper.showToast(this, getString(R.string.the_ar_feature_is_not_supported_by_your_device))
            this.finish()
        }
    }

   private fun startInitialization() {
        try {
            if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.N) {
                if (ContextCompat.checkSelfPermission(this, Manifest.permission.WRITE_EXTERNAL_STORAGE) == PackageManager.PERMISSION_GRANTED && ContextCompat.checkSelfPermission(this, Manifest.permission.READ_EXTERNAL_STORAGE) == PackageManager.PERMISSION_GRANTED) {
                    try {

                        arFragment = supportFragmentManager.findFragmentById(R.id.ux_fragment) as ArFragment


                        Toast.makeText(this@ArActivity, getString(R.string.downloading_model), Toast.LENGTH_SHORT).show()

                        // Init renderable
                        loadModel()

                         // Set tap listener
                        arFragment!!.setOnTapArPlaneListener { hitResult: HitResult, plane: Plane?, motionEvent: MotionEvent? ->
                            val anchor = hitResult.createAnchor()
                            if (anchorNode == null) {
                                anchorNode = AnchorNode(anchor)
                                anchorNode?.setParent(arFragment!!.arSceneView.scene)
                                createModel()
                            }
                        }
                    } catch (e: Exception) {
                        e.printStackTrace()
                    }
                } else {
                    val permissions = arrayOf(Manifest.permission.WRITE_EXTERNAL_STORAGE, Manifest.permission.READ_EXTERNAL_STORAGE)
                    requestPermissions(permissions, RC_AR)
                }
            }
        } catch (e: java.lang.Exception) {
            Log.d("TAG", e.printStackTrace().toString() + e.message.toString())
        }
    }

    private fun createModel() {
        try {
            if (anchorNode != null) {
                val node = TransformableNode(arFragment!!.transformationSystem)
                node.scaleController.maxScale = 0.02f
                node.scaleController.minScale = 0.01f
                node.setParent(anchorNode)
                node.renderable = objectRenderable

                node.select()
                mModelStateSnackBar = Snackbar.make(mContentViewBinding.arLayout, getString(R.string.model_ready), Snackbar.LENGTH_INDEFINITE).setAction(getString(R.string.dismiss)) {
                    mModelStateSnackBar?.dismiss()
                }
            }
        } catch (e: Exception) {
            Toast.makeText(this@ArActivity, getString(R.string.something_went_wrong), Toast.LENGTH_SHORT).show()

        }
    }

    @RequiresApi(api = Build.VERSION_CODES.N)
    private fun loadModel() {
        try {
            ModelRenderable.builder()
                    .setSource(this,
                            RenderableSource.builder().setSource(
                                    this,
                                    Uri.parse(arModel),
                                    RenderableSource.SourceType.GLB)/*RenderableSource.SourceType.GLTF2)*/
                                    .setScale(0.75f)
                                    .setRecenterMode(RenderableSource.RecenterMode.ROOT)
                                    .build())
                    .setRegistryId(arModel)
                    .build()
                    .thenAccept { renderable: ModelRenderable ->
                        objectRenderable = renderable
                        Toast.makeText(this@ArActivity, getString(R.string.model_ready), Toast.LENGTH_SHORT).show()

                    }
                    .exceptionally { throwable: Throwable? ->
                        Log.i("Model", "cant load")
                        mModelStateSnackBar = Snackbar.make(mContentViewBinding.arLayout, getString(R.string.model_error), Snackbar.LENGTH_INDEFINITE).setAction(getString(R.string.try_again)) {
                            mModelStateSnackBar?.dismiss()
                        }
                        null
                    }
        } catch (e: Exception) {
            e.printStackTrace()
        }
    }

private fun checkIsSupportedDeviceOrFinish(activity: Activity): Boolean {
        if (Build.VERSION.SDK_INT < Build.VERSION_CODES.N) {
            Log.e("error", "Sceneform requires Android N or later")
            return false
        }
        val openGlVersionString = (activity.getSystemService(Context.ACTIVITY_SERVICE) as ActivityManager)
                .deviceConfigurationInfo
                .glEsVersion
        if (java.lang.Double.parseDouble(openGlVersionString) < MIN_OPENGL_VERSION) {
            Log.e("error", "Sceneform requires OpenGL ES 3.1 later")
            return false
        }
        return true
    }

}

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

class MainActivity : AppCompatActivity() {

private lateinit var mContentViewBinding: ActivityArBinding

private var arFragment: ArFragment? = null

private var objectRenderable: ModelRenderable? = null

private var arModel: String? = null

private var mModelStateSnackBar: Snackbar? = null

private lateinit var mModelCompletableFuture: CompletableFuture<Void>

var anchorNode: AnchorNode? = null

override fun onCreate(savedInstanceState: Bundle?) {

super.onCreate(savedInstanceState)

arModel = "AR_MODEL_URL"

if (checkIsSupportedDeviceOrFinish(this)) {

mContentViewBinding = DataBindingUtil.setContentView(this, R.layout.activity_ar)

startInitialization()

} else {

ToastHelper.showToast(this, getString(R.string.the_ar_feature_is_not_supported_by_your_device))

this.finish()

}

private fun startInitialization() {

try {

if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.N) {

if (ContextCompat.checkSelfPermission(this, Manifest.permission.WRITE_EXTERNAL_STORAGE) == PackageManager.PERMISSION_GRANTED && ContextCompat.checkSelfPermission(this, Manifest.permission.READ_EXTERNAL_STORAGE) == PackageManager.PERMISSION_GRANTED) {

try {

arFragment = supportFragmentManager.findFragmentById(R.id.ux_fragment) as ArFragment

Toast.makeText(this@ArActivity, getString(R.string.downloading_model), Toast.LENGTH_SHORT).show()

// Init renderable

loadModel()

// Set tap listener

arFragment!!.setOnTapArPlaneListener { hitResult: HitResult, plane: Plane?, motionEvent: MotionEvent? ->

val anchor = hitResult.createAnchor()

if (anchorNode == null) {

anchorNode = AnchorNode(anchor)

anchorNode?.setParent(arFragment!!.arSceneView.scene)

createModel()

}

} catch (e: Exception) {

e.printStackTrace()

}

} else {

val permissions = arrayOf(Manifest.permission.WRITE_EXTERNAL_STORAGE, Manifest.permission.READ_EXTERNAL_STORAGE)

requestPermissions(permissions, RC_AR)

}

} catch (e: java.lang.Exception) {

Log.d("TAG", e.printStackTrace().toString() + e.message.toString())

}

private fun createModel() {

try {

if (anchorNode != null) {

val node = TransformableNode(arFragment!!.transformationSystem)

node.scaleController.maxScale = 0.02f

node.scaleController.minScale = 0.01f

node.setParent(anchorNode)

node.renderable = objectRenderable

node.select()

mModelStateSnackBar = Snackbar.make(mContentViewBinding.arLayout, getString(R.string.model_ready), Snackbar.LENGTH_INDEFINITE).setAction(getString(R.string.dismiss)) {

mModelStateSnackBar?.dismiss()

}

} catch (e: Exception) {

Toast.makeText(this@ArActivity, getString(R.string.something_went_wrong), Toast.LENGTH_SHORT).show()

}

@RequiresApi(api = Build.VERSION_CODES.N)

private fun loadModel() {

try {

ModelRenderable.builder()

.setSource(this,

RenderableSource.builder().setSource(

this,

Uri.parse(arModel),

RenderableSource.SourceType.GLB)/*RenderableSource.SourceType.GLTF2)*/

.setScale(0.75f)

.setRecenterMode(RenderableSource.RecenterMode.ROOT)

.build())

.setRegistryId(arModel)

.build()

.thenAccept { renderable: ModelRenderable ->

objectRenderable = renderable

Toast.makeText(this@ArActivity, getString(R.string.model_ready), Toast.LENGTH_SHORT).show()

}

.exceptionally { throwable: Throwable? ->

Log.i("Model", "cant load")

mModelStateSnackBar = Snackbar.make(mContentViewBinding.arLayout, getString(R.string.model_error), Snackbar.LENGTH_INDEFINITE).setAction(getString(R.string.try_again)) {

mModelStateSnackBar?.dismiss()

}

null

}

} catch (e: Exception) {

e.printStackTrace()

}

private fun checkIsSupportedDeviceOrFinish(activity: Activity): Boolean {

if (Build.VERSION.SDK_INT < Build.VERSION_CODES.N) {

Log.e("error", "Sceneform requires Android N or later")

return false

}

val openGlVersionString = (activity.getSystemService(Context.ACTIVITY_SERVICE) as ActivityManager)

.deviceConfigurationInfo

.glEsVersion

if (java.lang.Double.parseDouble(openGlVersionString) < MIN_OPENGL_VERSION) {

Log.e("error", "Sceneform requires OpenGL ES 3.1 later")

return false

}

return true

}

4. Let’s add the scene form ArFragment on the layout code of activity_ar.xml code:

<layout xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto">

    <RelativeLayout
        android:id="@+id/ar_layout"
        android:layout_width="match_parent"
        android:layout_height="match_parent">

        <fragment
            android:id="@+id/ux_fragment"
            android:name="com.google.ar.sceneform.ux.ArFragment"
            android:layout_width="match_parent"
            android:layout_height="match_parent"
            android:layout_below="@id/toolbar"
            android:tag="ar_fragment" />
    </RelativeLayout>
</layout>

<layout xmlns:android="http://schemas.android.com/apk/res/android"

xmlns:app="http://schemas.android.com/apk/res-auto">

<RelativeLayout

android:id="@+id/ar_layout"

android:layout_width="match_parent"

android:layout_height="match_parent">

<fragment

android:id="@+id/ux_fragment"

android:name="com.google.ar.sceneform.ux.ArFragment"

android:layout_width="match_parent"

android:layout_height="match_parent"

android:layout_below="@id/toolbar"

android:tag="ar_fragment" />

</RelativeLayout>

</layout>

Now you just have to pass the .glb/.gltf model URL on the “arModel” string. Now just run your app.

. . . . . . . . . .

That’s it from my side for today, thanks for reading it until now.

android AR Kotlin Sceneform

Deepak Rawat

16 Jun, 2023
Updated by - Akshay Singh
31 Oct, 2020
Created by - Deepak Rawat

. . .

4 comments

iulia 2 years ago

Very useful tutorial! Thanks. One question. My 3D model also has an animation which is not shown, I only see the 3D model. Do you happen to know what else needs to be configured to see the animation?

anchit (Moderator) 2 years ago

Hey,

Firstly, it depends on how you have imported the 3d model.

If you are seeing only the 3D model, then most probably the animations were not added in the glb/gltf file.

To make sure that your gltf file is proper and has animations while importing, open the gltf file in text editor and then check for the tag named “animations”.

If this tag is present, then you need to check the android end code further.

Supposedly, your glb/gltf file is correct and upto the mark, then in the android code you will need to get access to animation that you want to run by calling getAnimationData Function on your ModelRenderable object.

Now you will need to create an instance of ModelAnimator and pass both animationObject and Renderable object as arguments to the constructor of ModelAnimator.

Now you need to call the start function over ModelAnimator object and it should work.

Still if you face any issue, then please feel free to reply back over here.

Reply

Preeti 4 years ago

Project is not completly uploaded. Some classes and other things also missed.

Deepak Rawat (Moderator) 4 years ago
Can you please share the classes that are missing and errors you are facing.

Reply

AR core with glb/gltf model using Sceneform

Some important links for the AR model

1:Conversion from .gltf to .glb online :

2: Conversion from .obj to .glb offline using node.js: Click here

3: To get the sample models: Click here

Let’s get it into code

1. In your project build.gradle dependencies:

2. In your app build.gradle dependencies:

3. In the Android Manifest.xml file:

3. Adding ar code on the MainActivity.kt file:

4. Let’s add the scene form ArFragment on the layout code of activity_ar.xml code:

Leave a Comment Cancel Reply

4 comments

Message Sent!