Firebase ML-KIT In Flutter

Before starting the blog topic Firebase ML-KIT In Flutter, let’s discuss ML-KIT of the firebase. Machine-Learning kit is a mobile SDK that brings machine learning expertise to native as well as hybrid apps. ML-KIT comes with predefined ready-to-use APIs, that can be used by anyone who did not have deep knowledge of machine learning.

With the help of Firebase ML-KIT, we can perform various operations that can enhance or beautify the overall experience of the application. In this blog, we will discuss the text-recognition from an image with the help of Firebase ML-KIT In Flutter. Before starting the code level functionality we need to follow the mentioned steps:

1. Registration of the App

First of all, we need to register our app on the Firebase console for accessing the Firebase ML-KIT, we can follow the mentioned URL for registering the app on the Firebase console.

Firebase Console

2. Adding Firebase ML-VISION Package

Now we will add the ml-vision package in our pubspec.yaml folder inside our flutter project. I have added the following package, you can also add the updated version.

firebase_ml_vision: ^0.9.6+3

1	firebase_ml_vision: ^0.9.6+3

3 Adding package for Image

Now, according to our use case, we can use multiple types of packages for getting the image. In our demo, we have used an image from the camera and gallery both.

For accessing the camera, we have used the following package :

camera: ^0.5.7+4

1	camera: ^0.5.7+4

But after clicking the image from the camera, we need to save the image inside the directory of the app and we need the path of the image for creating the file. So we have also used the following packages.

path_provider: ^1.6.7
intl: ^0.16.1

1 2	path_provider: ^1.6.7 intl: ^0.16.1

For accessing the gallery, we have used the image picker package:

  image_picker: ^0.6.3+4

1	image_picker: ^0.6.3+4

Note: Please do not forget to add the firebase dependency to the android app build.Gradle file

4. Now, we can proceed with our main functionality of the blog i.e., Recognition of text from the image

>> Declare the CameraDescription list for accessing all available cameras of the devices.

List<CameraDescrption> cameraDescriptionList=[];
cameraDescriptionList=await availableCamera();

1 2	List<CameraDescrption> cameraDescriptionList=[]; cameraDescriptionList=await availableCamera();

Note: Please add this code inside the try-catch block for exception handling.

>> Initialised the Camera controller for making camera design

CameraController cameraController;

// use below-mentioned code inside the "initState" method
 cameraController = CameraController(cameraList[0], ResolutionPreset.medium);
    cameraController.initialize().then((_) {
      if (!mounted) {
        return;
      }
      setState(() {});
    });

// cameraList[0]=Rear camera of the device

CameraController cameraController;

// use below-mentioned code inside the "initState" method

cameraController = CameraController(cameraList[0], ResolutionPreset.medium);

cameraController.initialize().then((_) {

if (!mounted) {

return;

}

setState(() {});

});

// cameraList[0]=Rear camera of the device

>> Now, we will click the image and save it to the internal storage, then we can use it for image processing.

var rng = new Random();
    String imageName = rng.nextInt(100).toString() + ".jpg";
    final Directory appDocDir = await getApplicationDocumentsDirectory();
    final String visionDir = '${appDocDir.path}/images/Images';
    await Directory(visionDir).create(recursive: true);
    final String image = '$visionDir/image_$imageName';
    if (cameraController.value.isTakingPicture) {
      print("Processing is progress ...");
      return null;
    }
    try {
      await cameraController.takePicture(image);
      print("CLicked Image" + image);
      Navigator.push(
        context,
        MaterialPageRoute(
          builder: (context) => TextRecognise(image,pickedImage),
        ),
      );
    } on CameraException catch (e) {
      print("Exception" + e.description);
      return null;
    }

var rng = new Random();

String imageName = rng.nextInt(100).toString() + ".jpg";

final Directory appDocDir = await getApplicationDocumentsDirectory();

final String visionDir = '${appDocDir.path}/images/Images';

await Directory(visionDir).create(recursive: true);

final String image = '$visionDir/image_$imageName';

if (cameraController.value.isTakingPicture) {

print("Processing is progress ...");

return null;

}

try {

await cameraController.takePicture(image);

print("CLicked Image" + image);

Navigator.push(

context,

MaterialPageRoute(

builder: (context) => TextRecognise(image,pickedImage),

);

} on CameraException catch (e) {

print("Exception" + e.description);

return null;

}

We have used the random number to save the images inside the directory with new names each and every time we click the image. We can also use the image picker to select the image from the gallery:

var awaitImage = await ImagePicker.pickImage(source: ImageSource.gallery);
    setState(() {
      pickedImage = awaitImage;
    });

var awaitImage = await ImagePicker.pickImage(source: ImageSource.gallery);

setState(() {

pickedImage = awaitImage;

});

After getting the image, now we can recognize the app and extract the text.

    File file = File(image);

    FirebaseVisionImage firebaseVisionImage = FirebaseVisionImage.fromFile(file);

    TextRecognizer textRecognizer = FirebaseVision.instance.textRecognizer();

    VisionText visionText = await textRecognizer.processImage(firebaseVisionImage);

    // for detecting text block
    for (TextBlock block in visionText.blocks) {
      // for detecting lines in blocks
      for (TextLine line in block.lines) {
// for detecting word in one lines
        for (TextElement word in line.elements) {
          detectedText = detectedText + word.text + ' ';
        }
        detectedText = detectedText + '\n';
      }
    }
    print("detected "+detectedText);

    if (this.mounted) {
      setState(() {
        detectedText = detectedText;
      });
    }

File file = File(image);

FirebaseVisionImage firebaseVisionImage = FirebaseVisionImage.fromFile(file);

TextRecognizer textRecognizer = FirebaseVision.instance.textRecognizer();

VisionText visionText = await textRecognizer.processImage(firebaseVisionImage);

// for detecting text block

for (TextBlock block in visionText.blocks) {

// for detecting lines in blocks

for (TextLine line in block.lines) {

// for detecting word in one lines

for (TextElement word in line.elements) {

detectedText = detectedText + word.text + ' ';

}

detectedText = detectedText + '\n';

}

print("detected "+detectedText);

if (this.mounted) {

setState(() {

detectedText = detectedText;

});

}

>> Here is the output app result of our app:

. o/p

This is a basic example of text recognization through ML-KIT. You can perform multiple operations through Firebase ML-KIT.

Hope this blog helps you to use Firebase ML-KIT In Flutter

For more information, please go through the Flutter and Firebase console docs.

firebase Firebase Ml Kit flutter Text Recognition

Abhishek Rathore 26 February 2021

. . .