Appium Visual Testing -4 : Complete Details On ‘-image’ locator strategy Usage — “AppiumBy.Image()”

Get a step-by-step walkthrough in the video below!

In this article, you’ll learn complete details on ‘-image’ locator strategy “AppiumBy.Image()”

What is -image Locator Strategy?

The -image strategy typically refers to image recognition-based element location (e.g., using image or opencv strategies)

These will allow the user to locate UI elements by matching screenshots rather than by standard selectors (like XPath or CSS).

Image Locator Usage:

Example: —

driver.findElement(AppiumBy.image(base64image));

Pre-requites for This Strategy to Work:

The -image locator only works if the following are installed on the server (where Appium runs):

1. OpenCV Libraries:

→ OpenCV (Open Source Computer Vision Library) is used for image processing and matching.

→ It must be installed on the machine running the server (e.g., Appium server).

Please go through the below video for complete details regarding “OpenCV installation on Windows”

2. Node.js Bindings:

→ Tools like Appium are built on Node.js, so bindings for OpenCV must be present to allow Node.js code to interact with OpenCV.

Please go through the below video for complete details regarding “OpenCV plugin installation”

Run Appium Server with below flags:

Please note that to use ‘-image’ locator , User should activate “image” plugin explicitly , while launching the Appium server:

appium - use-plugins=images

Example1:

Launch the appium server without using ‘images’ plugin.

Observe that script will fail & will displaying below error message as per the screenshot

Example2:

Launch the appium server with ‘images’ plugin

Please go through the below video for complete details regarding “images plugin installation”:

“Complete Code: Optimized and Ready to Use”:

package com.appiumguide.appiumimages;

import org.openqa.selenium.WebElement;
import org.testng.annotations.AfterTest;
import org.testng.annotations.BeforeTest;
import org.testng.annotations.Test;

import io.appium.java_client.AppiumBy;
import io.appium.java_client.android.AndroidDriver;
import io.appium.java_client.android.options.UiAutomator2Options;

import java.io.File;
import java.io.IOException;
import java.net.*;
import java.nio.file.Files;
import java.time.Duration;
import java.util.Base64;

/**
 * Script Details - Appium-Images : Complete Details On ‘-image’ locator strategy AppiumBy.Image()
 * 
 * appium-java-client version: Latest
 * 
 * @author 'Ramesh Kodumuru' for AppiumGuide [appiumguide@gmail.com]
 */

public class ImageLocatorDemo {
 
 private AndroidDriver driver;
      
 @BeforeTest
 public void setup() throws MalformedURLException {
  UiAutomator2Options cap=new UiAutomator2Options();
  cap.setPlatformName("android");
  cap.setAutomationName("uiautomator2");
  cap.setDeviceName("Pixel9");  
     
  driver = new AndroidDriver(new URL("http://127.0.0.1:4723"), cap);
  driver.manage().timeouts().implicitlyWait(Duration.ofSeconds(100));
  
 }
 
 @Test
 public void test() throws InterruptedException, IOException {
 
  driver.activateApp("in.burgerking.android");
  Thread.sleep(8000);
  driver.findElement(AppiumBy.androidUIAutomator("new UiSelector().text(\"BK Recommended\")")).click();
  
  Thread.sleep(5000);
  System.out.println("Click on the Beverages image ");
  imagelocator("Beverages.png");
  Thread.sleep(5000);
  System.out.println("Click on the Snacks image ");
  imagelocator("Snacks.png");
  Thread.sleep(5000);
  System.out.println("Click on the Beverages image ");
  imagelocator("Beverages.png");

 }
 
 void imagelocator(String imagedetails) throws IOException {
  
  //Convert the image from file to base64String
  File file=new File(System.getProperty("user.dir")+"/"+imagedetails);
  byte[] filecontent=Files.readAllBytes(file.toPath());
  String base64image=Base64.getEncoder().encodeToString(filecontent);
  
  //Locating Ui element based on image using image locator
  WebElement imageElement=driver.findElement(AppiumBy.image(base64image));
  
  imageElement.click();
  
 }
 
    
  
 @AfterTest
 public void teardown() {
  
  if(driver!=null) {
   //driver.quit();
   System.out.println("Test Completed");
  }
  
 }

}

GitHub Link: