Run lint and unit tests of Android builds from Jenkins

Author: Roman Kushnarenko Dec 10, 2017 DevOps Jenkins Android

One of the most important things we want run part of every build is lint checks and junit tests. This is what we are going to setup and do in this blog post.

Previous blog post

Setup and build Android app from Jenkins

Step by step guide of how to setup Jenkins and Android SDK on VM and create a basic job that builds Android apps.

Build by types and sign Android app from Jenkins

In this part I will explain how to update Gradle file, setup signing options and build by types from Jenkins. Keeping sensitive keystore credential hidden.

Steps

Lint

The lint tool checks your Android project source files for potential bugs and optimization improvements for correctness, security, performance, usability, accessibility, and internationalization.

Update project

Application source files

The source files consist of files that make up your Android project, including Java and XML files, icons, and ProGuard configuration files.

Update build.gradle of every module and add this lines:

lintOptions {
    abortOnError false
    lintConfig file("../lint.xml")
}

It means that we don’t want fail our builds because of errors of lint checks and the second line is where our lint.xml file located.

Example: build.gradle

The lint.xml file

A configuration file that you can use to specify any lint checks that you want to exclude and to customize problem severity levels.

Add lint.xml file under root folder of the project.

This is example of lint.xml file you can add. In this example we ask lint tool to mark as error MissingTranslation check and to ignore ContentDescription issues. You can decide how you want handle lint checks. 

<?xml version="1.0" encoding="UTF-8"?>
<lint>
    <issue id="MissingTranslation" severity="error" />
    <issue id="ContentDescription" severity="ignore" />
</lint>
The lint tool

To run lint we just need call: ./gradlew lint and the report will be located under app/build/reports/lint-results.xml

Configure Jenkins Job

Install Necessary Plugins

Install these plugins: Manage Jenkins -> Manage Plugins -> Available

Configure Job

To present Lint results we will configure the job.

  1. Select job -> Configure

  2. Scroll to Post-build Actions

  3. Add post-build action ▼ -> Publish Android Lint results

  4. Fill and set the location of file with lint results. We’ll make sure that after running lint task the lint-results.xml file will be located under report folder:

    Lint files report/lint-results.xml

The jenkins job configuration looks like this - Click here for full screen

Execution script - build.sh

First of all create new folder in your root project and call it scripts (In case you didn’t do this in previous post). Then add new file and call it build.sh

The script will do next steps:

  1. Update key store values in gradle.properties file.
  2. Build debug or release APK.
  3. Run lint
  4. Move ready apk to artifacts folder.
  5. Move lint results to report folder.

Before I dive into the script details you can check this ready example. You can use this example as is, just make sure to rename the scripts-part-3 folder to scripts.

The steps 1,2 and 4 are covered in previous blog post. Now, I will cover the steps 3 and 5.

Run lint task

./gradlew lint
  1. We run lint task. If the lint.xml file is located under root folder of the project and we added lintConfig file("../lint.xml") as described above, the configuration will be taken into account.
  2. The report will be located here: app/build/reports/lint-results.xml

Move results to report folder

rm -r report/
mkdir report
cp app/build/reports/lint-results.xml report/
  1. Recreate the report folder.
  2. Copy the lint results to report folder.
The final script - Check here.

:tada: That’s it for lint. Try running your job and you will see lint trend graph next to your job results. You can configure the lint graph presentation from jenkins plugin configuration. Add androidLint/configure/ to url of your job on Jenkins for trend graph configuration.

Run twice

If you can’t see the trend graph -> build again. Sometimes the trend graph is not shown if you have only one build with results - no actual trend here ;) So, run and build again.

Tests

Tests in Android are separated into 2 types:

  • Local unit tests

    Located at module-name/src/test/java/ - These are tests that run on your machine’s local Java Virtual Machine (JVM). Use these tests to minimize execution time when your tests have no Android framework dependencies or when you can mock the Android framework dependencies.

  • Instrumented tests

    Located at module-name/src/androidTest/java/ - These are tests that run on a hardware device or emulator. These tests have access to Instrumentation APIs, give you access to information such as the Context of the app you are testing, and let you control the app under test from your test code. Use these tests when writing integration and functional UI tests to automate user interaction, or when your tests have Android dependencies that mock objects cannot satisfy.

Your project’s (1) instrumented tests and (2) local JVM tests are visible in either the Project view (left) or Android view (right).

In this post we are going to run Local unit tests and present the results in Jenkins job. The instrumentation tests building and execution will be explained in separate blog post.

Configure Jenkins Job

We continue upding the sample build.sh script.

Install Necessary Plugins

Install these plugins: Manage Jenkins -> Manage Plugins -> Available

  • JUnit Plugin https://wiki.jenkins.io/display/JENKINS/JUnit+Plugin. The JUnit plugin provides a publisher that consumes XML test reports generated during the builds and provides some graphical visualization of the historical test results as well as a web UI for viewing test reports, tracking failures, and so on.
Configure Job

To present tests results we will configure the job.

  1. Select job -> Configure

  2. Scroll to Post-build Actions

  3. Add post-build action ▼ -> Publish JUnit test result report

  4. Fill and set the location of folder with report XMLs. We’ll make sure that after running tests all reports files will be located under report/test-results folder:

    Test report XMLs report/test-results/*.xml
    Health report amplification factor 1.0

The jenkins job configuration looks like this - Click here for full screen

Execution script - build.sh

We continue updating build.sh script. The updated script will do next steps:

  1. Update key store values in gradle.properties file.
  2. Build debug or release APK.
  3. Run lint
  4. Run junit tests
  5. Move ready apk to artifacts folder.
  6. Move lint results to report folder.
  7. Move tests results from all modules to report/test-results folder.

Before I dive into the script details you can check this ready example. You can use this example as is, just make sure to rename the scripts-part-3 folder to scripts.

Run unit tests

if [ $buildType = 'debug' ]; then
    ./gradlew testDebugUnitTest --stacktrace
elif [ $buildType = 'release' ]; then
    ./gradlew testReleaseUnitTest --stacktrace
fi
  1. Based on build type run the test gradle task.
  2. The tests results for each module will be located under: {module}/build/test-results/testDebugUnitTest. (for debug build type).

Move results to report folder

mkdir report/test-results
modules=("app" "common")
for module in "${modules[@]}"
do

    testsDir=""
    if [ $buildType = 'debug' ]; then
        testsDir="$module/build/test-results/testDebugUnitTest"
    elif [ $buildType = 'release' ]; then
        testsDir="$module/build/test-results/testReleaseUnitTest"
    fi

    if [ ! "$(ls -A $testsDir)" ]; then
        echo "Unit tests report wasn't found for module: $module"
        continue
    fi

    # copy all files inside, to our folder
    cp $testsDir/* report/test-results/

done
  1. Create sub test-results folder inside report folder.
  2. Since each of the modules can contain junit tests we need collect all of them and place in one folder so Jenkins can find and present them. In this script we iterate over all modules in array modules=("app" "common") and copy the junit tests results into report/test-results/.
The final script - Check here.

:tada: That’s it for junit tests. Try running your job and you will see test result trend graph next to your job results.

Instrumented tests - The execution part

In one of the next blog posts I will explain how to build and execute instrumented tests part of Jenkins jobs. For now, these are 2 blog posts you can read of what can be achieved by running such tests.

Running Android UI Tests - Part 1
Sharding and running Android UI tests in parallel. How we improved execution tests time by 40%.		 Running Android UI Tests - Part 2
For each test — collect logs, record a video, dump db, preferences and more. Clear data, run by tags and with params. Easy extendable.

Questions

If you have any comments, please open an issue at https://github.com/sromku/build-android-jenkins

Copyright © Roman Kushnarenko

Home