Introduction to the LAVA Test Developer Guide

This guide aims to enable engineers to be able to

  1. Submit desired jobs/tests on target deployed where the LAVA server is located and report results.
  2. Understand how test job files need to be written so that jobs get submitted properly.
  3. Understand how test shell definitions need to be written.

Ensure you have read through the introduction on Writing a LAVA test definition.

Pay particular attention to sections on:

Assumptions at the start of this guide

  1. The desired board is already deployed at the LAVA Server location.
  2. A user account (username, password, email address) is already created by a LAVA administrator on your behalf, with permissions to submit jobs.
  3. lava-tool is already installed on your test system and a suitable token has been added.
  4. You are familiar with submitting jobs written by someone else, including viewing the logs file for a job, viewing the definition used for that job and accessing the complete log.

To install lava-tool, see lava-tool overview.

To authenticate lava-tool, see Authentication Tokens.

To find out more about submitting tests written by someone else, see Submitting your first job.

To find out more about viewing job details, see Job Submission.

Checking device availability

Use the LAVA scheduler to view available device types and devices. The main scheduler status page shows data for each device type as well as the currently active jobs. Also check the Devices pages:

  • All Devices - includes retired devices to which jobs cannot be submitted.
  • All Active Devices - lists only devices to which jobs can be submitted
  • All Devices Health - limited to just the latest health status of each device.
  • My Devices - available from your profile menu by clicking on your name once signed into the instance.

For a Multi-Node LAVA (JSON) job, you may need to check more than one device type.

Devices are considered available for new jobs according to the Device status.

  • Idle, Reserved, Offline, Offlining - jobs can be submitted.
  • restricted - only available for submissions made by declared users.
  • Retired - jobs will be rejected if all remaining devices of this type are retired.

Finding an image to run on the device

Start with an image which is already in use in LAVA. You can find one of these images by checking the device type in LAVA and viewing some of the jobs for devices of this type from the table on that page. e.g. for QEMU devices on validation.linaro.org:

https://validation.linaro.org/scheduler/device_type/qemu

The job definition will provide information on the image or the Interactive boot commands which started the job.

Actions to be run for a LAVA test

  1. Deploy: Each device type supports a range of deployment methods.
  2. Boot: Each device type supports a range of boot methods.
  3. Test: Run the lava test shell.

Examples

Deploying a pre-built QEMU image

actions:
  - deploy:
      timeout:
        minutes: 5
      to: tmpfs
      images:
          rootfs:
            image_arg: -drive format=raw,file={rootfs}
            url: http://images.validation.linaro.org/kvm-debian-wheezy.img.gz
            compression: gz
      os: debian

Using device tags

A device tag marks a specified device as having specific hardware capabilities which other devices of the same device type do not. To test these capabilities, a Test Job can specify a list of tags which the device must support. If no devices exist which match all of the required tags, the job submission will fail. If devices support a wider range of tags than required in the Test Job (or the Test Job requires no tags), any of those devices can be used for the Test Job.

Note

Test jobs which use device tag support can only be submitted to instances which have those tags defined and assigned to the requested boards. Check the device information on the instance to get the correct tag information.

Using LAVA Test Shell

The lava_test_shell action provides a way to employ a more black-box style testing approach with the target device. The action only requires that a deploy action (deploy_linaro_image/deploy_linaro_android_image) has been executed. Its format is:

- test:
    failure_retry: 3
    name: kvm-basic-singlenode  # is not present, use "test $N"
    # only s, m & h are supported.
    timeout:
      minutes: 5 # uses install:deps, so takes longer than singlenode01
    definitions:
        - repository:
            metadata:
                format: Lava-Test Test Definition 1.0
                name: smoke-tests-basic
                description: "Basic system test command for Linaro Ubuntu images"
            run:
                steps:
                    - printenv
          from: inline
          name: env-dut-inline
          path: inline/env-dut.yaml
        - repository: git://git.linaro.org/qa/test-definitions.git
          from: git
          path: ubuntu/smoke-tests-basic.yaml
          # name: if not present, use the name from the YAML. The name can
          # also be overriden from the actual commands being run by
          # calling the lava-test-suite-name API call (e.g.
          # `lava-test-suite-name FOO`).
          name: smoke-tests
        - repository: http://git.linaro.org/lava-team/lava-functional-tests.git
          from: git
          path: lava-test-shell/single-node/singlenode03.yaml
          name: singlenode-advanced

You can put multiple test definition URLs in “definitions” list. These will be run sequentially without reboot.

See also

Dispatcher Actions

See also

lava_test_shell developer documentation