About the data

Coverage over time

Data sources

LAVA tf.validation.linaro.org public API

Rolling 30 days at test-case level. Older jobs need a LAVA login, so links to them are marked not public and the stored record is shown instead. Last successful fetch 28 Oct 13:02 UTC, ok.

S3 exports lava-exported-data/tf

Job level only, Oct 2024 to Mar 2026. The coverage ribbon lists missing days and gaps, including Oct to Dec 2025 and April to early September 2026. Last successful fetch 07 Oct 09:53 UTC, ok.

Jenkins ci.trustedfirmware.org

Builders are kept 3 to 5 days and Gerrit pipelines since May 2026, so a build is tied to its change only where we read it in time. Last successful fetch 28 Oct 13:03 UTC, ok.

Gerrit review.trustedfirmware.org

Linked, never fetched: change numbers and subjects come from Jenkins parameters. Votes are not collected.

Ingest runs

Fetch errors

6509 open of 6509 recorded. A failed request never deletes data already fetched; it is listed here so gaps in fetching remain visible.

• jenkins-json gone: 6488
• jenkins-json http: 19
• lava-tests gone: 2

Test names

Display names come from naming.yaml (98aaf66937cc): 272 of 536 names have a non-provisional mapping. Provisional display names are marked like this and raw names are shown alongside display names. Board names are derived from the LAVA device type, so a new board needs no edits.

Glossary

Evaluation

One CI request to test one build on one board with one run configuration. It owns its attempts, and its outcome is the outcome of the final attempt.

Attempt

One LAVA job of an evaluation. A CI retry adds an attempt to the same evaluation.

Reproduction

A job resubmitted by hand. It is shown with the job it reproduces and never counted in a CI rate.

✔ Pass

The job completed and no assertion failed.

✖ Code

A completed job had a failed assertion, or a Test or Job error occurred with a known boot result. Historical exports can also identify job errors explicitly. Drawn solid.

▨ Lab

The job did not complete for an infrastructure, LAVA or configuration reason. Always drawn hatched: hatched means the lab, solid means the code.

○ Cancelled

Cancelled before it finished. Never in a pass rate.

? Unclassified

The job failed but the cause was not recorded, which is the case for almost every S3-era failure. Never guessed, never folded into lab or code.

Recovered on retry

The first attempt had a code, lab or unclassified failure and the final attempt passed. A cancelled first attempt is not a recovery. Reported apart from first-attempt failures and final failures.

Operational completion

Complete attempts divided by complete plus incomplete attempts. Cancelled attempts are shown beside it, never in the denominator. Available for the whole history, including S3.

Evaluated pass rate

Evaluations that passed divided by evaluations that passed or failed in the code, for the live window with test results only. Lab, canceled and unclassified evaluations are reported next to it, never inside it.

Assertion

A result that tests the firmware. LAVA's own steps (actions), wrapper rows and the job row are shown on job pages but are not tests.

Changes with a failed boot on this board

Gerrit change patchsets whose +2 run on the board failed to boot after retries, out of those with a boot verdict on that board. The Gerrit vote is not collected, so this is not a count of changes turned away.

Declared and observed

Declared means a job definition listed the suite; observed means it ran and reported. Declared never means passed.

This dataset

Data as of

28 Oct 13:02 UTC

Data version

e26fb76d96d0

Classification rules

version 1

Built

28 Oct 13:10 UTC

The same numbers are available as JSON at /api/v1.