The role of the Continuous Integration platform is to validate the different
commits pushed to OTB repository. When commits are pushed, either on the main
repository or on a fork, the platform triggers a pipeline with several jobs to
build and test OTB on different configurations.
Here is a figure describing the overall architecture of the CI platform.
Depending on where your commit was pushed, different pipelines are created:
push on a feature branch -> wip pipeline
push on a feature branch with a merge request (not in WIP) -> mr pipeline
push on develop branch -> develop pipeline
push on a release branch release_X.Y -> release pipeline
The wip pipeline is the fastest, with only one build job on Linux. The mr,
develop and release pipeline are more exhaustive and build on several platforms.
The different pipeline stages are:
precheck : fast build on Linux, no test
prepare : for configuration using a XDK, update it if SuperBuild has changed
build : compile, test and package OTB on various platforms
report : analyse quality reports and push them to SonarQube
deploy : upload generated binaries and documentation to OTB website. For the develop branch, the 11 latest days with uploads are kept. For the release branches, there is a unique staging folder.
external : contains the links to CDash submissions
When a pipeline ends, there are two cases:
if all the jobs succeed, you see a green pipeline, which means no problem was
found on your commit.
if one job fails, you see a red pipeline, which means something is broken in
your commit. The pipeline widget on Gitlab will tell you which job failed. You
will also find special jobs "cdash:..." in the stage external that provide
a link to the Dashboard
where you can look more in details into compilation errors and failed tests.
Failed jobs can be retried if you think the failure was not bound to your commit.
Q: What can I find in the job artifacts?
Most of the jobs will produce artifacts. They are often log files, but some of them
have more value for the developer:
all the *-prepare jobs produce a text file sb_branch.txt that identifies the
XDK to be used (it is actually a branch name in the superbuild-artifact repository).
the job centos-xdk-build produces a standalone binary OTB-*.run for Linux 64
the job ubuntu-xdk-build-doc produces a standalone binary OTB-*.run for Linux 64, plus
CookBook and Doxygen documentation
the job macos-xdk-build produces a standalone binary OTB-*.run for macOS
the job windows-10-build produces a standalone binary OTB-*.zip for Windows 10 64bit
the job windows-8-build produces a standalone binary OTB-*.zip for Windows 8.1 32bit
Note: the artifacts are kept only 24h on the Gitlab server
Q: Why some features are disabled when I push from my OTB fork?
First, you need an access to the Runners from main repository. You
can request it when doing your first merge request. During code review, someone
from CI admins will assign the runners to your fork.
Then, there are two security tokens that are needed to enable some features of
the CI platform:
K8S_SECRET_API_TOKEN: A token for Gitlab API. It enables the dashboard links
in the External pipeline section, and the handling of twin pipelines (for MR,
two pipelines are spawned).
SONAR_OTB_TOKEN: this token enables the QA jobs and the reporting to
These 2 tokens are already supplied for pipelines triggered on the main
repository. On your fork, you need to provide them. Note that they are optional,
you can still enjoy most of the CI platform without them. For your fork, these
tokens can be created as follows:
SONAR_OTB_TOKEN: login to SonarQube, then
go to your profile page, then Security, and create a token
Once a token is created, you can supply it to your OTB fork: go to your fork
project page. In the left sidebar, go to Settings -> CI/CD -> Variables. Then
add a new variable with the token name and token value. For security reasons,
it is advised to mask these variables.
Q: Can I get Docker build environments?
Yes, for Linux builds, we use Docker images to compile and test OTB. They are stored in a
You can find images for Ubuntu 18.04, CentOS 6.6, and debian unstable. There is also a Docker image otb-ubuntu-native-develop with the latest OTB build (from develop branch).
If you want to get one of these images, login to the registry with your Gitlab account and pull the image:
They are now stored in the main OTB repository, under
Data folder. You can update them with a plain commit on your feature branch.
Note that the data files are tracked with Git LFS
, so you need to install this extension.
Q: How do I get a test result back from CI?
You have a failing test and you want to analyse the output. Here is how you can do it:
First step is to check information available on the CDash report (link should be available at the end of the job logs).
If this is not enough to investigate, you can download the test output:
Find the file name of the test output (let say myOutput.tif)
Edit the file .gitlab-ci.yml and locate the job where your test fails