otb merge requestshttps://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/merge_requests2019-04-30T13:40:09Zhttps://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/merge_requests/483CI: fixed architecture2019-04-30T13:40:09ZAntoine RegimbeauCI: fixed architecture#### Summary
This MR aims at unifying architecture on the different platforms, now everything is done under `~/builds/otb` regardless of the namespace or the project.
#### Future work
A switch can be implemented in the CMake scripts so ...#### Summary
This MR aims at unifying architecture on the different platforms, now everything is done under `~/builds/otb` regardless of the namespace or the project.
#### Future work
A switch can be implemented in the CMake scripts so that local build using CI scripts will not pollute the source dir.
#### Copyright
The copyright owner is *CNES* and has signed the ORFEO ToolBox Contributor License Agreement.
<hr>
***Check before merging:***
- All discussions are resolved
- At least 2 :thumbsup: votes from core developers, no :thumbsdown: vote.
- The feature branch is (reasonably) up-to-date with the base branch
- Dashboard is green
- Copyright owner has signed the ORFEO ToolBox Contributor License Agreement
- Optionally, run `git diff develop... -U0 --no-color | clang-format-diff.py -p1 -i` on latest changes and commit7.0.0https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/merge_requests/589Mentions of copyright check2019-09-18T13:27:58ZSébastien DinotMentions of copyright checkThis merge request implements the check proposed in issue #1959.
The related continuous integration job fails for now (without affecting the continuous integration result), not because there is a bug in the check script, but because the...This merge request implements the check proposed in issue #1959.
The related continuous integration job fails for now (without affecting the continuous integration result), not because there is a bug in the check script, but because there are some mistakes in the mentions of copyright, as explained in issue #1959.7.0.0https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/merge_requests/584Fixes for NamespaceHandler2019-09-18T13:28:32ZGuillaume PaseroFixes for NamespaceHandlerTwo fixes for the NamespaceHandler:
* there was target names in `OTBStandaloneModuleMacros` that needed to be patched as well
* there was a `OTBApplicationEngine` in `OTBApplicationMacros`, but it should not be here, the dependency to O...Two fixes for the NamespaceHandler:
* there was target names in `OTBStandaloneModuleMacros` that needed to be patched as well
* there was a `OTBApplicationEngine` in `OTBApplicationMacros`, but it should not be here, the dependency to OTBApplicationEngine should be in the DEPENDS from `otb-module.cmake`7.0.0Guillaume PaseroGuillaume Paserohttps://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/merge_requests/456Various CI fixes for a lighter dashboard2019-09-19T07:12:52ZGuillaume PaseroVarious CI fixes for a lighter dashboard#### Summary
Fixes to enable tests on the new CI, and reduce tests verbosity.
#### Rationale
Enable the tests on new CI, and reduce the log output (#1867).
#### Implementation Details
There are different types of changes i...#### Summary
Fixes to enable tests on the new CI, and reduce tests verbosity.
#### Rationale
Enable the tests on new CI, and reduce the log output (#1867).
#### Implementation Details
There are different types of changes in this branch:
* CI scripts:
- enable testing
- use xvfb to launch test (needed for GUI tests)
- remove unused WRAP_PYTHON3 option
* Make all tests pass:
- can't use Shark on ubuntu-native due to #1871
- allow root to launch mpi-enabled tests (`EXTRA_OPT --allow-run-as-root`)
- the RPC estimation/optimization functions are broken on Ossim 2.2.0, the tests don't crash but I had to remove some baselines checks for Ossim >= 2.2.0
- add some multi-baselines...
- in `otb::WaveletOperatorBase`, a weird crash was breaking all wavelet tests.
- in `TestMain`, I made sure that the `MPIFinalize()` is called in all cases, to avoid the application hanging at closure. Removed early `return`.
- in `ApplicationLauncherCommandLine`, I had to disable `MPIInit()` when `-testenv` is present: since this option leaves exceptions uncaught, the `MPIFinalize()` at the end of the application launcher is not called, and the TestDriver hangs on closure (see #1845).
- I made a small refactor of `TestMain` and `TestHelper` : better parsing of options, support of the `--tolerance-ratio` option (more details in the next section).
* Reduce unnecessary test verbosity:
- massive use of `otbLogMacro(Debug, ... )` to control test verbosity at runtime.
- I changed the priority level of `No kwl metadata found in file ...` to Debug, I think it is more relevant to log information when metadata has actually been found.
- Code factorization in `Modules/Learning/Supervised/test/TrainMachineLearningModel.cxx`
- the log size has been reduced from ~12MB to 3MB
#### Additional notes
In order to reduce the number of baselines, I introduced an new option for the test driver: `--tolerance-ratio`. It defines what percentage of the data can fail the epsilon check. For the moment it only applies to the image comparisons. For instance, passing the option `--tolerance-ratio 0.01` means that the number of pixels with difference should be less than 1% of all pixels. By default, this tolerance is 0.
#### Copyright
The copyright owner is *CNES* and has signed the ORFEO ToolBox Contributor License Agreement.
<hr>
***Check before merging:***
- All discussions are resolved
- At least 2 :thumbsup: votes from core developers, no :thumbsdown: vote.
- The feature branch is (reasonably) up-to-date with the base branch
- Dashboard is green
- Copyright owner has signed the ORFEO ToolBox Contributor License Agreement7.0.0Guillaume PaseroGuillaume Paserohttps://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/merge_requests/443Link with CDASH2019-09-19T07:14:03ZAntoine RegimbeauLink with CDASH#### Summary
This MR is introducing a new widget to link CI build with the corresponding build summary in CDash.
#### Rationale
GitLab console is less clear than build summary of CDash, and in term we will have the test in GitLab and ...#### Summary
This MR is introducing a new widget to link CI build with the corresponding build summary in CDash.
#### Rationale
GitLab console is less clear than build summary of CDash, and in term we will have the test in GitLab and it will be nice to have sum up in CDash.
#### Implementation Details
This feature is implemented in python in the file `cdash_handler.py`. Tests have been added in the `CI/test` folder.
Basically we look for a file `Configure.xml` in the build tree (under the sub-directory `Testing`) to retrieve the needed information to query the buildid in CDash and then build the URL that we put in a new job in GitLab-CI.
For now what we choose is:
* this job is always passing
* the job name will be prefixed by cdash
You can have a look at the result on this MR.
#### Copyright
The copyright owner is *CNES* and has signed the ORFEO ToolBox Contributor License Agreement.
<hr>
***Check before merging:***
- All discussions are resolved
- At least 2 :thumbsup: votes from core developers, no :thumbsdown: vote.
- The feature branch is (reasonably) up-to-date with the base branch
- Dashboard is green
- Copyright owner has signed the ORFEO ToolBox Contributor License Agreement7.0.0https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/merge_requests/436Add Debian to CI2019-09-19T07:14:48ZAntoine RegimbeauAdd Debian to CI#### Summary
This MR has three goals:
* adding a Debian docker image
* changing the url for test submission
* modifying .gitlab.yml so that there is only one pipeline for merge request context
#### Rationale
The docker i...#### Summary
This MR has three goals:
* adding a Debian docker image
* changing the url for test submission
* modifying .gitlab.yml so that there is only one pipeline for merge request context
#### Rationale
The docker image aims at helping us in the release process. It will be used for merge request context (meaning that we won't be able to merge a branch if the build is failing on this image). The parameters of this build should be as close as the environment used be Debian packager.
Because of issue on migrating from CDash 2.2 to CDash 2.6, we have a new instance of CDash (https://cdash.orfeo-toolbox.org/). We can push build here now (When merged we will need to update some files in DevUtils).
#### Implementation
To allow compilation on the new platform some modifications have been made in `Modules/Adapters/GdalAdapters/include/otbOGRFieldWrapper.hxx`
#### Copyright
The copyright owner is *CNES* and has signed the ORFEO ToolBox Contributor License Agreement.
<hr>
***Check before merging:***
- All discussions are resolved
- At least 2 :thumbsup: votes from core developers, no :thumbsdown: vote.
- The feature branch is (reasonably) up-to-date with the base branch
- Dashboard is green
- Copyright owner has signed the ORFEO ToolBox Contributor License Agreement
7.0.0https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/merge_requests/385Allow CI to fail2019-09-19T07:22:15ZAntoine RegimbeauAllow CI to fail#### Summary
Ctest is always returning an exit value of 0 unless a message of type FATAL_ERROR or SEND_ERROR is made.
Which is now the case.
Please vote this MR while it is red, I will commit the revert once it has enough vote.
...#### Summary
Ctest is always returning an exit value of 0 unless a message of type FATAL_ERROR or SEND_ERROR is made.
Which is now the case.
Please vote this MR while it is red, I will commit the revert once it has enough vote.
#### Copyright
The copyright owner is *CNES* and has signed the ORFEO ToolBox Contributor License Agreement.
<hr>
***Check before merging:***
- All discussions are resolved
- At least 2 :thumbsup: votes from core developers, no :thumbsdown: vote.
- The feature branch is (reasonably) up-to-date with the base branch
- Dashboard is green
- Copyright owner has signed the ORFEO ToolBox Contributor License Agreement
7.0.0https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/merge_requests/582Script to setup a namespace for OTB imported targets2019-09-19T08:04:25ZGuillaume PaseroScript to setup a namespace for OTB imported targetsThis script allows to protect the imported target in OTB install with a namespace. By default, when doing `find_package(OTB)`, OTB libraries, applications and executables are declared as `IMPORTED` targets. But if a target name is in con...This script allows to protect the imported target in OTB install with a namespace. By default, when doing `find_package(OTB)`, OTB libraries, applications and executables are declared as `IMPORTED` targets. But if a target name is in conflict with your project, CMake will throw an error.
Let say I want to build SertitObject remote module as standalone. The target `otbapp_Aggregate` may be declared twice: in the OTB package, and in SertitObject sources.
With the script, `NamespaceHandler.cmake`, you can set a namespace in front of all `IMPORTED` targets of OTB install tree.
For instance, calling:
> ctest -DNAMESPACE:STRING="NS" -S NamespaceHandler.cmake
will transform `otbapp_Aggregate` into `NS::otbapp_Aggregate`. The double colon also has a special meaning for target names: CMake will check that the target is either an `IMPORTED` or `ALIAS`.
Note: this script will be used for the CI of remote module SertitObject, and maybe other remote modules that are already in OTB standalone packages.7.0.0Guillaume PaseroGuillaume Paserohttps://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/merge_requests/581Contributors check2019-09-19T08:04:31ZSébastien DinotContributors checkThe shell script and the modification of the CI script provided by this merge request aim to identify the unknown authors (i.e. authors of commits not listed in an external reference file) to invite the team to check if these authors hav...The shell script and the modification of the CI script provided by this merge request aim to identify the unknown authors (i.e. authors of commits not listed in an external reference file) to invite the team to check if these authors have already signed the contributor license agreement (CLA). When this is done, the authors must be added to the reference list.7.0.0https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/merge_requests/571BUG: fix parameter expansion in otbenv2019-09-19T08:04:53ZGuillaume PaseroBUG: fix parameter expansion in otbenvProblem with paths containing spaces (as shown [here](https://travis-ci.org/gpo-geo/remote-module-template-ci/jobs/568719671) ). Now uses `"$@"` for safe expansion of parameters.Problem with paths containing spaces (as shown [here](https://travis-ci.org/gpo-geo/remote-module-template-ci/jobs/568719671) ). Now uses `"$@"` for safe expansion of parameters.7.0.0Guillaume PaseroGuillaume Paserohttps://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/merge_requests/563CI move Superbuild Artifact repository2019-09-19T08:06:01ZGuillaume PaseroCI move Superbuild Artifact repositoryThe repository for XDK is moved to `orfeotoolbox` group.
Also, the `IMAGE_NAME` are renamed to OS-VERSION-COMPILER for more clarity.The repository for XDK is moved to `orfeotoolbox` group.
Also, the `IMAGE_NAME` are renamed to OS-VERSION-COMPILER for more clarity.7.0.0Guillaume PaseroGuillaume Paserohttps://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/merge_requests/559DOC: add badges for CI and coverage2019-09-19T08:06:26ZGuillaume PaseroDOC: add badges for CI and coverageAdd badges in the README.md
Given this small MR, I will skip the full MR pipeline on purpose. This is why I add the ~patch label.Add badges in the README.md
Given this small MR, I will skip the full MR pipeline on purpose. This is why I add the ~patch label.7.0.0Guillaume PaseroGuillaume Paserohttps://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/merge_requests/558CI: fixing cdash links on forks2019-09-19T08:06:30ZGuillaume PaseroCI: fixing cdash links on forksThe cdash links are a bit tricky when doing MR between a fork and the official repo:
* normally, the [statuses](https://docs.gitlab.com/12.0/ee/api/commits.html#post-the-build-status-to-a-commit) API should be used on the target project...The cdash links are a bit tricky when doing MR between a fork and the official repo:
* normally, the [statuses](https://docs.gitlab.com/12.0/ee/api/commits.html#post-the-build-status-to-a-commit) API should be used on the target project (because it is the one carrying the MR)
* however, the personal API token of the contributor may not allow to use Gitlab API on the official repository.
* but, it appears that using the fork ID and the default `CI_COMMIT_REF_NAME` as the `ref` parameter of `statuses` will send the links to the MR pipeline, maybe because in this particular case it also has the tag `latest` ...
Anyway, to illustrate this obscure issue, some example with different `id` and `ref` parameters for the `statuses` API:
* https://gitlab.orfeo-toolbox.org/atalon/otb/commit/64d95abbffc94732908588436e4636bbd9a78708/pipelines : pipeline 2090 with `id` = fork's ID and `ref = refs/merge-requests/541/head` : the MR is unknown in the fork project, so a blank pipeline is created (n° 2092) and received all the CDash links from pipeline 2090.
* https://gitlab.orfeo-toolbox.org/atalon/otb/commit/a15d589d8e4d438360874c89476e6f61f8d5632e/pipelines : pipeline 2262 with `id` = main repo ID and `ref = refs/merge-requests/555/head` : I get a HTTP Error 403, the user can't post statuses on the main repo. I manually ran the pipeline 2261 (`id` = fork's ID and `ref= $CI_COMMIT_REF_NAME` ) to see where the cdash link would go, and it went into pipeline 2262.
* https://gitlab.orfeo-toolbox.org/atalon/otb/commit/b77b5d957aa01f14ceb6af89211390097f46f2ff/pipelines : pipeline 2264 with `id` = fork's ID and `ref= $CI_COMMIT_REF_NAME` , the links are posted in the right pipeline.7.0.0Guillaume PaseroGuillaume Paserohttps://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/merge_requests/556CI simple contributing2019-09-19T08:06:51ZGuillaume PaseroCI simple contributingSmall updates for CONTRIBUTING.md (keep only the minimal level of information, details go to wiki).
Removed deprecated steps in HowToReleaseSmall updates for CONTRIBUTING.md (keep only the minimal level of information, details go to wiki).
Removed deprecated steps in HowToRelease7.0.0Guillaume PaseroGuillaume Paserohttps://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/merge_requests/550CI Artifacts2019-09-19T08:07:12ZGuillaume PaseroCI Artifacts* move some artifacts at the root level to be more visibles
* change name of the job generating documentation* move some artifacts at the root level to be more visibles
* change name of the job generating documentation7.0.0Guillaume PaseroGuillaume Paserohttps://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/merge_requests/543CI coverage2019-09-19T08:07:44ZGuillaume PaseroCI coverageBrings coverage estimation to the QA jobs:
* The coverage doesn't take into account the lines of code in:
- `Modules/*/*/test`
- `Modules/ThirdParty`
- `Examples`
* The coverage build is now in Debug
* Some tests have been m...Brings coverage estimation to the QA jobs:
* The coverage doesn't take into account the lines of code in:
- `Modules/*/*/test`
- `Modules/ThirdParty`
- `Examples`
* The coverage build is now in Debug
* Some tests have been modified to take less time (running the tests on this build is a lot slower)
* The QA jobs are disabled if no Sonar token is present
Although empty on SonarQube dashboard, the coverage after merging this branch should be around 59%7.0.0Guillaume PaseroGuillaume Paserohttps://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/merge_requests/539Add QA related jobs to CI pipeline2019-09-19T08:07:56ZSébastien DinotAdd QA related jobs to CI pipelineThis MR aims to add jobs related to QA (Quality Assurance) to OTB CI pipeline by:
1. Playing the unit tests and convert the report from CTest format to JUnit format (SonarQube only supports JUnit format for UT reports)
2. Running static...This MR aims to add jobs related to QA (Quality Assurance) to OTB CI pipeline by:
1. Playing the unit tests and convert the report from CTest format to JUnit format (SonarQube only supports JUnit format for UT reports)
2. Running static analysis on source code thanks to [CppCheck](http://cppcheck.sourceforge.net/)
3. Sending the below reports to SonarQube thanks to Sonar Scanner
The [resulting QA report](https://sonar.orfeo-toolbox.org/dashboard?id=orfeotoolbox-otb) is available in SonarQube.7.0.0https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/merge_requests/538Ci contributing update2019-09-19T08:08:03ZGuillaume PaseroCi contributing updateUpdate of contributing guidelines regarding new CI platform.
The CONTRIBUTING.md contains the essential user-oriented informations. The details are left in a [wiki page](https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/wikis/OTB-Contin...Update of contributing guidelines regarding new CI platform.
The CONTRIBUTING.md contains the essential user-oriented informations. The details are left in a [wiki page](https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/wikis/OTB-Continuous-Integration-platform).7.0.0Guillaume PaseroGuillaume Paserohttps://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/merge_requests/536CI Fixes episode N2019-09-19T08:08:13ZGuillaume PaseroCI Fixes episode NA bunch of fixes/improvements for new CI:
* Clarification of YAML structure
* Improvements of `cdash_handler` script
* Handling of CI on forks:
- deploy jobs reserved for main repository
- API token is now optional (warning message ...A bunch of fixes/improvements for new CI:
* Clarification of YAML structure
* Improvements of `cdash_handler` script
* Handling of CI on forks:
- deploy jobs reserved for main repository
- API token is now optional (warning message if missing token)
* Special job to build a container with a tagged release
* Reduce warnings for Linux packaging7.0.0Guillaume PaseroGuillaume Paserohttps://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/-/merge_requests/526CI Fix CDash error report2019-09-19T08:10:00ZGuillaume PaseroCI Fix CDash error reportRely on log files rather than CDash API to derive the job status (asynchronous submission issues).
Other fixes:
* artifacts fixed for documentation job
* no need to config git user anymore (done in docker images)
* doxygen only bui...Rely on log files rather than CDash API to derive the job status (asynchronous submission issues).
Other fixes:
* artifacts fixed for documentation job
* no need to config git user anymore (done in docker images)
* doxygen only built for develop and release profiles.
7.0.0Guillaume PaseroGuillaume Pasero