Welma test bench
Let’s take a moment to deep dive into the test bench implemented for Welma Yocto Linux.
2 min
3 min
5 min
3 min
3 min
3 min
4 min
3 min
5 min
2 min
2 min
A test bench is an environment designed to run tests (and in our case, automated tests). The concept is simple: have a set of reference equipment (our supported hardware targets) integrated into a test bank. On these targets, we automatically execute scenarios driven by our Continuous Integration (CI) pipeline.
The goal: validate Welma’s features, ensure stability, and detect regressions as soon as they appear.
Why a test bench matters for Yocto dev
- Quality and peace of mind: regular tests (daily, weekly) that confirm features stability.
- Guaranteed non-regression: all tests are executed after each new feature release to confirm there is no adverse impact.
- Agility and automation: in a YoctoProject context, integration is complex (kernel, OS, application) and automation is always a great help.
- Distributed collaboration: in remote, multidisciplinary teams, the test bench becomes a key asset for effective collaboration and meeting consistent quality standards.
How we implemented our test bench in Welma
We built our test bench using two complementary tools:
- Pluma, a test automation framework from The Embedded Kit ecosystem,
- and GitLab CI, which orchestrates our continuous integration.
The process is simple and fully automated:
- Image compilation: At the end of a build, before a release, or during a merge request, the CI generates a software image.
- New test suite: Instructions go to the test environment.
- Test execution: Pluma downloads and deploys the new image on the target. It then runs the predefined test suite to validate functionalities.
- Result visualization: Results are displayed in GitLab dashboards, thanks to JUnit report format support in Pluma & Gitlab.
Each hardware target we want to test is paired with a companion Raspberry Pi. This Raspberry Pi hosts the GitLab Runner and Pluma, enabling smart test distribution. Each runner is configured with a specific GitLab tag that identifies the connected board or device. This way, the CI knows exactly which test suite trigger to which runner.
Once the image is deployed, Pluma executes a set of tests covering Welma’s key features. This test suite is included with the distribution but is fully extensible: you can add your own scenarios, whether low-level tests (custom drivers, serial or network interfaces, sensors, …) or application-level tests simulating user interactions.
The test bench infrastructure includes Ethernet and serial connectivity, as well as a programmable power supply to reboot targets on demand. Wi-Fi connectivity is also supported by Pluma.
The test bench is essential to guarantee the quality of any Yocto-based product.
In an environment where kernel, OS, and applications coexist, automation is the key to reducing risks and delivering with confidence. Within Welma offering, the test bench is a central element of our delivery process.