Back to basics: the pyramid of tests
Testing embedded products can be challenging, but it is essential to ensure the reliability and functionality of these complex systems. To comprehend the hierarchy of tests, we turn to the well-known Pyramid of Tests, which is a framework for organizing testing efforts in an IoT/embedded project.
Unit testing
At the base of the pyramid lies unit testing. It involves the automated validation of specific components, algorithms, or functions within the code. Unit tests evaluate individual parts of the software with various parameters to ensure they function as expected.
Component testing
Moving up the pyramid, we encounter component testing. This level involves testing larger components or subsystems, such as an embedded system driver.
Integration testing
Integration tests examine how different components behave when assembled. This level of testing focuses on the interaction between various layers and services in an embedded solution, such as OTA updates, which require coordination between cloud platforms, firmware uploads, and data reporting.
System testing
At the peak of the pyramid is system testing. This level employs end-to-end testing, simulating user interactions with the complete IoT system. System tests validate all business features, providing a holistic view of the system’s functionality.
The pyramid of tests: from unit testing to system testing
What does on-board automated testing mean?
Automated testing for embedded and IoT devices comes in two flavors: “on-board” and “off-board”. On-board automated testing refers to testing on real hardware or in actual cloud environment. It is performed automatically but takes place within the real integration environment.
Onboard tests are ideal for checking system behaviors that are specific to the actual deployment environment.
What’s the difference between manual testing and automated testing?
It’s crucial to differentiate between manual testing and on-board automated testing. Manual testing remains essential for various aspects of product development. While on-board automated testing offers numerous benefits, manual testing is irreplaceable for specific tasks that require human intervention and expertise. The key is to strike a balance between manual and automated testing.
What are the benefits of automated testing?
Reducing the development-to-testing lag
One significant challenge in embedded development is the time gap between feature development and testing. In traditional testing methods, this gap can be quite substantial, making it more difficult to fix bugs or issues that emerge during testing. The aim of automated testing is to close this gap by allowing teams to discover issues earlier in the development process, making bug fixing quicker and more efficient.
Behavioral testing
Automated testing covers standard behavioral tests, akin to what manual testing accomplishes. This includes verifying that the software meets its expected functional requirements.
Edge case management
Automated testing is capable of managing edge cases that can be difficult and time-consuming to address manually. This ensures that the software is robust and reliable under various conditions.
Non-regression testing
While some non-regression testing can still be performed manually, automated testing can handle a substantial portion of these tests, ensuring that new changes do not break existing functionality.
Endurance testing
This type of testing examines how the system performs over time under normal usage. Automated testing is particularly valuable for such long-duration tests.
Robustness testing
Testing the system’s robustness, especially in edge cases, is best achieved with automated testing.
Performance measurement
Automated testing allows for comprehensive performance measurements, which are difficult to achieve manually. This includes assessing how well the system performs under various conditions and loads.
Automated testing not only reduces development-to-testing lag but excels in various critical areas, making it an irreplaceable asset in our quest for reliability and functionality in every innovation. That’s why we developed Pluma, an automated testing tool dedicated to embedded and IoT devices.
