Project context
This project aimed to document a reproducible methodology of boot time optimization for our dev team and clients on iMX93 (and on other boards in the future).
- Hardware target: MSC SM2S-IMX93 board equipped with an NXP i.MX93 SoC
- Software: Welma, our custom Yocto Linux distribution
Initial boot time on iMX93: 13.2s
Measuring boot time
Before optimizing boot time on i.MX93, it must be measured and measured well. We used three complementary approaches:
- Serial logs: easy to implement with Grabserial, but imprecise and intrusive
- Internal clocks: via BOOTSTAGE (U-Boot), PRINTK_TIME (kernel), and systemd-analyze. Accurate, but dependent on system stability
- GPIO + logic analyzer: the most reliable method. A GPIO is triggered at each key boot stage, and the logic analyzer measures timings with millisecond precision
Linux boot sequence & initial boot time
The boot process on iMX93 follows a well-defined software chain:
- Hardware initialization: clocks, voltages, etc.
- ROM code: loads U-Boot SPL from eMMC
- U-Boot SPL: initializes DRAM, launches TF-A
- TF-A & OP-TEE OS: sets up the Trusted Execution Environment
- U-Boot proper: loads the kernel, initramfs, and device tree
- Linux kernel: system initialization, mounts initramfs
- systemd: manages services and launches the application
Initial boot time: 13.2 seconds – far too long for our target.
5 optimization levers
1. Removing U-Boot delay
By default, U-Boot proper introduces a 2-second delay to allow user interruption. While useful during development, it’s unnecessary in production.
We disabled this delay via U-Boot configuration options, instantly saving 2 seconds.
Gain: 2 seconds
Remaining time: ~11.2 s
2. Reducing kernel and systemd logs
Serial logs slow down boot time, especially when verbose. Linux uses log levels from 0 (emergency) to 7 (debug). Initially set to 7, all logs were displayed.
We progressively reduced the level:
- From 7 to 6: removed debug logs → ~1.9 s gain
- From 6 to 5: marginal gain of ~0.2 s
- Below 5: negligible impact
We then activated the quiet parameter, which:
- Sets log level to 4
- Makes systemd silent (logs only errors)
This added ~4 seconds of gain, while retaining critical logs for diagnostics.
Gain: 4.3 seconds
Remaining time: ~6.9 s
3. Launching the application before systemd
DOOM only requires access to the LVDS screen and a USB keyboard. It doesn’t rely on systemd-managed services like networking.
We modified the initramfs init script to launch DOOM as soon as devfs is mounted, before systemd starts. This involved:
- Including DOOM and its dependencies in the initramfs
- Launching the app in the background via the init script
Though rudimentary (no process supervision), this approach reduced boot time by ~40%.
Gain: 2.5 seconds
Remaining time: ~4.37 s
4. Reducing kernel features
The Linux kernel offers extensive configurability through a vast number of optional features. Many of these are defined as tristate options, meaning they can be:
- Disabled
- Built directly into the kernel
- Compiled as loadable modules (.ko files)
When an option is built into the kernel, its object file is linked into the final kernel executable and initialized during boot—regardless of whether it is actually used. In contrast, options compiled as modules are built separately and only loaded by the kernel at runtime if needed (for example, when a new peripheral is connected).
In our case, DOOM is launched before systemd starts, and module files are stored in the rootfs, not in the initramfs. Therefore, any option compiled as a module has no impact on boot time.
We conducted a thorough review of kernel options and disabled or converted to modules those that were unnecessary for system startup and for running DOOM. This was done using Linux’s menuconfig interface, which organizes kernel options into categories and allows us to inspect descriptions and dependencies.
This optimization led to:
- A significant reduction in the size of the uncompressed kernel (from 31 MB to 13 MB)
- A boot time improvement of approximately 0.6 seconds
Though time-consuming, this step was essential for system refinement.
Gain: 0.6 seconds
Remaining time: ~3.78 s
5. Enabling Falcon mode in U-Boot SPL
Falcon mode allows U-Boot SPL to load the kernel directly, bypassing U-Boot proper.
By skipping this intermediate stage, we significantly reduced the number of boot steps and shaved valuable time off the startup sequence. However, Falcon mode comes with several constraints and requires substantial engineering effort:
Limitations:
- No initramfs support (we embedded it in the kernel image)
- No device tree overlay handling (prepared during first boot via U-Boot proper)
- SPL size limit (160 KiB on i.MX93)
Required adaptations:
- Upgraded U-Boot to version 2024.04
- Activated LTO to reduce SPL size
- Disabled non-essential SPL options (watchdog, ext4, etc.)
- Adjusted Welma’s A/B update system (to be handled in SPL)
Despite over a month of effort, Falcon mode reduced boot time to 1.86 seconds.
Gain: 1.9 seconds
Remaining time: ~1.86 s
Results: achieving a boot time on iMX93 of 1.8s
After applying all optimizations, we achieved a boot time of 1.81 seconds of our embedded software system on iMX93, broken down as follows:
This project demonstrates that it is possible to drastically reduce the boot time on iMX93 of an embedded Linux system without compromising stability or maintainability.
Through a methodical approach and targeted technical choices, we’ve established a reproducible foundation for customers and partners deploying Welma Yocto Linux distribution on embedded platforms.
Written by Théo Gaigé, engineer at Witekio.
