In-Tree or Out-of-Tree Linux Driver: How to Choose

Choosing whether to keep a Linux driver in the kernel source tree or build it separately affects maintenance, compatibility, and distribution. This guide explains the trade-offs and gives a practical decision process for kernel developers and system integrators.

What You’ll Need

• A Linux development environment with a kernel source tree (if you plan to build in-tree).
• Kernel headers for the target system (for out-of-tree builds).
• Basic knowledge of C and the Linux kernel build system (Kbuild).
• Access to the kernel documentation, especially Documentation/process/submitting-patches.rst for in-tree submissions.

Step-by-Step

1. Identify the driver’s purpose and audience. Ask: Is this a generic device driver that many users will need, or a vendor-specific driver for a niche or proprietary device? In-tree is for widely useful, well-tested code; out-of-tree suits experimental or proprietary drivers.
2. Check the kernel’s staging area. If your driver is not yet polished, consider drivers/staging—it allows in-tree development while still being marked as unstable. This is a middle ground.
3. Evaluate the maintenance burden. In-tree drivers are updated with every kernel release, and API changes are handled by the community. Out-of-tree drivers must be updated manually for each kernel version, which can be time-consuming.
4. Consider distribution requirements. If you want your driver included in mainline distros (Ubuntu, Fedora, etc.), it must be in-tree. Out-of-tree drivers require users to compile or install DKMS packages.
5. Assess the licensing. In-tree drivers must be GPL-compatible. If your driver uses a proprietary license, you cannot submit it to the kernel; out-of-tree is your only option.
6. Test with the latest kernel. Before deciding, try building your driver against a recent mainline kernel. If it breaks often due to API changes, in-tree is more sustainable.
7. Make the final choice:
   • Choose in-tree if: the driver is for a common device, you can maintain it long-term, and you are willing to follow kernel coding style and review processes.
   • Choose out-of-tree if: the driver is proprietary, not ready for public review, or only needed for a specific internal project.

Common Mistakes

• Assuming in-tree is always better. If you cannot commit to long-term maintenance, an in-tree driver that becomes unmaintained will be removed, causing more harm.
• Ignoring kernel API changes. Out-of-tree drivers often break with new kernels; always test against the latest stable version.
• Submitting code without checking coding style. The kernel has strict style rules (see scripts/checkpatch.pl). Failing them delays acceptance.
• Not using DKMS for out-of-tree drivers. DKMS automatically rebuilds modules when the kernel updates, avoiding manual recompilation.

Tips

• Start with out-of-tree development, then propose for in-tree once the driver is stable and well-tested.
• Use the kernel’s modinfo and modprobe tools to verify that your out-of-tree module loads correctly.
• For in-tree, join the relevant mailing list (e.g., linux-kernel@vger.kernel.org) and follow the discussion to understand expectations.
• Keep a changelog of kernel version compatibility for out-of-tree drivers.

Troubleshooting

• Driver fails to compile against a new kernel. Check the kernel’s Documentation/process/deprecated.rst for removed APIs. Update your code accordingly.
• In-tree submission is rejected. Read the reviewer feedback carefully. Common issues: missing documentation, improper use of kernel APIs, or lacking a signed-off-by line.
• Out-of-tree module won’t load. Run dmesg to see the error. Often it’s a version mismatch or missing symbol. Ensure you built against the exact kernel headers of the running kernel.
• DKMS build fails. Verify that the DKMS configuration (dkms.conf) has the correct module name and source directory.

Frequently Asked Questions

Can I move a driver from out-of-tree to in-tree later?

Yes, many drivers start out-of-tree and are later accepted into the kernel. The process is the same as submitting a new driver.

Does in-tree guarantee better performance?

Not necessarily. Performance depends on the driver’s code, not its location. In-tree drivers benefit from community optimization, but that’s not guaranteed.

What is the staging tree exactly?

Staging is a part of the kernel source where drivers that need work are placed. They are compiled but marked as experimental. It’s a way to get in-tree visibility while still being developed.

Can I use out-of-tree drivers with secure boot?

Yes, but you need to sign the modules with a key trusted by your UEFI firmware. Tools like mokutil can enroll your key.

How often do kernel APIs change?

Every release can bring changes, but many are backward-compatible. Major changes happen occasionally, and out-of-tree drivers must adapt each time.

Is it worth the effort to get a driver in-tree?

If you want your driver to be widely used and maintained by the community, yes. It also ensures long-term compatibility and reduces your maintenance burden.

Related

• Battlefield 6 Graphics Settings for High FPS
• Fix Microsoft Account Wasn’t Changed to Local Account
• Word vs WordPad: Key Differences in a Table