Cover image for Qemu emulating Raspberry Pi 4b Linux module development environment

Qemu emulating Raspberry Pi 4b Linux module development environment

Created:Updated:
Linux
, 
Words 2.5k
Views
Visitors
Timeline

Timeline

2025-12-31

init

2025-01-14

add buildroot

This article introduces the method of using Qemu to emulate a Raspberry Pi 4b Linux module development environment, detailing the selection and configuration of cross-compilers, the compilation of the kernel and root file system, and discussing the configuration options and precautions when customizing the system based on Buildroot and Yocto build systems.

Environment

Host OS

Using Arch Linux with WSL2

Environment
Environment

Using Ubuntu is also possible, refer to:

WSL Ubuntu20.04Installation and Configuration

https://even629.com/posts/2512299

compiler

ARM has currently released a total of 8 architectures: ARMv1, ARMv2, ARMv3, ARMv4, ARMv5, ARMv6, ARMv7, ARMv8.

For processors supporting the ARMv8 instruction set, you can use-march=armv8-aparameters to compile code, the ARM GNU compiler can be downloaded via the link below

Arm GNU Toolchain Downloads

https://developer.arm.com/downloads/-/arm-gnu-toolchain-downloads

For example, the aarch64-linux-gnu-gcc version downloaded via pacman on Arch Linux is 15.1.0, and its files are in a flat layout. It is not suitable for the buildroot compilation toolchain.

aarch64-linux-gnu-gcc
aarch64-linux-gnu-gcc

It is recommended to use the compiler from the official ARM website, which is portable and ready to use after decompression.

This article uses the 11.2 version of the cross-compiler

aarch64-none-linux-gnu 11.2

https://developer.arm.com/-/media/files/downloads/gnu/11.2-2022.02/binrel/gcc-arm-11.2-2022.02-x86_64-aarch64-none-linux-gnu.tar.xz

aarch64-none-linux-gnu 11.2

https://pan.quark.cn/s/c5681aa0b3ef

Extraction code:71g1

You can add it to the environment variables

1
2
3
# ~/.bashrc or ~/.bash_profile
emacs ~/.bashrc
export PATH="$HOME/tools/gcc-arm-11.2-2022.02-x86_64-aarch64-none-linux-gnu/bin:$PATH"

Using version 11.2.1 of aarch64-none-linux-gnu-gcc

aarch64-none-linux-gnu-gcc
aarch64-none-linux-gnu-gcc

Compile the kernel

1
2
3
4
5
6
7
8
9
10
11
sudo pacman -Syu bc
git clone --depth=1 --branch rpi-5.10.y --single-branch https://github.com/raspberrypi/linux.git linux-5.10.y-raspi
make mrproper
make ARCH=arm64 CROSS_COMPILE=aarch64-none-linux-gnu- bcm2711_defconfig
make ARCH=arm64 CROSS_COMPILE=aarch64-none-linux-gnu- modules_prepare
make ARCH=arm64 CROSS_COMPILE=aarch64-none-linux-gnu- Image modules dtbs -j$(nproc)

# Place the compiled kernel image and device tree in the specified directory
mkdir -p ~/tftp/raspi4b
cp arch/arm64/boot/Image ~/tftp/raspi4b
cp arch/arm64/boot/dts/broadcom/bcm2711-rpi-4-b.dtb ~/tftp/raspi4b

Output:

  • arch/arm64/boot/Image
  • arch/arm64/boot/dts/broadcom/bcm2711-rpi-4-b.dtb

Compile the root file system

busybox

Reference:

MinimalLinuxSystem compilation and running

https://even629.com/posts/42869

Note that this blog uses the aarch64-linux-gnu-gcc compiler downloaded via pacman on Arch Linux. It is best to keep the compiler consistent with the one used to compile the kernel, so here we choose aarch64-none-linux-gnu-gcc

When copying dynamic library files, copy those from the compiler directory:

1
2
cp ~/tools/gcc-arm-11.2-2022.02-x86_64-aarch64-none-linux-gnu/aarch64-none-linux-gnu/libc/lib/ld-linux-aarch64.so.1 ./lib/
cp ~/tools/gcc-arm-11.2-2022.02-x86_64-aarch64-none-linux-gnu/aarch64-none-linux-gnu/libc/lib64/* ./lib64/

After compilation, enter the Linux kernel code and install the driver:

1
2
3
4
5
6
7
# Install driver, copy to root filesystem
cd /home/zhaohang/repository/linux/linux-5.10.y-raspi
# busybox
make ARCH=arm64 \
     CROSS_COMPILE=aarch64-none-linux-gnu- \
     INSTALL_MOD_PATH=/home/zhaohang/repository/linux/busybox-1.37.0/_install/ \
     modules_install

buildroot

The default toolchain of Archlinux and the cross toolchain provided by AUR cannot be copied and used, because Buildroot copies the toolchain to the working directory for execution during the build process, therefore we need tochoose a Portable toolchain。

1
2
3
4
5
6
7
wget https://buildroot.org/downloads/buildroot-2025.11.tar.gz
# PATH under WSL contains spaces, temporarily reset it
export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/usr/lib/wsl/lib

tar xvf buildroot-2025.11.tar.gz
cd buildroot-2025.11
make menuconfig

The configuration is as follows

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
Target Options  --->
	Target Architecture (AArch64 (little endian))  --->
 	Target Architecture Variant (cortex-A72)  --->
 	Floating point strategy (FP-ARMv8)  --->
 	MMU Page Size (4KB)  --->
 	Target Binary Format (ELF)  --->

Toolchain  --->
	Toolchain type (External toolchain)  --->
  	*** Toolchain External Options ***
	Toolchain (Custom toolchain)  --->
	Toolchain origin (Pre-installed toolchain)  ---> (/home/zhaohang/tools/gcc-arm-11.2-2022.02-x86_64-aarch64-none-linux-gnu) Toolchain path
	($(ARCH)-none-linux-gnu) Toolchain prefix
	External toolchain gcc version (11.x)  --->
	External toolchain kernel headers series (4.20.x)  --->
	External toolchain C library (glibc)  --->
	[*] Toolchain has SSP support? (NEW)
	[*]   Toolchain has SSP strong support? (NEW)
	[ ] Toolchain has RPC support? (NEW)
	[*] Toolchain has C++ support?
	[ ] Toolchain has D support? (NEW)
	[*] Toolchain has Fortran support?
	[*] Toolchain has OpenMP support?
	[ ] Copy gdb server to the Target (NEW)


System configuration  --->
        /dev management (Dynamic using devtmpfs + mdev)  --->
	[*] Enable root login with password (NEW)
	(root) Root password


Filesystem images  --->
	[*] ext2/3/4 root filesystem
		ext2/3/4 variant (ext4)  --->
	(rootfs) filesystem label
	(512M) exact size

Do not enable the options in the kernel here (they are disabled by default), because we want to compile the kernel ourselves; do not enable the bootloader options either, because using qemu to boot does not require a bootloader.

External toolchain kernel headers series (4.20.x) here refers to the kernel version corresponding to the toolchain, which can be viewed in thegcc-arm-11.2-2022.02-x86_64-aarch64-none-linux-gnu/aarch64-none-linux-gnu/libc/usr/include/linux/version.hdirectory, here it should be 4.20.x

Compile:

1
2
sudo pacman -Syu unzip cpio rsync
make

Enter the linux kernel directory, install the driver

1
2
3
4
5
6
7
8
9
10
11
12
# Install driver, copy to root filesystem
cd /home/zhaohang/repository/linux/linux-5.10.y-raspi

# buildroot
make ARCH=arm64 \
     CROSS_COMPILE=aarch64-none-linux-gnu- \
     INSTALL_MOD_PATH=/home/zhaohang/repository/linux/buildroot-2025.11/output/target \
     modules_install

make
# Compile to generate rootfs.ext4
cp output/images/rootfs.ext4 ~/tftp

yocto

1
2
3
4
5
6
7
mkdir -p ~/repository/linux/yocto
cd ~/repository/linux/yocto
# use git to cline bitbake-setup
git clone https://git.openembedded.org/bitbake

sudo pacman -Sy chrpath diffstat inetutils rpcsvc-proto
python3 ./bitbake/bin/bitbake-setup init --non-interactive poky-whinlatter poky distro/poky machine/qemuarm64

Yocto does not automatically recognize any external toolchain!
There must be a dedicated layer to provide support for this toolchain (viatcmode-*.incandtoolchain-*.conffile).
Common support layers:meta-arm→ Supports ARM official GNU Toolchain

Yocto Project / OpenEmbedded build system available in distribution configuration

  1. poky-master

    • Poky is the reference distribution of the Yocto Project.

    • -masterindicates the use of the latest development main branch code, which is the unreleased, under-development version。

  2. oe-nodistro-master

    • OpenEmbedded (OE) is the underlying build framework of the Yocto Project.

    • nodistroIndicates “no distribution” mode —Not bound to any specific distribution policy, only providing the most basic build capabilities.

    • -masterAlso indicates using development mainline。

  3. oe-nodistro-whinlatter

    • Based on OpenEmbedded ofnodistroconfiguration;

    • Use Yocto 5.3 version codename “whinlatter” stable branch;

    • Is a long-term support version;

    • nodistroIndicates “no distribution” mode.

  4. poky-whinlatter

    • Poky distribution of 5.3 “whinlatter” stable version；

    • Includes complete default configurations (such as using systemd, optional RPM/deb package formats, default toolchain, etc.);

    • Is a long-term support version;

    • Is the reference platform officially tested and certified by the Yocto Project.

“whinlatter” is the codename for Yocto Project 5.3(Yocto versions have been named after birds since 4.0, 5.3 = Whinchat + Lark → “Whinlatter”).

BitBake build configuration (layers / templates)

  1. poky

    • This is Standard Poky build configuration。

    • All packages (recipes) will be built completely from source (unless you already have a local cache).

    • Secure, reliable, and predictable behavior.

  2. poky-with-sstate

    • Enabled Remote shared state (sstate) cache。
    • During the build, it will attempt to fetch from Yocto official or specified sstate mirror servers Download pre-compiled intermediate artifacts (such as compiled libraries, kernel modules, etc.), therebysignificantly speeding up the build process。
    • but there are strict prerequisites：
    • your local network must be very stable and have sufficient bandwidth；
    • must be able to access external sstate servers (usually requires internet);
    • if the network is interrupted or the mirror does not match, it may lead to build failure or inconsistency;
    • “Use with caution” is exactly a reminder of this.

sstate is Yocto’s caching mechanism: it saves the output of tasks (such asdo_compileresults), which can be directly reused next time the same content is built, without redoing it.

Target Machine

  1. machine/qemux86-64

    • PC simulating the x86_64 architecture, using QEMU virtual machine。

    • based on standard PC hardware models (such as Intel/AMD 64-bit CPUs, common peripherals).

  2. machine/qemuarm64

    • Simulating the ARM64 (AArch64) architecture virtual devices (such as systems based on Cortex-A57).

    • Also runs via QEMU, without the need for a real ARM board.

  3. machine/qemuriscv64

    • Simulating the RISC-V 64-bit architecture virtual machine.

  4. machine/genericarm64

    • Generic ARM64 real hardware reference configuration (not an emulator).

    • Not tied to specific boards (such as Raspberry Pi, BeagleBone), but provides a “generic” ARM64 BSP.

    • Need to flash the generated image toReal ARM64 development boardrunning on.

    • May lack specific board-level drivers (such as GPU, WiFi), requiring manual adaptation.

  5. machine/genericx86-64

    • Generic x86_64 real hardware reference configuration (e.g., regular PC, Intel NUC, industrial computer).

    • The generated image can be directly written to a USB drive and booted on a real x86_64 computer.

Distribution configuration variants

  1. distro/poky

    • Standard Poky distribution configuration. Includes a complete Linux system: glibc, systemd, package manager (RPM or IPK), common tools (bash, coreutils, networking, etc).
    • Uses systemd as the init system by default.
    • Suitable for general development, learning, and most embedded application scenarios.
    • The image is large (a few hundred MB), but fully functional.

  2. distro/poky-altcfg

    • Alternative Poky configuration

    • Usually used to test different combinations of underlying components, for example:

      • Use musl libc Alternative to glibc;
      • Use busybox + sysvinit Alternative to systemd;
      • Smaller default image or different security policies.

    • Specific behavior depends on the Yocto version,Less documentation, stability is not as good as standard Poky。

    • Mainly used for Yocto internal testing or advanced customization scenarios。

  3. distro/poky-tiny

    • Minimalist Poky, designed specifically for resource-constrained devices.
    • Features:
      • Use musl libc(smaller, faster);
      • Use busybox + sysvinit(no systemd);
      • Removed a large number of non-essential packages;
      • The final root filesystem can be as small as 10–20 MB。
    • Limited functionality: may lack network tools, simplified shell features, no package management.
    • Suitable for: microcontroller-level applications, minimal environments after bootloader, safety-critical systems.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
# Enter the build environment
cd /home/zhaohang/repository/linux/yocto/bitbake-builds/poky-whinlatter

# Configuration
# Format of the generated filesystem
emacs build/conf/local.conf
# Add the following content
IMAGE_FSTYPES = "ext4 cpio.gz"
# Specify the toolchain installation path
# EXTERNAL_TOOLCHAIN = "/home/zhaohang/tools/gcc-arm-11.2-2022.02-x86_64-aarch64-none-linux-gnu"

# Activate the current build
source ./build/init-build-env
# Detect the current configuration
bitbake-config-build list-fragments

# Build minimal rootfs
bitbake core-image-minimal

Kernel module

Kernel module example code

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
#include <linux/init.h>
#include <linux/module.h>

static int __init hello_world_init(void){
        printk(KERN_INFO "hello world!\n");
        return 0;
}

static void __exit hello_world_exit(void){
        pr_info("hello world module exit\n");
}
module_init(hello_world_init);
module_exit(hello_world_exit);

MODULE_LICENSE("GPL");
MODULE_AUTHOR("even629<asqwgo@outlook.com>");
MODULE_DESCRIPTION("hello world!");

external module

Here, the busybox root filesystem is taken as an example

Makefile

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
DRIVER_NAME := hello_world

obj-m += $(DRIVER_NAME).o
KERNEL_SRC:=/home/zhaohang/repository/linux/linux-5.10.y-raspi

PWD ?=$(shell pwd)
ARCH = arm64
CROSS_COMPILE = aarch64-none-linux-gnu-


all: build

build:
	$(MAKE) ARCH=$(ARCH) CROSS_COMPILE=$(CROSS_COMPILE) -C $(KERNEL_SRC) M=$(PWD) modules

clean:
	$(MAKE) ARCH=$(ARCH) CROSS_COMPILE=$(CROSS_COMPILE) -C $(KERNEL_SRC) M=$(PWD) modules clean
	rm -rf *.ko *.o *.mod.o *.mod.c *.symvers *.order .tmp*

.PHONY: all deploy qemu
deploy:
	cp $(DRIVER_NAME).ko /home/zhaohang/repository/linux/busybox-1.37.0/_install
	/bin/bash /home/zhaohang/repository/linux/busybox-1.37.0/deploy.sh

qemu:
	qemu-system-aarch64 \
		-M raspi4b \
		-cpu cortex-a72 \
		-m 2G \
		-nographic \
		-kernel /home/zhaohang/tftp/raspi4b/Image \
		-dtb /home/zhaohang/tftp/raspi4b/bcm2711-rpi-4-b.dtb \
		-initrd /home/zhaohang/tftp/initramfs.cpio.gz \
		-append "console=ttyAMA0 earlycon=pl011,0xfe201000 rdinit=/linuxrc"

The script that packages the root filesystemdeploy.shAs follows:

1
2
3
4
5
6
7
8
9
10
11
12
13
#!/bin/bash

INSTALL_PATH=/home/zhaohang/repository/linux/busybox-1.37.0/_install

cd "$INSTALL_PATH" || exit 1

rm -rf ../initramfs.cpio.gz ~/tftp/initramfs.cpio.gz

find . | cpio -o -H newc | gzip -c > ../initramfs.cpio.gz

cp ../initramfs.cpio.gz ~/tftp

echo "cp initramfs.cpio.gz ~/tftp"

Encountered a problem where there is no print when compiling with aarch64-linux-gnu-gcc on WSL2 archlinux (aarch64-none-linux-gnu-gcc also doesn’t work), haven’t found a solution yet, described the specific problem in detail on stackoverflow:

No gcc compiling log CC [M] showing up

https://stackoverflow.com/questions/79858012/

However, this problem does not occur when using WSL2 Ubuntu20.04.

built-in module

Indrivers/char(Taking character driver as an example) Create a folder named helloworld, then put the driver source code into it, and then create a Kconfig file

1
2
3
4
5
config HELLO_WORLD
  bool "helloworld support"
  default y
  help
      helloworld

Modify drivers

1
2
3
emacs ../Kconfig
# Add
source "drivers/char/helloworld/Kconfig"

Create a Makefile in the driver source code

1
obj-$(CONFIG_helloworld) += helloworld.o

Then add to the upper-level Makefile:

1
2
3
emacs ../Makefile
# Add
obj-y += helloworld/

Then compile the kernel

qemu run

qemu version

1
2
3
4
$ sudo pacman -Syu qemu-system-aarch64
$ qemu-system-aarch64 --version
QEMU emulator version 10.1.2
Copyright (c) 2003-2025 Fabrice Bellard and the QEMU Project developers

busybox root filesystem

Here we compile busybox into initramfs.cpio.gz as the value of the initrd parameter.

Run

1
2
3
4
5
6
7
8
9
qemu-system-aarch64 \
  -M raspi4b \
  -cpu cortex-a72 \
  -m 2G \
  -nographic \
  -kernel /home/zhaohang/tftp/raspi4b/Image \
  -dtb /home/zhaohang/tftp/raspi4b/bcm2711-rpi-4-b.dtb \
  -initrd /home/zhaohang/tftp/initramfs.cpio.gz \
  -append "console=ttyAMA0 earlycon=pl011,0xfe201000 rdinit=/linuxrc"

look up bycommand + a, then press x to exit qemu

buildroot root filesystem

In buildroot, we use an SD card to mount the filesystem

1
2
3
4
5
6
7
8
9
$ qemu-system-aarch64 \
  -M raspi4b \
  -cpu cortex-a72 \
  -m 2G \
  -nographic \
  -kernel /home/zhaohang/tftp/raspi4b/Image \
  -dtb /home/zhaohang/tftp/raspi4b/bcm2711-rpi-4-b.dtb \
  -sd /home/zhaohang/tftp/rootfs.ext4 \
  -append "console=ttyAMA0 earlycon=pl011,0xfe201000 root=/dev/mmcblk1 rw rootwait"

look up bycommand + a, then press x to exit qemu

yocto root filesystem

Use initramfs.cpio.gz or use an SD card to mount the filesystem

1
2
3
4
5
6
7
8
9
$ qemu-system-aarch64 \
  -M raspi4b \
  -cpu cortex-a72 \
  -m 2G \
  -nographic \
  -kernel /home/zhaohang/tftp/raspi4b/Image \
  -dtb /home/zhaohang/tftp/raspi4b/bcm2711-rpi-4-b.dtb \
  -initrd /home/zhaohang/repository/linux/yocto/bitbake-builds/poky-whinlatter/build/tmp/deploy/images/qemuarm64/core-image-minimal-qemuarm64.rootfs.cpio.gz \
  -append "console=ttyAMA0 earlycon=pl011,0xfe201000 init=/sbin/init"

Reference

[1]
InQEMUboot inU-Bootand kernel
[2]
Arch Linuxconfigure using the pre-installed toolchain underBuildrootsolution for
[3]
Yocto Project