QEMU Address Space Abstraction

Timeline

Timeline

2025-11-21

1. init

This article introduces the core mechanism of address space abstraction in QEMU, elaborating on two key concepts: AddressSpace and MemoryRegion. Starting from the CPU memory access flow, it explains how QEMU distinguishes different devices, implements discrete mapping and remapping through address space management. The article shows an example of the virtual machine address space layout, explains the overlapping handling of memory-region and the role of the alias mechanism, and discusses the assignment process of key fields during initialization. Through the discussion in this article, readers can gain an in-depth understanding of the underlying implementation principles of QEMU's simulation of memory and peripherals.

Environment

Source code

wget https://download.qemu.org/qemu-10.1.2.tar.xz
tar xvJf qemu-10.1.2.tar.xz
cd qemu-10.1.2
mkdir -p output
./configure --prefix=$PWD/output --target-list=aarch64-softmmu,riscv64-softmmu --enable-debug
bear -- make -j$(nproc)

Create .clangd

CompileFlags:
  Add: -Wno-unknown-warning-option
  Remove: [-m*, -f*]

gdb

gdb -args ./build/qemu-system-riscv64 -M virt -device edu,id=edu1 -nographic

Basic Introduction

From the CPU’s perspective, all memory access operations are performed on addresses (load/store). The CPU does not care what device corresponds to the address, as long as it can read/write the correct result.

CPU Memory Access Flow

1. After the CPU calculates the target address (via the Arithmetic Logic Unit, ALU), it sends it to the address bus, and at the same time the CPU also provides read/write control signals;
2. **The device corresponding to the address may be ordinary memory or an I/O device (**specifically referring to peripherals here), which responds to the signals on the address bus;
3. If it is a read operation, the data corresponding to the address is transmitted back through the bus according to the bit width specified by the CPU, typically stored in the register specified by the CPU’s memory access instruction;
4. If it is a write operation, the data passed over the bus is written to the specified address according to the bit width specified by the CPU. If it is an I/O device, it generally updates the register corresponding to this address and may produce side effects.

QEMU Simulating Memory and Peripherals

In order to simulate the behavior of memory/peripherals, QEMU must implement at least the following mechanisms:

1. Basic address space management,Be able to distinguish which device it is based on the address delivered by the CPU；
2. Implement discrete mapping of addresses; some peripherals’ addresses are not necessarily contiguous；
3. Implement address remapping, for example, the RAM and XRAM of MCS-51 both start from address 0;

To this end, QEMU provides two concepts:address-space and memory-region(hereinafter referred to as mr). The former is used to describe the mapping relationship of the entire address space (different components may see different address spaces), while the latter is used to describe the mapping rules for a certain address range within the address space.

Address Space Layout

Execution:

$ ./build/qemu-system-riscv64 -M virt -monitor stdio -s -S -display none

Then we enterinfo mtreecommand to see the layout of the address space:

(qemu) info mtree
address-space: cpu-memory-0
address-space: memory
  0000000000000000-ffffffffffffffff (prio 0, i/o): system
    0000000000001000-000000000000ffff (prio 0, rom): riscv_virt_board.mrom
    0000000000100000-0000000000100fff (prio 0, i/o): riscv.sifive.test
    0000000000101000-0000000000101023 (prio 0, i/o): goldfish_rtc
    0000000002000000-0000000002003fff (prio 0, i/o): riscv.aclint.swi
    0000000002004000-000000000200bfff (prio 0, i/o): riscv.aclint.mtimer
    0000000003000000-000000000300ffff (prio 0, i/o): gpex_ioport_window
      0000000003000000-000000000300ffff (prio 0, i/o): gpex_ioport
    0000000004000000-0000000005ffffff (prio 0, i/o): platform bus
    000000000c000000-000000000c5fffff (prio 0, i/o): riscv.sifive.plic
    0000000010000000-0000000010000007 (prio 0, i/o): serial
    0000000010001000-00000000100011ff (prio 0, i/o): virtio-mmio
    0000000010002000-00000000100021ff (prio 0, i/o): virtio-mmio
    0000000010003000-00000000100031ff (prio 0, i/o): virtio-mmio
    0000000010004000-00000000100041ff (prio 0, i/o): virtio-mmio
    0000000010005000-00000000100051ff (prio 0, i/o): virtio-mmio
    0000000010006000-00000000100061ff (prio 0, i/o): virtio-mmio
    0000000010007000-00000000100071ff (prio 0, i/o): virtio-mmio
    0000000010008000-00000000100081ff (prio 0, i/o): virtio-mmio
    0000000010100000-0000000010100007 (prio 0, i/o): fwcfg.data
    0000000010100008-0000000010100009 (prio 0, i/o): fwcfg.ctl
    0000000010100010-0000000010100017 (prio 0, i/o): fwcfg.dma
    0000000020000000-0000000021ffffff (prio 0, romd): virt.flash0
    0000000022000000-0000000023ffffff (prio 0, romd): virt.flash1
    0000000030000000-000000003fffffff (prio 0, i/o): alias pcie-ecam @pcie-mmcfg-mmio 0000000000000000-000000000fffffff
    0000000040000000-000000007fffffff (prio 0, i/o): alias pcie-mmio @gpex_mmio_window 0000000040000000-000000007fffffff
    0000000080000000-0000000087ffffff (prio 0, ram): riscv_virt_board.ram
    0000000400000000-00000007ffffffff (prio 0, i/o): alias pcie-mmio-high @gpex_mmio_window 0000000400000000-00000007ffffffff

address-space: gpex-root
  0000000000000000-ffffffffffffffff (prio 0, i/o): bus master container

address-space: I/O
  0000000000000000-000000000000ffff (prio 0, i/o): io

memory-region: pcie-mmcfg-mmio
  0000000000000000-000000000fffffff (prio 0, i/o): pcie-mmcfg-mmio

memory-region: gpex_mmio_window
  0000000000000000-ffffffffffffffff (prio 0, i/o): gpex_mmio_window
    0000000000000000-ffffffffffffffff (prio 0, i/o): gpex_mmio

memory-region: system
  0000000000000000-ffffffffffffffff (prio 0, i/o): system
    0000000000001000-000000000000ffff (prio 0, rom): riscv_virt_board.mrom
    0000000000100000-0000000000100fff (prio 0, i/o): riscv.sifive.test
    0000000000101000-0000000000101023 (prio 0, i/o): goldfish_rtc
    0000000002000000-0000000002003fff (prio 0, i/o): riscv.aclint.swi
    0000000002004000-000000000200bfff (prio 0, i/o): riscv.aclint.mtimer
    0000000003000000-000000000300ffff (prio 0, i/o): gpex_ioport_window
      0000000003000000-000000000300ffff (prio 0, i/o): gpex_ioport
    0000000004000000-0000000005ffffff (prio 0, i/o): platform bus
    000000000c000000-000000000c5fffff (prio 0, i/o): riscv.sifive.plic
    0000000010000000-0000000010000007 (prio 0, i/o): serial
    0000000010001000-00000000100011ff (prio 0, i/o): virtio-mmio
    0000000010002000-00000000100021ff (prio 0, i/o): virtio-mmio
    0000000010003000-00000000100031ff (prio 0, i/o): virtio-mmio
    0000000010004000-00000000100041ff (prio 0, i/o): virtio-mmio
    0000000010005000-00000000100051ff (prio 0, i/o): virtio-mmio
    0000000010006000-00000000100061ff (prio 0, i/o): virtio-mmio
    0000000010007000-00000000100071ff (prio 0, i/o): virtio-mmio
    0000000010008000-00000000100081ff (prio 0, i/o): virtio-mmio
    0000000010100000-0000000010100007 (prio 0, i/o): fwcfg.data
    0000000010100008-0000000010100009 (prio 0, i/o): fwcfg.ctl
    0000000010100010-0000000010100017 (prio 0, i/o): fwcfg.dma
    0000000020000000-0000000021ffffff (prio 0, romd): virt.flash0
    0000000022000000-0000000023ffffff (prio 0, romd): virt.flash1
    0000000030000000-000000003fffffff (prio 0, i/o): alias pcie-ecam @pcie-mmcfg-mmio 0000000000000000-000000000fffffff
    0000000040000000-000000007fffffff (prio 0, i/o): alias pcie-mmio @gpex_mmio_window 0000000040000000-000000007fffffff
    0000000080000000-0000000087ffffff (prio 0, ram): riscv_virt_board.ram
    0000000400000000-00000007ffffffff (prio 0, i/o): alias pcie-mmio-high @gpex_mmio_window 0000000400000000-00000007ffffffff

It can be seen that:

Address Range	Type	Corresponding Device
0x00001000~0xFFFF	ROM	Onboard Firmware
0x00100000~0x00101023	I/O	Test device + RTC
0x02000000~0x020BFFFF	I/O	CLINT (software/clock interrupts)
0x03000000~0x030FFFFF	I/O	PCI I/O window
0x04000000~0x05FFFFFF	I/O	Platform Bus device
0x0C000000~0x0C5FFFFF	I/O	PLIC
0x10000000~0x100081FF	I/O	UART + Virtio-MMIO
0x20000000~0x23FFFFFF	ROMD	flash0 / flash1
0x80000000~0x87FFFFFF	RAM	Guest memory
0x300000000~0x7FFFFFFFF	I/O	PCI MMIO alias

• A Guest (the simulated object, here referring to the virt machine) can have multiple address-spaces，The address mapping relationships described by each address-space are not necessarily the same, typically I/O and memory。
• Each address-space corresponds to one mr tree, for example, the root node of the mr tree corresponding to address-space: memory is system, and the child nodes are arranged in order of address size.

Since mr describes the mapping rules within a specific address range, thus making it easy to implement discrete mapping of devices.

Example:

address-space: cpu-memory-0
  0000000000000000-ffffffffffffffff (prio 0, i/o): system
    0000000000001000-000000000000ffff (prio 0, rom): riscv_virt_board.mrom
    0000000000100000-0000000000100fff (prio 0, i/o): riscv.sifive.test
    ...

• cpu-memory-0is address-space
• systemis the top-level memory-region (the container of the entire virtual system)
• riscv_virt_board.mrom、sifive.testetc. are child memory-region

Each memory-region is attached under an address-space and provides access handler

memory_region address overlap

mr supports address range overlap among regions at the same level, the overlapping parts are presented according to priority, and the overlapping part with higher priority is the access target. In (prio 0, type), the value following prio is the priority. There is no address overlap among virt’s peripherals, so the priorities are all 0.

Here is an example:

0x8000   0x70000  0x60000  0x50000  0x40000  0x30000  0x20000  0x10000    0
  |--------|--------|--------|--------|--------|--------|--------|--------|
A:[-----------------------------------------------------------------------] prio:0
B:[-----------------------------------------------------] prio:1
C:[-----------------------------------] prio:2
D:[-----------------] prio:3

For mr A, its address range can be viewed as:

0x8000   0x70000  0x60000  0x50000  0x40000  0x30000  0x20000  0x10000    0
  |--------|--------|--------|--------|--------|--------|--------|--------|
A:[DDDDDDDDDDDDDDDDD|CCCCCCCCCCCCCCCCC|BBBBBBBBBBBBBBBBB|AAAAAAAAAAAAAAAAA]

To implement the above mechanism,QEMU uses alias to describe the overlapping parts in mr. Using alias, a part of one mr can be mapped to another mr, thereby simplifying the complexity of memory simulation (comparable to mmap).。

alias example

0000000030000000-000000003fffffff (prio 0, i/o): alias pcie-ecam @pcie-mmcfg-mmio 0000000000000000-000000000fffffff

• alias memory-region maps a region to another address range of the address-space
• To allow different buses to access the same physical device
• An alias is also a memory-region, but it internally references another region.

AddressSpace

• represents The complete address space seen by the CPU or bus.
• Including:
  • All mapped memory-region
  • Priority (prio) of each region
  • Type (RAM / ROM / I/O / alias)
• Can be understood as The VM’s “physical address space view”

A CPU can have multiple address-spaces (for example, a RISC-V CPU hascpu-memory-0, and other I/O spaces or PCI bus spaces).

include/system/memory.h

/**
 * struct AddressSpace: describes a mapping of addresses to #MemoryRegion objects
 */
struct AddressSpace {
    /* private: */
    struct rcu_head rcu;
    char *name;
    MemoryRegion *root;

    /* Accessed via RCU.  */
    struct FlatView *current_map;

    int ioeventfd_nb;
    int ioeventfd_notifiers;
    struct MemoryRegionIoeventfd *ioeventfds;
    QTAILQ_HEAD(, MemoryListener) listeners;
    QTAILQ_ENTRY(AddressSpace) address_spaces_link;

    /*
     * Maximum DMA bounce buffer size used for indirect memory map requests.
     * This limits the total size of bounce buffer allocations made for
     * DMA requests to indirect memory regions within this AddressSpace. DMA
     * requests that exceed the limit (e.g. due to overly large requested size
     * or concurrent DMA requests having claimed too much buffer space) will be
     * rejected and left to the caller to handle.
     */
    size_t max_bounce_buffer_size;
    /* Total size of bounce buffers currently allocated, atomically accessed */
    size_t bounce_buffer_size;
    /* List of callbacks to invoke when buffers free up */
    QemuMutex map_client_list_lock;
    QLIST_HEAD(, AddressSpaceMapClient) map_client_list;
};

MemoryRegion

• Of memory or I/O Specific block
• Including:
  • Start address range (offset relative to address-space)
  • Size
  • Type (RAM / ROM / I/O / alias / container)
  • Child memory-region (supports nesting)
  • Corresponding read/write handler or object pointer

Can be understood as A “single block” in the address-space, it can be physical memory, device registers, PCI BAR, etc.

include/system/memory.h

/** MemoryRegion:
 *
 * A struct representing a memory region.
 */
struct MemoryRegion {
    Object parent_obj;

    /* private: */

    /* The following fields should fit in a cache line */
    bool romd_mode;
    bool ram;
    bool subpage;
    bool readonly; /* For RAM regions */
    bool nonvolatile;
    bool rom_device;
    bool flush_coalesced_mmio;
    bool unmergeable;
    uint8_t dirty_log_mask;
    bool is_iommu;
    RAMBlock *ram_block;
    Object *owner;
    /* owner as TYPE_DEVICE. Used for re-entrancy checks in MR access hotpath */
    DeviceState *dev;

    const MemoryRegionOps *ops;
    void *opaque;
    MemoryRegion *container;
    int mapped_via_alias; /* Mapped via an alias, container might be NULL */
    Int128 size;
    hwaddr addr;
    void (*destructor)(MemoryRegion *mr);
    uint64_t align;
    bool terminates;
    bool ram_device;
    bool enabled;
    uint8_t vga_logging_count;
    MemoryRegion *alias;
    hwaddr alias_offset;
    int32_t priority;
    QTAILQ_HEAD(, MemoryRegion) subregions;
    QTAILQ_ENTRY(MemoryRegion) subregions_link;
    QTAILQ_HEAD(, CoalescedMemoryRange) coalesced;
    const char *name;
    unsigned ioeventfd_nb;
    MemoryRegionIoeventfd *ioeventfds;
    RamDiscardManager *rdm; /* Only for RAM */

    /* For devices designed to perform re-entrant IO into their own IO MRs */
    bool disable_reentrancy_guard;
};

Initialization process

We use the QEMU initialization process to understand the relationship between mr and address-space:

main() // system/main.c
|--qemu_init(argc, argv) // system/vlc.c
|  |--cpu_exec_init_all() // system/physmem.c
|  |  |--io_mem_init()
|  |  |  |--memory_region_init_io(&io_mem_unassigned, NULL, &unassigned_mem_ops, NULL, NULL, UINT64_MAX)
|  |  |--memory_map_init()
|  |  |  |--memory_region_init(system_memory, NULL, "system", UINT64_MAX)
|  |  |  |--address_space_init(&address_space_memory, system_memory, "memory")
|  |  |  |--memory_region_init_io(system_io, NULL, &unassigned_io_ops, NULL, "io", 65536)
|  |  |  |--address_space_init(&address_space_io, system_io, "I/O")

memory_map_init

system/physmem.c

static void memory_map_init(void)
{
    system_memory = g_malloc(sizeof(*system_memory));

    memory_region_init(system_memory, NULL, "system", UINT64_MAX);
    address_space_init(&address_space_memory, system_memory, "memory");

    system_io = g_malloc(sizeof(*system_io));
    memory_region_init_io(system_io, NULL, &unassigned_io_ops, NULL, "io",
                          65536);
    address_space_init(&address_space_io, system_io, "I/O");
}

Formemory_region_init(), finally callingmemory_region_do_init()：

system/memory.c

void memory_region_init(MemoryRegion *mr,
                        Object *owner,
                        const char *name,
                        uint64_t size)
{
    object_initialize(mr, sizeof(*mr), TYPE_MEMORY_REGION);
    memory_region_do_init(mr, owner, name, size);
}

static void memory_region_do_init(MemoryRegion *mr,
                                  Object *owner,
                                  const char *name,
                                  uint64_t size)
{
    mr->size = int128_make64(size);
    if (size == UINT64_MAX) {
        mr->size = int128_2_64();
    }
    mr->name = g_strdup(name);
    mr->owner = owner;
    mr->dev = (DeviceState *) object_dynamic_cast(mr->owner, TYPE_DEVICE);
    mr->ram_block = NULL;

    if (name) {
        char *escaped_name = memory_region_escape_name(name);
        char *name_array = g_strdup_printf("%s[*]", escaped_name);

        if (!owner) {
            owner = machine_get_container("unattached");
        }

        object_property_add_child(owner, name_array, OBJECT(mr));
        object_unref(OBJECT(mr));
        g_free(name_array);
        g_free(escaped_name);
    }
}

In this code, initialization of some key fields of mr is completed.

/** MemoryRegion:
 *
 * A struct representing a memory region.
 */
struct MemoryRegion {
    Object parent_obj;

    /* private: */
    Object *owner;
    const MemoryRegionOps *ops;
    Int128 size;
    QTAILQ_HEAD(, MemoryRegion) subregions;
    QTAILQ_ENTRY(MemoryRegion) subregions_link;
    ...
};

• ops points to the actual interface for mr memory access;
• subregions point to other mrs, through subregions, all associated mrs can be linked together.

Some of this initialization code consists of registered function callbacks. It is not convenient to understand the logic by static code review; you can use gdb to operate.

system_memory is a global variable pointer，pointing to the root node of mr, we can monitor system_memory->ops and system_memory->subregions to see in which function they are initialized.

First observe system_memory->ops, the commands and process are as follows:

watch system_memory->ops

The first and second hits of the watchpoint are reset operations on ops; the third hit is where the real initialization occurs. We can observe the call stack:

ops initialization

It can be seen thatmemory_region_initfn()is inobject_init_with_type()is called in, this is QEMU’s QOM module,can be simply understood as the initialization of the mr object., thisThe initialization method is a registered function pointer.。

By analogy, we can find out where system_memory->subregions is initialized:

system_memory->subregions

Node relationship

Watch furthersystem_memory->subregions

How other subregions are added

memory_region_update_container_subregions()The process is very simple, and the final execution result is as follows:

                      struct MemoryRegion
                    +------------------------+
                    |subregions              |
                    |    QTAILQ_HEAD()       |
                    +------------------------+
                                |
            +-------------------+---------------------+
            |                                         |
            |                                         |
    struct MemoryRegion                      struct MemoryRegion
+------------------------+                +------------------------+
|subregions              |                |subregions              |
|    QTAILQ_HEAD()       |                |    QTAILQ_HEAD()       |
+------------------------+                +------------------------+
      ...                                            ...

QEMU’s memory-region is organized as a tree structure, and address lookup usesRed-black tree(Interval Tree) to efficiently manage address ranges.

There is a root field in address-space, pointing to the root node of memory-region, thus realizing that one address-space corresponds to one memory-region tree., as follows:

     AddressSpace
+-------------------------+
|name                     |
|   (char *)              |
|                         |     MemoryRegion(system_memory/system_io)
+-------------------------+          +------------------------+
|root                     |          |subregions              |
|   (MemoryRegion *)      | -------->|    QTAILQ_HEAD()       |
+-------------------------+          +------------------------+
                                                  |
                                                  |
                              +-------------------+---------------------+
                              |                                         |
                   struct MemoryRegion                         struct MemoryRegion
                   +------------------------+                  +------------------------+
                   |subregions              |                  |subregions              |
                   |    QTAILQ_HEAD()       |                  |    QTAILQ_HEAD()       |
                   +------------------------+                  +------------------------+

Each mr corresponds to a specific memory block RAMBlock.，This memory block is allocated from the Host, as storage for Guest peripheral devices.

mr provides several types for describing storage devices; common ones include RAM, ROM, IOMMU, and container.

Returning to the QEMU interactive terminal, using the following command, we can print the memory-region distribution of virt and its corresponding peripherals:

qom-tree

memory-region (mr) They all represent an accessible address block, such as:

• RAM/ROM → storage content
• I/O → device registers
• alias → address mapping
• container → “container”, manages sub-regions

Each mr has starting offset(relative to the parent region) and Size。

For memory-region container type, which contains other mrs and records the offset of each mr.

container is a special type of memory-region

It itself does not store data, nor does it have a direct read/write handler

Purpose:

1. manage sub memory-regions
2. provide address offset mapping
3. establish a hierarchical structure

In practical application scenarios, wecan use mr container to create different address hierarchy relationships, which can clearly describe the relationships of different subsystems at the address space level, and is very helpful for achieving modularity.

Reference:

Learning Qemu Docs

https://gevico.github.io/learning-qemu-docs/