Timeline
Timeline
2025-10-24
init
This article introduces the basic concepts and paging mechanism of ARM memory management, discusses the evolution from single-level page tables to multi-level page tables, and the address translation principles of the MMU hardware unit under the VMSA architecture. It summarizes the page table format, virtual address space partitioning, and the types and memory attributes of four-level page table descriptors under the ARMv8 architecture.
Reference documents:
Basic knowledge and concepts of memory management
Disadvantages of directly using physical memory
- Process address space protection issue. All user processes can access the entire physical memory, so malicious programs can modify the memory data of other programs.
- Low memory usage efficiency. If the memory space required by a process to be run is insufficient, a process must be selected for complete swapping out. This mechanism results in a large amount of data needing to be swapped in and out, making it very inefficient.
- Program runtime address relocation issue

Basic concepts of paging mechanism
- Virtual Memory
- Virtual Address Space
- Physical Memory
- Page Frame
- Virtual Page Frame Number
- Physical Frame Number
- Page Table (PT)
- Page Table Entry (PTE)
Process of mapping virtual addresses to physical addresses

Single-level page table

Disadvantages of using a single-level page table
- The processor uses a single-level page table. The virtual address space is 32 bits wide, with an addressing range of 4GB. The physical address space is also 32 bits, supporting up to 4GB of physical memory, and the page size is 4KB. To map the entire 4GB address space, 4GB/4KB = 1 million page table entries are needed. Each page table entry occupies 4 bytes, requiring a total of 4MB of physical memory to store this page table.
- **Each process has its own set of page tables, and the page table base address needs to be switched during process context switching.**As with the single-level page table mentioned above, each process needs to be allocated 4MB of contiguous physical memory to store the page table, which is unacceptable because it wastes too much memory.
- Multi-level page table: maps level by level on demand, without mapping the entire address space at once.
Two-level page table

VMSA
Virtual Memory System Architecture
- VMSA provides the MMU hardware unit
- Translation from virtual address to physical address
- Access permissions
- Memory attribute checking
- The MMU hardware unit is used to implement the translation from VA to PA
- Hardware page table walking
- The TTBR register holds the base address of the page table
- The TLB stores recent translation page table entries

Without virtualization, translation has only one stage, mapping VA to PA
With virtualization, translation first needs to convert to IPA

TTBR0_ELxUsed for each process’s address space
**TTBR1_ELx **Used for kernel space, shared by all processes

ARMv8 page tables
AArch64 only supports the Long Descriptor page table format
AArch32 supports two page table formats
- Armv7-A Short Descriptor format
- Armv7-A (LPAE) Long Descriptor format
AArch64 supports three different page sizes: 4KB, 16KB, and 64KB.
A larger page size can reduce the size of the page table.
The address bus width supports 48 bits or 52 bits.
52-bit width: ARMv8.2-LVA is implemented and the 64 KB translation granule.
Take a 48-bit address bus width as an example.
The virtual address (VA) is divided into two spaces.,Each space supports a maximum of 256 TB.
- The lower virtual address space ranges from 0x0000_0000_0000_0000 to 0x0000_FFFF_FFFF_FFFF.
- The upper virtual address space ranges from 0xFFFF_0000_0000_0000 to 0xFFFF_FFFF_FFFF_FFFF.

Virtual address space division. (Fault is a non-canonical region that the CPU cannot access.)
Four-level page table.

4-level page table.
Page table descriptors in AArch64.
The following are all 48-bit virtual addresses with 4KB page size.
Page table descriptor for L0~L2

Block typeIndicates that a very large block of memory is described

L3 page table descriptor





| Level | Corresponding Linux abstraction | Possible types of descriptors |
|---|---|---|
| L0 (optional) | PGD | Table / Fault |
| L1 | PUD | Block (1GB, granule=4K) / Table / Fault |
| L2 | PMD | Block (2MB, granule=4K) / Table / Fault |
| L3 | PTE | Page (4KB, granule=4K) / Fault |
Note:
- Block entryCan only appear in intermediate levels (L1/L2), indicating a huge page mapping that maps a large physical address space, equivalent to the final level page table.
- **PTE (L3)**Cannot be Block, only Page or Fault.
- The output address is that of the next-level page tablePAi.e., the physical address
Page table attributes
Only entries pointing to**the final physical page (block/page)**require page table attributes
Armv8.6 D5.3.3Chapter
- **bit[0]**→ Valid or not:
0= Invalid (Fault entry)1= Valid entry
- **bit[1]**→ Type (meaningful only when valid):
0= Block entry (block mapping, large page mapping)1= Table entry (points to next-level page table; becomes Page entry at the last level)
Thus: page table attributes for block and page

High-order attributes and low-order attributes
only describeBlockandPage




Share Domain
Non-shareable
This represents memory accessible only by a single processor or other agent, so memory accesses never need to be synchronized with other processors. This domain is not typically used in SMP systems.
Inner shareable
This represents a shareability domain that can be shared by multiple processors, but not necessarily all of the agents in the system. A system might have multiple Inner Shareable domains. An operation that affects one Inner Shareable domain does not affect other Inner Shareable domains in the system. An example of such a domain might be a quad-core Cortex-A57 cluster.
Outer shareable
An outer shareable (OSH) domain re-order is shared by multiple agents and can consist of one or more inner shareable domains. An operation that affects an outer shareable domain also implicitly affects all inner shareable domains inside it.
However, it does not otherwise behave as an inner shareable operation.
Full system
An operation on the full system (SY) affects all observers in the system.
Contiguous Block entries
- ARMv8an optimization using the TLB: usingone TLB entry to complete the VA-to-PA translation of multiple consecutive pages
- conditions for using the Contiguous bit
- the VAs corresponding to the pages must be contiguous
- for 4KB pages, 16 consecutive pages
- for 16KB pages, 32 or 128 consecutive pages
- for 64KB pages, 32 consecutive pages
- consecutive pages must have the same attributes
- the starting address must be page-aligned
4KB page table
- 4-level page table
- 48-bit virtual address
- Each level of page table uses 9 bits for indexing (512 entries)

16KB page table
4-level page table
48-bit virtual address
L0 page table has only two entries
L1, L2, L3 page tables use 11 bits for indexing (2048 entries)

64KB page table
- 3-level page table
- 48-bit virtual address
- L1 page table has only 64 entries
- L2 and L3 page tables use 13 bits

Separate two-page table design
- User space (EL0) and kernel space (EL1) adopt two separate page table base address designs
- When the high 16 bits of the virtual address are 1, TTBR1_EL1 is selected
- When the high 16 bits of the virtual address are 0, TTBR0_EL0 is selected

Example of address lookup

In a simple address translation, only one level of lookup is involved. Assume we are using a 64KB granule with a 42-bit virtual
address. The MMU maps the virtual address as follows:
- If VA[63:42] = 1, then TTBR1 is used for the base address of the first page table. When VA[63:42] = 0, TTBR0 is used for the first page table’s
base address. - The page table contains 8192 64-bit page table entries and is indexed using VA[41:29]. The MMU reads the relevant level 2 table entry from the table.
- The MMU checks the validity of the page table entry and whether the requested memory access is allowed. Assuming it is valid, the memory access is permitted.
- In Figure 12-7, the page table entry refers to a 512MB page (it is a block descriptor).
- Bits[47:29] are taken from this page table entry and form Bits[47:29] of the physical address.
- Since we have a 512MB page, Bits[28:0] of the VA are taken as PA[28:0]. See page 12-15 for the effect of granule size on the translation table
effect. - Return the complete PA[47:0], along with other information from the page table entry.

Assumes a 64KB granule and a 42-bit virtual address space.
- If VA[63:42] = 1, TTBR1 is used for the base address of the first-level page table. When VA[63:42] = 0, TTBR0 is used for the base address of the first-level page table.
base address. - The page table contains 8192 64-bit page table entries and is indexed by VA[41:29]. The MMU reads the relevant second-level table
entry from the table. - The MMU checks the validity of the second-level page table entry and whether the requested memory access is allowed. Assuming it is valid, the memory access is permitted.
- In Figure 12-8, the second-level page table entry points to the address of the third-level page table (it is a table descriptor).
- Bits[47:16] are taken from the second-level page table entry to form the base address of the third-level page table.
- Bits[28:16] of the VA are used to index the third-level page table entry. The MMU reads the relevant third-level table entry from the table.
- The MMU checks the validity of the level 3 page table entry and whether the requested memory access is allowed. If it is valid, the memory access is permitted.
- In Figure 12-8, the level 3 page table entry refers to a 64KB page (it is a page descriptor).
- Bits[47:16] are taken from the level 3 page table entry to form PA[47:16].
- Since we have a 64KB page, VA[15:0] is taken as PA[15:0].
- The complete PA[47:0] is returned, along with other information from the page table entry.
System registers related to page tables
TCR_EL1
Translation Control Register
Configure address space size and page table granularity

Configure address space size and page table granularity

- IPS: Intermediate Physical Address Size, used to configure the physical address size, for example, 48 bits, a physical space of 256 TB
- TG1andTG0: Configure the page table granularity size, for example, 4KB, 16KB, 64KB
- T1SZ: Used to configure the size that the TTBR_EL1 page table can manage, calculated as 2^(64-T1SZ) bytes
- T0SZ: Used to configure the size that the TTBR_EL0 page table can manage, calculated as 2^(64-T0SZ) bytes
In ARM64, the high bits of a virtual address are not used arbitrarily, but are restricted byTCR_EL1.T0SZ / T1SZrestrictions.
Cache-related fields

- SH1: Sets memory-related cache attributes for memory accessed via the TTBR_EL1 page table. For example, Non-shareable, Outer Shareable, Inner Shareable
- SH0: Sets memory-related cache attributes for memory accessed via the TTBR_EL0 page table


- ORGN1: Sets attributes related to Outer Shareable
- ORGN0: Sets attributes related to Outer Shareable
- IRGN1: Sets attributes related to Inner Shareable
- IRGN0: Sets attributes related to Inner Shareable

TCR_The fields SH0/SH1, IRGN0/IRGN1, ORGN0/ORGN1 in EL1 are only used for memory regions that are not undergoing address translation (i.e., when the MMU is disabled), or as “default” attributes in certain special contexts. Once the MMU is enabled, memory attributes are entirely determined by page table entries + MAIR_EL1.
SCTLR_EL1
System Control Register (EL1)

- M: Enables and disables the MMU
- I: Enables the instruction cache
- C: Open data cache
TTBR0_EL1
Points to the base address of the TTBR0 page table, typically used for EL1/EL0 page table mappings

TTBR1_EL1
Points to the base address of the TTBR1 page table, typically used for EL1/EL0 page table mappings

MAIR_EL1
Memory Attribute Indirection Register







ARM architecture designers believe:In the vast majority of systems, 8 memory types are sufficient to cover all usage scenarios
1 | // Typical MAIR configuration in the Linux kernel |
ID_AA64MMFR0_EL1
AArch64 Memory Model Feature Register 0, reports the processor’s support forpage tables, address ranges, and memory feature support


| Field | Bits | Meaning |
|---|---|---|
| PARANGE | [3:0] | Supported physical address width |
| ASID | [7:4] | Supported ASID (Address Space ID) bits |
| BIGENDEL | [11:8] | Support for EL1/EL0 large page extension |
| SNSMEM | [15:12] | Supports secure memory access |
| BIGENDEL0 | [19:16] | Supports EL0 large page extension |
| TGRAN16 | [23:20] | Supports 16KB pages |
| TGRAN64 | [27:24] | Supports 64KB pages |
| TGRAN4 | [31:28] | Supports 4KB pages |
CPACR_EL1
Architectural Feature Access Control Register


FPEN field (CPACR_EL1[21:20])
FPEN = Floating-point Enable controls, controlsWhether EL0/EL1 access to SVE, Advanced SIMD, and floating-point registers is trapped by EL1/EL2。
- Register impact:
- AArch64:
- FPCR、FPSR
- SIMD/Floating-point registers
V0-V31(including D0-D31 / S0-S31 views)
- AArch32 / Advanced SIMD:
- FPSCR
- Q0-Q15 (including D0-D31 / S0-S31 views)
- AArch64:
- Exception reporting:
- EL0/EL1 trap → EC syndrome =
0x07 - EL2 trap → EC syndrome =
0x00(When EL2 is enabled andHCR_EL2.TGE = 1)
- EL0/EL1 trap → EC syndrome =
| FPEN | Behavior Description |
|---|---|
| 0b00 | Both EL0 and EL1 instructions are trapped, unless CPACR_EL1.ZEN already traps them |
| 0b01 | Only EL0 instructions are trapped, EL1 is not trapped |
| 0b10 | Both EL0 and EL1 instructions are trapped(Same as 0b00) |
| 0b11 | No instructions are trapped(Registers are freely accessible) |
In simple terms:
FPEN controls whetheruser mode (EL0) or kernel mode (EL1) can directly use floating-point/SIMD/SVE registers。It is usually enabled before turning on the MMU
Together with CPACR_EL1.ZEN, fine-grained control over access at different levels can be achieved.
Common configurations:
0b11→ No trap, allow all EL0/EL1 instructions to access FP/SIMD.
0b00/0b10→ Trap, typically used in security or virtualization scenarios.

MDSCR_EL1


TDCC
TDCC = 0→ User-mode EL0 can directly read/write the DCC register.
TDCC = 1→ EL0 access to the DCC register will trap to EL1/EL2, commonly used insecurity/virtualization/debug control。
DCC is theDebug Communication Channel, which providesa data transfer interface between the CPU and the debug host, and must be enabled to use JTAG

Enable MMU
1 | // enable_mmu.S |
TBI(Top Byte Ignore)
The TBI (Top Byte Ignore) feature in the ARMv8-A architecture, also calledTop Byte Ignore, it allows you touse the top 8 bits ([63:56]) of a virtual address to store additional information without affecting memory access. This mechanism is also calledTagged Pointers, a feature provided by ARM to programming languages and runtime systems.
Under normal circumstances, the address must be valid.
In a 64-bit system, registers are 64 bits, but virtual addresses are not full 64 bits—ARMv8 typically supports48-bit or 52-bit virtual addresses. So the virtual addressthe top 16 bits must be sign-extended:
| virtual address type | the high 16 bits must be |
|---|---|
| user-space memory | 0x0000 |
| kernel high address | 0xFFFF |
If you write[63:48]as any other value, the CPU will trigger anaddress size fault。
After enablingTBI in TCR_EL1(there are respectivelyTBI0andTBI1control), the CPU will ignore the highest 8 bits of the address[63:56], meaning you can freely write tag data into these 8 bits tostore additional information without affecting memory access!
Note: Kernel/user addresses are still determined by VA[55], where 1 indicates a kernel address and 0 indicates a user address space.
InTCR_EL1there are two fields:
TBI0→ Controlswhether EL0 enables pointer tagging when accessing addresses via TTBR0Whether to enable pointer labelTBI1→ Controlswhether EL1 enables pointer tagging when accessing addresses via TTBR1Whether to enable pointer label
Memory Attributes
Memory attributes defined by ARMv8
The ARMv8 architecture processor provides two memory attributes
- Normal Memory
Normal memory is weakly ordered, with no additional constraints, providing the highest memory access performance
- Device Memory
Processor access to device memory has many restrictions, such as no speculative access. Device memory is executed strictly in instruction order. The ARMv8 architecture defines multiple device memory attributes
- **Device-nGnRnE**(Does not support gathering, does not support reordering, does not support early write acknowledgment)
- **Device-nGnRE**(Does not support gathering, does not support reordering, supports early write acknowledgment)
- **Device-nGRE**(Does not support gathering, supports reordering, supports early write acknowledgment)
- **Device-GRE**(Supports gathering, supports reordering, supports early write acknowledgment)
- Gathering or non Gathering (G or nG)
This attribute determines whethermultiple accesses can be merged into a single bus transaction to this memory region。
- If the address is marked as non-gathering (nG), thenthe number and size of accesses to that location on the memory bus must exactly match the number and size of explicit accesses in the code。
- If the address is marked as Gathering (G), the processor canfor example, merge two byte writes into a single halfword write. For regions marked as Gathering, it is also possible tomerge multiple memory accesses to the same memory location。
For example, if the program reads the same location twice, the core only needs to perform one read and can return the same result for both instructions. For reads from regions marked as non-Gathering, the data value must come from the terminal device. It cannot be snooped from the write buffer or other locations.
- Re-ordering (R or nR)
This determineswhether accesses to the same device can be reordered with respect to each other。If the address is marked as non-reordering (nR), accesses within the same block always appear on the bus in program order. The size of this block is IMPLEMENTATION DEFINED. If the block size is large, it can span several table entries. In this case, ordering rules are observed for any other accesses also marked as nR.
- Early Write Acknowledgement (E or nE)
This determineswhether intermediate write buffers between the processor and the slave device being accessed are allowed to send write completion acknowledgments. In modern SoCs (System-on-Chips), processors (such as CPUs) access peripherals or memory through an interconnect. To improve performance, the interconnect typically includeswrite buffers, used to temporarily store data that has not yet been actually written to the target device.
However, in some scenarios,it must be ensured that the data has actually reached the target device before the write operation is considered complete(for example, writing to a register to trigger a hardware action), while in other scenarios it is acceptableAcknowledge write completion earlyto improve throughput (e.g., writing to normal memory).
This leads to the distinction between**E (Early) and nE (not Early)**the difference.
If the address is marked as non-early write acknowledgment (nE), the write response must come from the peripheral, i.e.,prohibitearly acknowledgment. The write completion signalmust be issued by the target peripheral itself, meaning the dataafter it is actually received by the peripheral, only then notify the processor that the write operation is complete.
If the address is marked for early write acknowledgment (E), thenthe buffer in the interconnect logic is allowed to issue a write acceptance signal before the end device actually receives the write. This is essentially information to the external memory system allowing the write buffer in the interconnect toWhen the data has not actually reached the target peripheral, a confirmation signal of ‘write complete’ is sent to the processor.
Defined in the Linux kernel

Memory attributes are not stored in the page table entries, but are stored inMAIR_ELnthe register (Memory Attribute Indirection Register)。
There is a 3-bit index value (AttrInx[2:0]) in the page table entry to look up the MAIR_ELn register。

Memory attributes defined in the Linux kernel
- The operating system (Linux system) defines a series of attributes based on the memory attributes defined by ARMv8, as well as memory read/write and other properties
- PAGE_KERNEL: The most common memory pages
- PAGE_KERNEL_RO: Read-only normal memory pages in the kernel
- PAGE_KERNEL_ROX: Read-only executable normal pages in the kernel
- PAGE_KERNEL_EXEC: Executable normal pages in the kernel
- PAGE_KERNEL_EXEC_CONT: Executable normal pages in the kernel, and are multiple physically contiguous pages

Example of creating page tables in Linux 5.0
- Global directory entryPGD (Page Global Directory) corresponds to the L0 page table of arm64
- Upper directory entryPUD (Page Upper Directory) corresponds to the L1 page table of arm64
- Middle directory entryPMD (Page Middle Directory) corresponds to the L2 page table of arm64
- Page table entry (Page Table Entry) corresponds to the L3 page table of arm64
arch/arm64/mm/mmu.c






- Look up the PGD entry by address addr

- Find the end address of the range managed by the corresponding pgd via addr

- Set the pgd page table entry

- Look up the PUD entry by address addr

Experiment
Experiment 1: Establish identity mapping


ARM64 uses4-level page table(PGD → PUD → PMD → PTE), each level uses9-bit index(512 entries), page size is4KB:
| Level | Name | Index bits | Mapping range | Number of entries | Table size |
|---|---|---|---|---|---|
| L0 | PGD | [47:39] | 512 GB | 512 | 4KB |
| L1 | PUD | [38:30] | 1 GB | 512 | 4KB |
| L2 | PMD | [29:21] | 2 MB | 512 | 4KB |
| L3 | PTE | [20:12] | 4 KB | 512 | 4KB |
Note: If a page table entry at a certain level is marked asSECTION (block mapping), then skip the next level and directly map a large block of memory.













Why identity mapping
To avoid issues when enabling the MMU:
- When the MMU is disabled, all addresses accessed by the processor are physical addresses.When the MMU is enabled, the addresses accessed by the processor become virtual addresses.
- Modern processors all usemulti-stage pipeline architecture,The processor prefetches multiple instructions into the pipeline in advance.. When enabling the MMU, the processor has already prefetched multiple instructions, and these instructions are prefetched using physical addresses. After the instruction to enable the MMU is executed, the MMU functionality of the processor takes effect. Therefore, this is toensure that the processor can fetch instructions continuously before and after enabling the MMU.。
Experiment 2: Why the MMU fails to run

Obviously,_text_boot to _the memory attribute of etext should be mapped as PAGE_KERNEL_ROX
from_etext to TOTAL_MEMORY is mapped as PAGE_KERNEL
Experiment 3: Implement an MMU page table dump function

Experiment 4: System crash caused by modifying page attributes

Xiao Ming conducted this experiment. He allocated a 4KB read-only page in the text section of the linker script, then implemented a walk_pgtable() function to traverse the page table and find the corresponding PTE, but found that no matter how he set it, he could not make the page writable.



The problem lies here

should be changed to

Experiment 5: Verify ldxr and stxr instructions


The reason is that we did not set the shareable attribute



Experiment 6: Address translation AT instruction



Reading the MMU chip manual
TODO
