ARM Interrupt Controller

This article introduces the interrupt mechanism in ARM64 exception handling, including the PSTATE status bits, the differences between the GIC controller and traditional interrupt methods, and discusses in detail the interrupt source classification, routing mechanism, and reading process of the interrupt status register for Legacy Interrupts on the Raspberry Pi 4B. Finally, using the generic timer as an example, this article summarizes the complete flow of EL1 interrupt handling, the interrupt context save and restore mechanism, and the recommended initialization sequence in a bare-metal environment.

Reference documents:

Arm Architecture Reference Manual Armv8

https://developer.arm.com/documentation/ddi0487/fc/

ARM64 Exception Handling: Interrupt Processing

• ARMTwo core interrupt-related pins:nIRQandnFIQ
• EachCPUcore has a pair of such interrupt-related pins

• There are two bits in the PSTATE status related to interrupts
  • I is used to mask IRQ interrupts
  • F is used to mask FIQ interrupts

ARM64inGICcontroller

• ARM provides a standard GIC controller, for example, the Raspberry Pi 4B supports GIC-400
• The Raspberry Pi 3B supports the traditional interrupt method (legacy interrupt)

Legacy Interrupt

Interrupt handling process

When the processor takes an exception to the AArch64 execution state, all PSTATE interrupt masks are automatically set. This means subsequent exceptions are disabled. If software wants to support exception nesting, for example, allowing a high-priority interrupt to preempt the handling of a low-priority source, then software must explicitly re-enable interrupts.

For the following instruction:

1

MSR DAIFClr, #imm

The nested handler requires some additional code. It must save SPSR_EL1 and ELR_contents of EL1. After identifying (and clearing) the interrupt source, we must also re-enable IRQ.

Interrupt sources on the Raspberry Pi 4B

• ARM Core n(ARM core interrupt sources)

  • PNS timer IRQCorresponds toA Non-secure EL1 physical timer
  • PS timer IRQCorresponds toA secure EL1 physical timer
  • HP timer IRQ(Hypervisor) corresponds toA Non-secure EL2 physical timer
  • V timer IRQcorresponds toA virtual timer
  • PMUYesPerformance Monitor Unit
  • repeated 4 times indicates that each core has a set of such interrupt sources; the Raspberry Pi has 4 cores, hence repeated 4 times

  

• ARM_LOCAL(Interrupt source accessible only by CPU)

• ARMC(Interrupt source accessible by both CPU and GPU)

• VideoCore(GPU core interrupt source)

The included interrupts are as follows:

VC(VideoCore) peripheral IRQs

• ETH_PCIe(PCIe interrupt)

Raspberry Pi 4B’sLegacy Interrupt Routing

ARM Core IRQs are directly routed to pre-core routing

ARMC and VC are routed through the ARMC routing hardware unit

Legacy IRQ status registers

• FIQn/IRQn_PENDING2

• FIQn/IRQn_PENDING0

• FIQn/IRQn_PENDING1

• FIQ/IRQ_SOURCEn

When bit8 of the source register is set, you need to read the PENDING2 status register

If bit24 of PENDING2 is set, you need to read the PENDING0 status register

If bit25 of PENDING2 is set, you need to read the PENDING1 register

The software needs to read the interrupt status register step by step

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

       ┌──────────────┐
       │ SOURCE寄存器 │   ← ARM_LOCAL_IRQ_SOURCE0
       └───────┬──────┘
               │
    ┌──────────┴──────────┐
    │                     │
 本地中断？             bit8=1？
(定时器等)             (有外设/GPU中断)
    │                     │
    ↓                     ↓
handle_timer_irq()   ┌───────────────┐
                     │  PENDING2寄存器│
                     └───────┬───────┘
                             │
          ┌───────────┬───────────┐
          │           │           │
     bit24=1？    bit25=1？     其它位？
     去PENDING0   去PENDING1   直接是GPU中断

Example

Taking the ARM Core’s generic timer as an example

• Cortex-A72 supports 4 ARM Core generic timers
  • CNT_PS_IRQ Secure EL1 Physical Timer Event Interrupt
  • CNT_PNS_IRQ Nonsecure EL1 Physical Timer Event interrupt
  • CNT_HP_IRQ Hypervisor Physical Timer Event interrupt, EL2
  • CNT_V_IRQ Virtual Timer Event interrupt EL3

The timer supports two trigger modes

XXX_ELn→ indicates this system registerAccessible at ELn and higher privilege levels。

• For exampleCNTP_CTL_EL0isBoth EL0 and EL1 can access。

Interrupt handling flow of the EL1 Non-secure generic timer

1. Initialize the timer, setcntp_ctl_el0register’senable field is 1
2. for the timerTimeValuean initial value, setcntp_tval_el0register
3. enable the timer-related interrupt in the Raspberry Pi interrupt controller, setTIMER_CNTRL0in the registerCNT_PNS_IRQto 1

4. enable the IRQ interrupt master switch in the PSTATE register

5. Timer interrupt occurs

6. CPU jumps to the el1_irq assembly function

7. save interrupt context(usingkernel_entrymacro)

8. jump to the interrupt handler function

9. read ARM_LOCAL interrupt status register IRQ_SOURCE0

10. Determine whetherCNT_PNS_IRQan interrupt occurred

11. If so, reset TimeValue

12. Return to the el1_irq assembly function

13. Restore interrupt context

14. Return to interrupt scene

Interrupt scene

• At the moment an interrupt occurs, the CPU state includes:
  • PSTATE register
  • PC value
  • SP value
  • x0~x30 registers
• Use a stack frame data structure to describe the interrupt scene to be saved (struct pt_regs)

Save interrupt context

Restore interrupt context

Interrupt Experiment 1: Implementing generic timer on Raspberry Pi

Raspberry Pi firmware loads GIC controller by default instead of using Legacy Interrupt, so it can run on QEMU

1. Initialize timer, setcntp_ctl_el0register’senable field to 1

   

2. Assign to timer’sTimeValuean initial value, setcntp_tval_el0register

   

3. Enable the timer-related interrupts in the Raspberry Pi interrupt controller, setTIMER_CNTRL0the register’sCNT_PNS_IRQto 1

   

   

   

4. Enable the IRQ interrupt master switch in the PSTATE register

   

5. Timer interrupt occurs

6. CPU jumps to the el1_irq assembly function

   

   

7. Save interrupt context(usingkernel_entryMacro)

   

8. Jump to interrupt handler

9. Read ARM_Interrupt status register IRQ in LOCAL_SOURCE0

   

   

10. Determine whetherCNT_PNS_IRQInterrupt occurred

11. If so, reset TimeValue

12. Return to el1_irq assembly function
13. Restore interrupt context
14. Return to interrupt scene

Another implementation:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110

#include "timer.h"
#include "asm/arm_local_reg.h"
#include "io.h"
#include "irq.h"
#include "mydef.h"
#include "printk.h"

static inline unsigned long read_cntfrq(void) {
  unsigned long v;
  asm volatile("mrs %0, cntfrq_el0" : "=r"(v));
  return v;
}
// ms -> ticks
static inline unsigned long ms_to_ticks(unsigned int ms) {
  unsigned long f = read_cntfrq();
  return (f / 1000UL) * (unsigned long)ms;
}

// Set cntp_ctl_Set el0 enable field to 1 to enable EL1 Nonsecure generic timer
static int generic_timer_init(void) {
  asm volatile("mov x0, #1\n"
               "msr cntp_ctl_el0, x0"
               :
               :
               : "memory");
  return 0;
}
// Set cntp_ctl_Set el0 enable field to 0 to disable EL1 Nonsecure generic timer
static int generic_timer_deinit(void) {
  asm volatile("mov x0, #0\n"
               "msr cntp_ctl_el0, x0"
               :
               :
               : "memory");
  return 0;
}
// %[name] → Reference constraint variable
// %w[name] → Reference lower 32-bit register (e.g., w0, w1)
// %x[name] → Reference full 64-bit register (e.g., x0, x1)

static int generic_timer_reset(unsigned int val) {
  asm volatile("msr cntp_tval_el0, %x[timer_val]"
               :
               : [timer_val] "r"(val)
               : "memory");
  return 0;
}

static void enable_timer_interrupt(void) { writel(CNT_PNS_IRQ, TIMER_CNTRL0); }

static void (*timer_callback)(void) = NULL;
void timer_init(unsigned int ms, void (*callback)(void)) {

  // Initialize timer, cntp_ctl_el0 enable is 1
  unsigned int ticks = ms_to_ticks(ms); // ms -> tick
  generic_timer_reset(ticks);
  // Enable interrupt controller and timer-related interrupts
  enable_timer_interrupt();
  timer_callback = callback;
}

void timer_start(void) {
  raw_local_irq_disable(); // Disable PSTATE interrupt controller master switch to prevent interruption during initialization
  generic_timer_init();    // Enable cntp_ctl_Set el0.enable=1 to start counting
  raw_local_irq_enable();  // Enable PSTATE interrupt controller master switch to prevent interruption during initialization
}

void timer_stop(void) {
  raw_local_irq_disable(); // Disable PSTATE interrupt controller master switch to prevent interruption during initialization
  generic_timer_deinit();
  raw_local_irq_enable(); // Turn on the PSTATE interrupt controller master switch to prevent interruptions during initialization
}

void timer_reset(unsigned int ms) {
  unsigned int ticks = ms_to_ticks(ms); // ms -> tick
  generic_timer_reset(ticks);
}

void handle_timer_irq(void) {
  generic_timer_deinit();
  if (timer_callback) {
    timer_callback();
    timer_callback = NULL;
  }
  printk("Core0 Timer interrupt received\r\n");
}

void kernel_main(void) {
  uart_init();
  init_printk_done();
  printk("Welcome to arm64 mini OS!\r\n");

  raw_local_irq_enable(); // Turn on the PSTATE interrupt controller master switch
  printk("timer test\n");
  timer_init(2000, test_function);
  timer_start();
  asm volatile("wfi");
  /* my test*/
  my_test();
  // data_abort does not work in QEMU
  // trigger_sync_data_abort();

  trigger_sync_instruction_alignment();

  while (1) {
    uart_send(uart_recv());
  }
}

Order (recommended order)

1. cntp_tval_el0← Write timer value
2. TIMER_CNTRL0.CNT_PNS_IRQ← Enable peripheral interrupt path
3. PSTATE.I← Disable CPU IRQ master switch
4. cntp_ctl_el0.enable← Start timer
5. PSTATE.I← Enable CPU IRQ master switch

This order ensuresthat before the CPU enables IRQ, both the peripheral and timer are ready, so once an interrupt is triggered, the CPU can immediately receive and handle it.

Duringthe bare-metal/kernel initialization phase, the timer is not yet ready. If the CPU at this timeEnable IRQ, it may beinterrupted by other peripheral interrupts, resulting in:

• The initialization process is interrupted, and the configuration register may only be half done.
• Before the timer or peripheral interrupt configuration is complete, an interrupt is pending → resulting in lost interrupts or out-of-order execution.

Interrupt Experiment 2: Using Assembly Functions to Save and Restore Interrupt Context

The key point isUsing the function method, the lr register is corrupted

GIC Interrupt Controller

• Traditional interrupt controllers, such as the legacy interrupt controller on the Raspberry Pi 4B
  • Interrupt Enable Register
  • Interrupt Disable Register
  • Interrupt Status Register
• The traditional method of managing interrupts using simple status registers is becoming increasingly difficult to manage.
  • Interrupt sources are becoming more and more numerous.
  • Different types of interrupts, such as inter-core interrupts, interrupt priorities, software-defined interrupts, etc.

The GIC mainly has two major functional blocks：

• Distributor: All interrupt sources in the system are connected to the distributor. The distributor provides registers to control the attributes of each interrupt, such as priority, status, security, routing information, and enable state. The distributor determines which interrupt to forward to the core through the attached CPU interface.
• CPU Interface: The CPU receives interrupts through it. The CPU interface provides registers to mask, identify, and control the state of interrupts forwarded to that core. Each core in the system has a separate CPU interface.

Interrupt types supported by the GIC

• SGI （Software Generated Interrupt), software-generated interrupts, soft interrupts, used to send interrupt signals to other CPU cores

• PPI （Private Peripheral Interrupt), private peripheral interrupts, which are unique to a specific CPU

• SPI（Shared Peripheral Interrupt), shared peripheral interrupts, which can be accessed by all CPUs

• LPI （Locality-specific Peripheral Interrupt), local special peripheral interrupts,New interrupt types added in GICv3。Message-based interrupt types

  

Interrupt trigger type

Each interrupt type is eitherEdge-triggeredorLevel-sensitive:

controlled byGICD_ICFGRnregister

• Edge-triggered: It is considered active when the GIC detects a rising edge on the relevant input, and remains active until cleared
• Level-sensitive: It is active only when the relevant input of the GIC is at a high level

Interrupt ID

banked interrupt means that forSGI and PPI, although their IDs are the same across the system (for example, Timer PPI isID30on all cores), butThey are private, each CPU has its own copy, and they do not interfere with each other.

• SGI (0–15): When a core triggers SGI0, it does not affect the SGI0 of other cores.
• PPI (16–31): The Timer PPI of core 0 (e.g., ID30) and the Timer PPI of core 1 (ID30) are independent and do not share.

Interrupt IDs 1020~1023 are reserved

Interrupt priority

Each interrupt priority is set in theGICD_IPRIORITYRnregister

Width of the priority field

• Each interrupt priority field is 8 bits
• The number of priority levels supported by the actual implementation = 2ⁿ, n ∈ [4, 8] (i.e.,16 to 256 levels）。
• If the hardware implements fewer than 8 bits, for example only 5 bits, then writing to the lower 3 bits isRAZ/WI(read as 0, write ignored).

Value size and priority

• The smaller the value, the higher the priority
• The maximum priority value is implementation-defined
• By default, initialization should assign a medium priority to all interrupts to avoid disrupting system scheduling, commonly 0xa0 or 0x80

Interrupt state

• inactive(Inactive state):The interrupt is in an invalid state

• pending(Pending state):The interrupt is in a valid state but waiting for the CPU to respond

• active(Active state):The CPU has responded to the interrupt

• active and pending(Active and pending state):The CPU is responding to the interrupt, but the interrupt source has sent another interrupt

  

Interrupt routing

GICD_ITARGETSRn (Interrupt Processor Targets Registers)

GICD_ITARGETSRn registerUsed to configure which CPU the Distributor can route interrupts to., is a 32-bit register, typically everyinterrupt has a corresponding register.。

• 8 bits represent one interrupt source, with four interrupt sources total, each bit indicating a routable CPU.
• byte-accessible
• If a bit is set, it means the interrupt source can be routed to that CPU.
• The routing configuration for the first 32 interrupt sources is set by hardware, read-only.
• Interrupts 33 to 1019 can have their routing configured by software, read-write.

Corresponds toCalculation method：

Assume m is the interrupt ID, n is the corresponding register number.

• Relationship between GICD_ITARGETSRn registers and interrupt sources
  • Each GICD_ITARGETSRn controls four interrupt sources.
  • n = m DIV 4
  • After calculating n, add the base address offset 0x800 of the GICD_ITARGETSRn register.
• Byte offset of the Priority field
  • m MOD 4Used to determine the byte offset corresponding to the target interrupt source
    • Offset 0corresponds to the register’s**[7:0]bits (i.e., the lowest byte), corresponding to interrupt IDm % 4 == 0**。
    • Offset 1corresponds to the register’s**[15:8]bits, applicable tom % 4 == 1**。
    • Offset 2corresponds to the register’s**[23:16]bits, applicable tom % 4 == 2**。
    • Offset 3corresponds to the register’s**[31:24]bits, applicable tom % 4 == 3**。

GICD_ITARGETSRn is an array register

GICv2 interrupt controller

• The Distributor registers (GICD_) includesinterrupt setup and configuration
• The CPU Interface registers (GICC_) includesCPU-specific special settings

The priority and target core for interrupt delivery are configured in the distributor.

Interrupts sent by the peripheral distributor are in a pending state. The distributor determines the highest priority pending interrupt that can be delivered to a core and forwards it to the CPU interface. At the CPU interface, the interrupt is sent to the core, which then accepts a FIQ or IRQ exception. The core executes the exception handler in response. The handler queries the interrupt ID from the CPU interface register and begins executing the interrupt service routine. Upon completion, the handler must write to the CPU interface register to report the end of processing.

The distributor provides registers that report the current status of different interrupt IDs. In multi-core/multi-processor systems, a single GIC can be shared by multiple cores (up to 8 in GICv2). The GIC provides registers to control which cores correspond to SPIs. This mechanism enables the operating system to share and distribute interrupts across cores and coordinate work.

Configuration

The GIC’s register implementations are allin an external memory-mapped form. All cores can access the common distributor, but the CPU interface is banked, meaning each core uses the same address to access its own private CPU interface. One core cannot access another core’s CPU interface.

The distributor contains many registers that you can use to configure the attributes of individual interrupts. These configurable attributes include:

• Interrupt priority（GICD_IPRIORITY), which the distributor uses to determine which interrupt is forwarded next to the CPU interface.
• Interrupt configuration（GICD_ICFGR). This determines whether the interrupt is level-sensitive or edge-sensitive. Not applicable to SGI.
• An interrupt target core（GICD_ITARGETSR). This determines which cores the interrupt can be routed to. Applicable only to SPI.
• Interrupt enable or disable status（GICD_ISENABLERGICD_ICENABLER). Only those interrupts enabled in the distributor are eligible to be routed to the CPU interface when they are in a pending state.
• Interrupt security (GICD_IGROUPR) Determines whether the interrupt is assigned to secure or non-secure.
• Interrupt status。

The distributor also provides priority masking, which can prevent interrupts below a certain priority from reaching the core. The distributor uses this when determining whether a pending interrupt can be forwarded to a specific core. The CPU interface on each core helps fine-tune interrupt control and processing.

Initialization：

Both the distributor and CPU interface are disabled at reset. The GIC must be initialized after reset before it can deliver interrupts to the core.

In the distributor, software must configure priority, target core, security, and enable individual interrupts. Subsequently, it must enable the distributor through its control register (GICD_CTLR) Enable distributor。

For each CPU interface, software must set the priority mask and priority preemption. Each CPU interface itself must be enabled through its control register (GICD_CTLR) Enable.

Before the CPU processes interrupts, the software configures the CPU to be ready to accept valid interrupt vectors from the interrupt vector table, clears the interrupt mask bit in PSTATE, and sets up routing control.

The entire interrupt mechanism in the system can be disabled by disabling the Distributor. Interrupt delivery to a single core can also be disabled by disabling its CPU interface. Individual interrupts can also be disabled (or enabled) in the Distributor.

For an interrupt to reach a core, the individual interrupt, the Distributor, and the CPU interface must all be enabled. The interrupt also needs sufficient priority, i.e., higher than the core’s priority mask.

Interrupt Handling

When the core accepts an interrupt, it jumps to the top-level interrupt vector table and begins execution. The top-levelinterrupt handler reads a register from the CPU interface to obtain the interrupt ID。

Reading the register not only returns the interrupt ID but also causes the interrupt to be marked as active in the Distributor. Once the interrupt ID is known (identifying the interrupt source), the top-level handler can schedule a device-specific interrupt handler to service the interrupt.

When the interrupt handler finishes execution, the top-level handler writes the same interrupt ID to the End of Interrupt (EoI) register in the CPU interface block, indicating the end of interrupt processing.

Besides removing the active state, making the final interrupt state Inactive or Pending (if the state was Active and Pending), this allows the CPU interface to forward more pending interrupts to the core. This concludes the processing of a single interrupt.

There may be multiple interrupts waiting to be serviced on the same core, but the CPU interface can only signal one interrupt at a time. The top-level interrupt handler can repeat the above sequence until it readsthe special interrupt ID value 1023，indicating that there are no more pending interrupts on this core. This special interrupt ID is calledthe spurious interrupt identifierThe spurious interrupt ID is a reserved value and cannot be assigned to the system.After the top-level handler reads the spurious interrupt ID, it can complete its execution and prepare the kernel to resume the task it was executing before the interrupt.

The Generic Interrupt Controller (GIC) typically manages inputs from multiple interrupt sources and distributes them to IRQ or FIQ.

Interrupt State Timing Diagram

M and N represent two interrupts respectively, Input indicates the interrupt signal, State indicates the interrupt state machine, and nFIQCPU[n] indicates the FIQ signal connected to the CPU core.

Assume that both M and N are SPI interrupts, and N has a higher priority than M.

GICv2 Registers

Some registers are described by interrupt number, for example, using certain bits to describe the attributes of an interrupt number. There can be n instances of the same register, such as the GICD_ISENABLERn register, which is used to enable a specific interrupt number. ‘n’ indicates that there are n such registers.

Calculation method: Calculate the corresponding register based on the interrupt number

For some registers, it will prompt:

A register bit corresponding to an unimplemented interrupt is RAZ/WI.

RAZ = Read-As-Zero
If you read this register bit, the return value is always0, even if you have written other values.

WI = Write-Ignored
If you write data to this bit,the hardware will ignore it, without changing or reporting an error.

GICD_CTLR

Distributor Control Register

GICD_ITARGETSRn

Interrupt Processor Targets Registers, see the interrupt routing chapter

GICD_TYPER （Interrupt Controller Type Register）

Describes the capabilities and configuration parameters of this GIC, such as how many interrupts, how many CPU interfaces, whether certain features are supported, etc.

GICD_ICFGRn

Interrupt Configuration Registers

GIC is used to configureinterrupt trigger typeregisters

EachGICD_ICFGRnis a32-bit register。

distribution method

• Each interrupt requires2 bit （Int_config) to describe, soOne register can configure 16 interrupts。

• Thenth register (ICFGRn) corresponds to the interrupt range:

  1	中断 ID = n*16 ~ (n*16 + 15)

Int_config field encoding

According to ARM GICv2 TRM (Table 4-18):

ForPPI(Private Peripheral Interrupt) andSPI(Private Peripheral Interrupt)

GICD_ISENABLERn

Interrupt Set-Enable Registers

Responsible forEnabling interrupt forwarding

• GICD_ISENABLERnisa 32-bit register

• EachGICD_ISENABLERncontrols32 interrupt sources。

  

• Write 1→ enables the corresponding interrupt (allowing the Distributor to forward it to the CPU interface)

• Write 0→ has no effect

• Read→ shows whether an interrupt is currently enabled (1=enabled, 0=disabled)

Based on the interrupt numbercalculate n of GICD_ISENABLERn

GICD_ICENABLERn

Interrupt Clear-Enable Registers

• W1C (Write 1 to Clear), disables an interrupt (prevents distribution to CPU)

GICD_ISACTIVERn

Interrupt Set-Active Registers

• W1S (Write 1 to Set), software canManually mark an interrupt state as active。

GICD_ICACTIVERn

Interrupt Clear-Active Registers

• W1C (Write-1-to-Clear), clears the active state of the interrupt.

• It is somewhat similar to EOI (End of Interrupt), butEOI is a CPU interface register, and can only be used by the current CPU to perform the ‘interrupt service completion’ action for interrupts it receives;

• ICACTIVERnInDistributor, software canglobally force-clear the active bit of a specific interrupt。

GICD_IPRIORITYRn

Interrupt Priority Registers

Used forSetting the priority of each interrupt

• Each interrupt occupies 8 bits

• A 32-bit register manages 4 interrupts

Address calculation:

GICC_CTLR

CPU Interface Control Register

GICC_PMR

Interrupt Priority Mask Register

Priority mask register of the CPU interface, which determineswhich priority interrupts the CPU can receive。

• Onlyinterrupts with priority value ≤ PMRcan be received by the CPU.
• Note: The priority of the GIC isthe smaller the value, the higher the priority。

Usage example: Assume your GIC implementation has 8-bit priority:

1. IfGICC_PMR = 0xff
   • the CPU receives interrupts of all priorities (most common initialization setting).
2. IfGICC_PMR = 0xa0
   • only receivesinterrupts with priority value ≤ 0xa0interrupts, those with a lower priority will be masked.
3. IfGICC_PMR = 0x0
   • only the highest priority (0) interrupt is accepted, all others are masked

GICC_IAR

Interrupt Acknowledge Register

When the CPU receives an IRQ signal, the software needs to read the currentID of the pending interrupt, and confirm which IRQ it is.

GICC_EOIR

End of Interrupt Register

After completing interrupt processing, the software must write to this register to tell the GIC: “This IRQ has been processed, the active state can be cleared, and subsequent same interrupts are allowed to trigger again.”

GIC-400 of Raspberry Pi 4B

• ARM Core interrupts:
  • Core n HP tiemr IRQ
  • Core n V timer IRQ
  • Legacy FIQn
  • Core n PS timer IRQ
  • Core n PNS timer IRQ (PPI ID 30)
  • Legacy IRQn
• ARM local interrupts
  • ARM Mailbox IRQs
  • Core 0 PMU IRQ
  • Core 1 PMU IRQ
  • Core 2 PMU IRQ
  • Core 3 PMU IRQ
  • AXIERR IRQ
  • Local timer IRQ
• 16 ARM Core peripheral interrupts
• 64 VC peripheral interrupts
• 51 PCI-related peripheral interrupts

Accessing GIC-400 registers

• Base address of GIC-400 on Raspberry Pi 4B

Access: Raspberry Pi GIC-400’sBase address + GIC-400 memory map offset + Register offset

GIC400 initialization flow

1. Set the base addresses of the distributor and CPU interface register groups
2. ReadGICD_TYPERthe register, calculate the maximum number of interrupt sources currently supported by the GIC
3. Initialize the distributor
   1. Disable distributor, setGICD_CTLR(This step can be omitted because it is disabled by default at reset)
   2. Set the routing of SPI interrupts
      1. The routing of the first 32 interrupts is fixed by the GIC chip, so first readGICD_ITARGETSRnthe previous value to obtain all CPUs that can be routed
      2. SetSPI (Shared Peripheral Interrupt)routing to distribute interrupts to all routable CPUs, because SPI interrupts are shared interrupts, set the SPI’sGICD_ITARGETSRn
   3. Set the trigger type of SPI interrupts, for example, level-triggered, setGICD_ICFGRn
   4. Deactivate and disable all SPI interrupt sources
   5. Enable SGI interrupts (0-15), used by SMP
   6. Enable distributor, setGICD_CTLR
4. Initialize CPU interface
   1. Set default interrupt priority for the first 32 interrupt sources, setGICD_IPRIORITYRn
   2. SetGICC_PRM, set interrupt priority mask level
   3. Enable CPU interface, setGICC_PRM

Register interrupt

1. Initialize peripheral
2. Find the interrupt number of this peripheral in GIC-400, for example, the interrupt number of PNS timer is 30
3. SetGICD_ISENABLERnregister to enable this interrupt number
4. Enable device-related interrupts, for example, the generic timer on Raspberry Pi requires enabling ARM_TIMER in LOCAL register_The relevant enable bit in the CNTRL0 register
5. Set the I bit in the CPU’s PSTATE

Interrupt response

1. Interrupt occurrence
2. Exception vector table
3. Jump to the GIC interrupt function, gic_handle_irq()
4. ReadGICC_IARthe register to obtain the interrupt number
5. Perform corresponding interrupt handling based on the interrupt number. For example, if the read interrupt number is 30, it indicates the PNS generic timer, then jump to the generic timer’s handler function.

GIC interrupt experiment 1: Implementing the generic timer

GIC-400 initialization

1. SetdistributorandCPU interfacethe base address of the register group

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42

#ifndef __GIC_V2_H
#define __GIC_V2_H

#include "asm/base.h"

#define GIC_V2_DISTRIBUTOR_OFFSET 0x1000
#define GIC_V2_CPU_INTERFACE_OFFSET 0x2000
#define GIC_V2_DISTRIBUTOR_BASE (GIC_400_BASE + GIC_V2_DISTRIBUTOR_OFFSET)
#define GIC_V2_CPU_INTERFACE_BASE (GIC_400_BASE + GIC_V2_CPU_INTERFACE_OFFSET)

#define GICD_CTLR (GIC_V2_DISTRIBUTOR_BASE + 0x000)

#define GICD_TYPER (GIC_V2_DISTRIBUTOR_BASE + 0x004)
#define GICD_TYPER_ITLINESNUMBER 0x1f
// Using n / 4 or n / 16 directly gives the register index
// Then multiply by 4 in the address calculation (4 bytes per register)
#define GICD_ITARGETSRn(n) (GIC_V2_DISTRIBUTOR_BASE + 0x800 + (n / 4) * 4)
#define GICD_ICFGRn(n) (GIC_V2_DISTRIBUTOR_BASE + 0xc00 + (n / 16) * 4)

#define GICD_ICFG_LEVEL_SENSITIVE 0x0
#define GICD_ICFG_LEVEL_EDGE_TRIGGERED 0x2

#define GICD_ISENABLERn(n) (GIC_V2_DISTRIBUTOR_BASE + 0x100 + (n / 32) * 4)
#define GICD_ICENABLERn(n) (GIC_V2_DISTRIBUTOR_BASE + 0x180 + (n / 32) * 4)
#define GICD_ICACTIVERn(n) (GIC_V2_DISTRIBUTOR_BASE + 0x380 + (n / 32) * 4)

#define GICD_IPRIORITYRn(n) (GIC_V2_DISTRIBUTOR_BASE + 0x400 + (n / 4) * 4)
#define DAFAULT_IPRIORITY 0xa0a0a0a0

#define GICC_PMR (GIC_V2_CPU_INTERFACE_BASE + 0x4)
#define GICC_CTLR (GIC_V2_CPU_INTERFACE_BASE + 0x0)

#define GICC_IAR (GIC_V2_CPU_INTERFACE_BASE + 0xc)
#define GICC_IAR_CPU_ID_MASK 0x1c00
#define GICC_IAR_INT_ID_MASK 0x3ff
#define GICC_EOIR (GIC_V2_CPU_INTERFACE_BASE + 0x10)

int gic_init(void);
void gic_enable_irq(int irq);

#endif

2. ReadGICD_TYPERRegister, calculate the maximum number of interrupt sources currently supported by the GIC

3. Initializedistributor

   

   

4. Initialize CPU interface

   

5. Enable PNS_TIMER_IRQ routing

4. Test

GIC Interrupt Experiment 2: Implementing System Timer on Raspberry Pi

GICv3 Interrupt Controller

What improvements does GICv3 have over GICv2

• GICv3 is compatible with GICv2
• Supports more CPU cores, >8
• Supports message-based interrupts (Message Signaled Interrupt，MSI）
• SupportsITS（Interrupt Translation Service) service
• Supports more hardware interrupt numbers, >1020
• To better comply with the ARMv8 exception model, supportsinterrupt groups（Interrupt grouping）
• To optimize access latency,provides system register access to the CPU Interface

Interrupt types supported by GICv3

• Private Peripheral Interrupt（PPI）
  • PPI refers to interrupts specific to a local CPU, such as internal CPU timers. Different CPUs can use the same PPI interrupt number
  • PPI can be in group0 and group1
  • Can be edge-triggered or level-sensitive
• Shared Peripheral Interrupt（SPI）
  • SPI is typically used for peripheral interrupts and can be routed to any CPU
  • SPI can be in group0 and group1
  • Can be edge-triggered or level-sensitive
• Software Generated Interrupt（SGI）
  • SGI is typically a software-triggered interrupt used for inter-core communication, such as IPI (Inter-Processor Interrupts)
  • SGI can only be edge-triggered
• New:Locality-specific Peripheral Interrupt（LPI）
  • In non-secure interrupt group 1
  • Edge-triggered
  • Using ITS service
  • No active state
  • Message-based interrupt

Wired interrupt and message-based interrupt

• Both SPI and LPI support message-based interrupts
  • SPI message-based interrupts do not need to go through ITS; writingGICD_SETSPI_NSRthe interrupt number to a register triggers the interrupt
  • LPI message-based interrupts require ITS

Interrupt number assignment

Interrupt state machine

• InactiveInactive state
• PendingInterrupt triggered but not yet acknowledged by the CPU
• ActiveInterrupt acknowledged and being processed by the CPU
• Active & PendingWhen an interrupt is being acknowledged and processed by the CPU, and another identical interrupt triggers, the new interrupt is set to active & pending
• LPINo active and active & pending states

Question: What if, while the GIC is acknowledging an interrupt, another identical interrupt triggers?

Two cases need to be considered here:

• If the GIC is responding to the first interrupt, meaning the first interrupt’s state is active, and a second identical interrupt triggers, the second interrupt’s state becomes active & pending, thus preventing loss of the interrupt.
• If the first interrupt is still in the pending state and another identical interrupt arrives, they will merge into one.

Interrupt Affinity Routing Hierarchy

• GICv3 supports 4-level routing
• Level 0 faces the redistributor

Interrupt Affinity Routing of GIC-500

• GIC-500 supports two-level affinity routing
  • Level 0 is the core
  • Level 1 is the cluster
• GIC-500 supports up to 128 cores and 32 clusters

0.0.0.1 indicates the 1st core in the 0th cluster

0.0.1.1 indicates the first core in the first cluster

• Interrupt groups and security modes
  • ARMv8 supports secure and non-secure modes
  • GICv3 supports EL0 to EL3, so each interrupt source needs to be configured with the corresponding interrupt group and security mode
    • Group0 is used for EL3
    • Secure Group1: used for Trust OS on EL2
    • Non-secure Group1: used for VMM or OS

Group0 uses FIQ, Group1 uses IRQ or FIQ depending on the situation

Example

Note: Whether an interrupt is routed to EL3 depends on the FIQ and IRQ fields of the SCR_EL3 register

• In non-secure mode:
  • Non-secure Group1 interrupts are handled directly in the Rich OS
  • Secure Group1 and Group0 FIQ interrupts are routed to EL3
• In secure mode
  • IRQ interrupts of secure group 1 are directly responded to in the Trusted OS
  • FIQ interrupts of non-secure group 1 and group 0 are routed to EL3

Special interrupt number

Usage of interrupt number 1021 - 1

1. When the CPU is running in the Trusted OS in secure mode and an interrupt from a non-secure OS arrives, it needs to trap to EL3 via FIQ for handling
2. The CPU traps to the secure monitor at EL3, which reads the IAR register and obtains 1021, indicating that this interrupt is expected to be handled in non-secure mode. It then switches to the Rich OS in non-secure mode
3. The CPU switches to the Rich OS in non-secure mode to handle this interrupt

Usage of interrupt number 1021 - 2

1. When the CPU is running in the Trusted OS in secure mode and an interrupt from a non-secure OS arrives, it needs to trap to EL3 via FIQ for handling. However, because SCR_EL3.FIQ=0, it can only be handled in EL2
2. The Trusted OS at EL2 executes an SMC system call to trap to EL3
3. The secure monitor reads the IAR register and obtains 1021, indicating that this interrupt is expected to be handled in non-secure mode. It then switches to the Rich OS in non-secure mode
4. The CPU switches to the Rich OS in non-secure mode to handle this interrupt

GICC and ICC

ICC（Interrupt Controller CPU interface）Specifically refers to the part of the GICthat directly interacts with the CPU, accessed viaSystem Registersvisit

Why are the CPU interface registers of GICv2 calledGICC_*, while those of GICv3 are calledICC_*？

The core of the answer lies in:There has been a fundamental change in the implementation of the CPU Interface between GICv2 and GICv3— fromMemory-Mapped I/O (MMIO)toSystem Registers. ARM clearly distinguishes these two architectures through naming differences.

Interrupt Priority

• GICv3 supports 8-bit priority, up to 256 levels
  • When supporting 2 security modes, at least 32 interrupt priorities are supported, up to 256
  • When supporting 1 security mode, at least 16 interrupt priorities are supported
• Interrupt Priority
  • The smaller the value, the higher the interrupt priority; 0 indicates the highest priority, 255 indicates the lowest priority, idle priority
  • GICR_IPRIORITYR<n>Set PPI and SGI interrupt priority
  • GICD_IPRIORITYR<n>Set SPI interrupt priority
  • The LPI configuration table stores the interrupt priority of LPIs

Interrupt Priority Group Register (ICC_BPR0_EL1/ICC_BPR1_EL1）

• Each interrupt in the GIC has a8-bit priority value (priority field), the smaller the value, the higher the priority (e.g., 0x00 is the highest priority, 0xFF is the lowest).
• The CPU Interface, when deciding whether topreempt the currently handled interruptWhen comparing, it checks the priority of the new interrupt against the current interrupt.

However, the GIC does not simply compare the entire 8-bit value — it uses**Binary Point Register（BPR）**to divide the priority into two parts:

• ICC_BPR0_EL1: Used forGroup 0interrupts
• ICC_BPR1_EL1: Used forGroup 1interrupts

Note: The grouping for group1 and group0 is slightly different

Interrupt priority threshold and running interrupt priority

• Interrupt threshold
  • Register:ICC_PMR_EL1（Priority Mask Register）
  • PMR determines the priority threshold of the target CPU. The GIC only sends an interrupt to the CPU when the priority of a pending interrupt is higher than this interrupt priority threshold.
  • PMR being 0 means all interrupts sent to the CPU are masked.
• Running priority
  • Register:ICC_RPR_EL1（Running Priority Register）
  • Read-only register, returns theGroup Priority of the highest priority interrupt currently being processed on the CPU。

Interrupt priority preemption

• GICv3Supports interrupt preemption, when an interrupt priority simultaneously meets the following conditions
  • Priority is higher than the priority threshold PMR of the CPU interface
  • Group priority is higher than the running priority being processed

GICv3 internal architecture

• Distributor: Priority queuing, dispatching SPIs and SGIs to redistributors
• Redistributor: Connects to the CPU interface. One redistributor per CPU
• CPU interface: Sends interrupts to the CPU, acknowledges interrupts, etc.
• ITS: Interrupt translation service, converts LPI interrupt requests to interrupt IDs and sends them to the redistributor

ITS service (Interrupt Translation Service)

• ITS function: Converts device_ID’s Event_ID into:
  • Hardware interrupt number (INTID)
  • Target redistributor
• ITS translation process
  • Use Device_ID to query the device table
  • Use Event_ID to query the Interrupt translation table
    • Physical interrupt number INTID
    • interrupt collection number
  • Query the Collection table by ICID to obtain the target redistributor
• Five ITS tables
  • Interrupt configuration table (configure table)
  • Interrupt pending table
  • device table
  • Interrupt translation table
  • Collection table

Configuration table and pending table

• Interrupt configuration tableUsed to store the priority and enable bit of each LPI interrupt

  • Each table entry occupies 8 bits
  • Configuration table base address: GICR_PROPBASER.Physical_Address
  • Number of table entries: 2^(GICR_PROPBASER.IDbits)

  

• Interrupt pendingUsed to indicate the pending status of each LPI interrupt
  • Each table entry occupies 1 bit
  • Each redistributor has an interrupt pending table

Creation of Device Table

• Device Table is created by OS software

  • Allocate memory, create table
  • Set the base address of the table to GITS_BASER.Physical_Address

• Important parameters of Device Table are inGITS_BASER\<n\>registers

  • Table type: GITS_BASER.Type
  • Entry size: GITS_BASER.Entry_Size (8 bytes)
  • Size of each page in the table: GITS_BASER.Page_Size（eg, 64KB）
  • Whether a second-level page table is needed: GITS_BASER_Indirect
  • Set table base address: GITS_BASER.Physical_Address
  • Number of entries needed: 2^(device_id bit width), device_id bit width in GITS_TYPER.devbits

• Device Table supports level-1 or level-2 tables

• The content of a Device Table entry consists of:

  • Software sends commands to hardware via the command queue, and hardware fills the table entries
  • Map deviceID to the ITT table via the MAPD command

• Device Table entries are called DTEs, used to point to the ITT table

• ITT entries are called ITEs, used to describe the relationship between EventID and the final physical ID number

Creation of the Interrupt Translation Table

• ITT entries are used to mapEventID -> physical interrupt number INTID and ICID
• ITT Important Parameters
  • ITT Entry Size: GITS_TYPER.ITT_entry_size (e.g., 16 bytes)
  • Number of ITT Entries: Number of vectors requested when applying for interrupts
  • Base Address of ITT Table: Allocated by the OS
• Content of ITT Entries:
  • Software sends commands to hardware via the command queue, and hardware completes them
  • Map deviceID to the ITT table via the MAPD command

Creation of Collection Table

• Collection Table entries are used to map:ICID -> redistributor

• This table does not require memory allocation in memory

• Content of Collection Table Entries:

  • Software sends commands to hardware via the command queue, and hardware completes them
  • Software informs: physical address of the redistributor or GICR_TYPER.Processor_Number
  • Map ICID -> redistributor via the MAPC command
  • During GIC initialization, traverse all redistributors and call the MAPC command to initialize.

  1 2 3 4 5 6 7

  gic_smp_init()-> 遍历所有present CPU()-> gic_starting_cpu()-> its_cpu_init()-> its_cpu_init_collections()-> its_cpu_init_collection()-> its_send_mapc()发送MAPC命令初始化collection table

• Collection table entry CTE

ITS Command Queue

• Three registers related to the command queue
  • GITS_CBASER: Specifies the size and base address of the command queue. The base address must be 64KB aligned, and the size must be a multiple of 4KB.
  • GITS_CREADR: The next command to be processed by the ITS
  • GITS_CWRITER: The next command to be written
• Common Commands
  • MAPD: Maps device ID to the ITT table
    • MAPD <DeviceID>, <ITT_addr>, <size>
  • MAPI: Maps event ID and device ID to the ITT
    • MAPI <DeviceID>, <EventID>, <Collection ID>
  • MAPTI: Maps event ID, device ID, and hardware interrupt number to the ITT
    • MAPTI <DeviceID>, <EventID>, <INTID>, <Collection ID>
  • MAPC: Maps collection ID to the target redistributor
    • MAPC <Collection ID>, <Target Redistributor>

Example

Assume a device has Device ID 5, and we want to map Event ID=0 to physical interrupt number 8192, with the ITT table base address at 0x850000. The corresponding Collection ID is 3, and the target redistributor’s physical address is 0x78400000.

The Linux kernel prefers to use vectors to represent event IDs

Implementation of ITS in Linux

IRQ domain interrupt control domain

• Possibility of multi-level interrupt controllers in the system
  • Traditional interrupt controller - GIC
  • Can be abstracted as interrupt controllers: GIC, ITS, GPIO, etc.
  • Virtual interrupt controller, platform irqdomain
• IRQ Domain is seen as a software abstraction of the IRQ Controller

ITS driver framework

API for creating MSI interrupts

Example of using MSI interrupts in a platform device - SMMUv3

Flowchart of platform requesting MSI interrupt allocation

Flowchart of PCI device allocating MSI interrupt

How does an IO device trigger an interrupt?

For GICv3 interrupt controller, the IO device needs to write the event ID to the GITS_TRANSLATER register to trigger an MSI interrupt

The Linux kernel encapsulates a struct msi_msg data structure, which includes the address of this register and the data to be written

1
2
3
4
5

struct msi_msg{
	u32 address_lo;
	u32 address_hi;
	u32 data;
}

In irq_The chip’s ops has an its_irq_compose_msi_msg callback function, used to fill this msi_msg, which writes the GITS_physical address of the TRANSLATER register into the msi_msg->address field

When the device driver uses platform_msi_domain_alloc_irqs() to register MSI, it will get from msi_Obtain GITS from the msg data structure_The physical address and eventID of the TRANSLATER register, then write them into the IO device’s own registers (e.g., SMMU in SMMU_EVENTQ_IRQ_CFG0 and CFG1).

When a device wants to trigger an MSI, it automatically writes the eventID into its own register to trigger the interrupt.

Take SMMU as an example:

ITS Debug Tips

• The latest QEMU supports ITS. You can use QEMU + Linux kernel to step-debug ITS and MSI.
• Add “irq_gic_v3_its.dyndbg=+pflmt irqdomain.dyndbg=+pflmt” to enable related dynamic prints:

• You can add “dump_stack()” in the its_domain_ops callback to print the call trace of function calls.