Timeline
Timeline
2025-10-31
init
This article introduces the fifth-generation hardware cache coherence protocol CHI in the AMBA 5 architecture, detailing its mesh structure, layered architecture, and packet-based communication mechanism designed specifically for high-performance multi-core systems. It further discusses the differences between CHI and the ACE protocol, the functional classification of various nodes (RN, HN, SN), the Flit transmission format and ID allocation mechanism at the link layer, as well as key technical details such as system address mapping and completion acknowledgment.
Reference Documents:
CHI Bus
It is the fifth-generation protocol defined in AMBA 5, an evolution of ACE, designed specifically for high-performance, multi-core processor systems, supporting more complex cache coherence management and large-scale system integration, suitable for large-scale multi-core system scenarios requiring high performance and complex cache coherence management.
Document:
- AMBA 5 CHI Architecture Specification，issue E.b
- Arm CoreLink CI-700 Coherent Interconnect Technical Reference Manual
- CHI (Coherent Hub Interface) is the next-generation hardware cache coherence protocol, aiming to adapt to varying numbers of processors and peripherals.
- Small systems: Embedded
- Medium systems: Mobile phones
- Large systems: Data center
- The cache coherence protocol is similar to ACE
- Supports hierarchical design
- Protocol layer
- Transport layer
- Link layer
Common bus connection structure
Mesh structure: CI-700 or CMN-600
Introduction to CI-700 controller
Supports up to 8 CPU clusters
Supports up to 12 crosspoints (XP): hardware components for routing or switching
- CI-700 forms a mesh through XPs
- Each XP can have up to 4 neighboring XPs: up, down, left, and right
- Each mesh can support 4 device ports
- Each device port can be used to connect a cache-coherent master (RN-F) or slave device (SN-F)
Supports up to 8 RN-F interfaces for connecting CPU clusters, GPUs, accelerator cards, or other cached master devices
Supports up to 8 HN-F and up to 32MB of system cache
Supports up to 8 SN interfaces
Differences between ACE and CHI
- Similarities:
- Goal: hardware cache coherence solution
- Uses similar cache state transitions
- Understanding of snoop transactions is very similar
- Differences
- ACE uses a crossbar structure, CHI uses a mesh structure
- CHI adopts a layered design: protocol layer, transport layer, link layer
- CHI uses packet-based communication
- CHI uses the concepts of request node, home node, and slave node to describe transactions
- CHI supports more snoop transactions
- CHI supports DCT, DMT, DWT and other optimizations for transmission performance
- CHI supports atomic operations and cache stash
Node
RN (Request Node): Request Node
- RN - F: Cache Coherent Request Node
- Cache with built-in hardware cache coherence
- Allows generation of all transaction types
- Supports all snoop transactions
- RN - D: IO Cache Coherent Request Node with DVM (similar to ACE-Lite + DVM)
- Does not include hardware cache coherent cache
- Receives DVM transactions
- Can generate a subset of transaction types
- RN - I: IO Request Node
- Does not include hardware cache coherent cache
- Does not receive DVM transactions
- Can generate a subset of transaction types
- Does not support snoop
HN (home node): The master node on the system bus, used to receive transmission transactions from request nodes
- HN-F: Cache-coherent home node
- Accepts all request types
- Manages snoop requests from RN-F in a PoC manner
- Manages the order of memory requests in a PoS manner
- Includes directory or snoop filter to reduce redundant snoops
- HN-I: Home node that does not support cache coherence
- Can only handle a subset of requests
- Does not include PoC, cannot handle snoopable requests,
- PoS to handle the order of IO requests
SN: Slave node
- **SN - F：**A slave device using normal memory, it can handle non-snoopable reads/writes, atomic requests (exclusive requests), and CMO requests
- SN - I: Similar to SN-F, used for peripherals or normal memory
Examples
Cache State Machine
Compared to ACE, UCE and UDP are added
Channel
- The channels defined by CHI are completely different from ACE
- The handshake protocol in Channel is different from AXI/ACE
- FLITV signal is asserted high, indicating the transmitter is ready to send a packet and the packet is valid
- LCDRV signal is asserted high, indicating the receiver sends a credit to the transmitter: you may send
Link Layer
- The link layer provides a streamlined mechanism for packet-based communication between nodes and interconnect IP
- Provides a two-way link transmission mode
- Transmitter -> Receiver
- Receiver -> Transmitter
Packet Format - Flits
- Flit = Flow control unIT, which is the smallest unit transmitted at the link layer. A packet contains multiple Flits
- Protocol Flit: Used to transmit protocol information
- Link Flit: Used to transmit link maintenance information
- CHI uses protocol messages to convey information, including various IDs, opcodes, memory attributes, addresses, data, error responses, etc.
Flit Packet Format
Protocol flit
- CHI defines 4 types of protocol flits
- Request flit
- Response flit
- Snoop flit
- Data flit
- Each flit has its own format
ID
The CHI protocol defines many IDs:
- Source ID (SrcID): Indicates the ID of the node sending the flit packet
- Target ID (TgtID): Indicates the ID of the target node receiving the flit packet
- Transaction ID (TxnID): Each transaction has a unique ID, used for outstanding requests, supporting up to 256 outstanding transaction IDs. Similar to the transaction ID in AXI.
- Request opcode (Opcode): Used to specify the transaction type
- Data Buffer ID (DBID): Used for responses and data packets, allowing the Completer of a transaction to provide its own identifier
ID Allocation and Binding
- CHI uses the System Address Map (SAM) to convert physical addresses in transactions into target Node IDs
- Each RN and HN has a SAM
- The CHI specification does not prescribe how SAM is implemented, including the format and structure of SAM.
- Requirements imposed by CHI on SAM:
- Describe the full system address space; all SAMs must be globally consistent. For example, address 0xFF00_0000 must map to the same HN.
- For unmapped addresses, an error response mechanism must be provided.
Completion acknowledgement
- Similar to the RACK and WACK signals in ACE, used to ensure transaction ordering.
- CompACK ensures: HN-F can only process other snooping transactions after receiving the completed CompACK.
- For read transactions:
- Except for ReadNoSnp and ReadOnce* transactions, all other read transactions require CompACK.
- RN-F sends CompACK only after receiving signals such as Comp, CompData, RespSepData, etc.
- HN-F must wait for CompACK before sending snooping for other request transactions to the same address.
- For write transactions:
- Only WriteUnique and WriteNoSnp transactions require CompACK.
exclusive access
- The exclusive flow is similar to ACE.
- The flow of exclusive access:
- Execute exclusive load
- Compute
- Execute exclusive store
- If another master writes to this address, fail
- If no other master writes to this address, success
- An LP (Logical Processor) monitor must be implemented on the RN-F (master) side
- A PoC (Point of Coherence) monitor must be implemented on the HN-F node within the CHI interconnect
LP monitor is located at RN-F:
- Each RN-F must implement an exclusive monitor to observe and monitor the memory address accessed exclusively.
- When the CPU initiates an exclusive load, the LP monitor is set.
- The LP monitor is reset under the following conditions:
- If this address is modified by another LP
- If the LP performs another store operation to this address
POC monitor is located at HN-F:
- The POC monitor records every snoop transaction for exclusive access performed by an LP.
- The monitor will concurrently monitor all LP’s exclusive accesses.
- When HN-F receives an exclusive load or store operation, the monitor registers this information: someone is attempting an exclusive access.
- After an LP fails to execute an exclusive store, the LP needs to restart the access sequence of exclusive load and store.
- When HN-F receives an exclusive store operation:
- If an exclusive access record for this address has been registered in the PoC monitor and has not been reset by another LP, then the exclusive store will succeed, and all other attempted exclusive access records will be reset.
- If an LP performs an exclusive access but it is not found in the PoC monitor, then the exclusive store will fail.
Atomic access
- Atomic access is added in the CHI.B protocol.
- Atomic access allows operations and computations to be performed close to the data.
- HN-F or SN contains an ALU logic unit.
- Benefits of atomic access:
- More accurate and predictable latency.
- No contention with other requesters for the cache, reducing memory access blocking and cache thrashing.
- Fairness. When multiple requesters access the same memory address simultaneously, arbitration is done through POS or POC.
- There are four types of atomic transactions.
- AtomicStore
- AtomicLoad
- AtomicSwap
- AtomicCompare
Comparison between atomic access and exclusive access.
Atomic types
New features
cache stash
- IO device writes data directly into the target RN-F’s cache
- Similar to Intel DDIO technology
- There are 4 types of transactions supported by cache stash
- WriteUniquePtlStash
- WriteUniqueFullStash
- WriteUniqueFullStash
- StashOnceShared
DMT and DCT
- In the CHI.A protocol, read data and snoop data must first be sent to the home node before being sent to the receiver node
- Disadvantage: increased transmission length and latency
