Timeline
Timeline
2025-10-31
init
This article introduces the ARM CHI (Coherent Hub Interface) bus protocol, which is the fifth-generation protocol defined in AMBA 5. It is an evolution of ACE, designed for high-performance, multi-core processor systems, supporting complex cache coherence management and large-scale system integration. The article elaborates on CHI's layered design (protocol layer, transport layer, link layer), mesh topology, and key concepts such as Request Node (RN), Home Node (HN), and Slave Node (SN). It also compares CHI and ACE, pointing out that CHI uses packet-based communication, supports more snoop transactions and optimization features such as DCT, DMT, and DWT. In addition, it introduces mechanisms such as the Flit format at the link layer, ID allocation, and System Address Mapping (SAM).
Reference documents:
CHI bus
It is the fifth-generation protocol defined in AMBA 5, an evolution of ACE, designed for high-performance, multi-core processor systems, supporting more complex cache coherence management and large-scale system integration, suitable for large-scale multi-core system scenarios that require high performance and complex cache coherence management.
Document:
- AMBA 5 CHI Architecture Specification，issue E.b
- Arm CoreLink CI-700 Coherent Interconnect Technical Reference Manual
- CHI (Coherent Hub Interface) is a next-generation hardware cache coherence protocol, aiming to adapt to different numbers of processors and peripherals.
- Small systems: embedded
- Medium systems: mobile phones
- Large systems: data center
- Cache coherence protocol is similar to ACE.
- Supports layered design
- Protocol layer
- Transport Layer
- Link layer
Common bus connection structures
Mesh structure: CI-700 or CMN-600
Introduction to the CI-700 controller
Supports up to 8 CPU clusters
Supports up to 12 crosspoints (XP): hardware components for routing or switching.
- CI-700 forms a mesh via XPs.
- Each XP can have 4 neighboring XPs: up, down, left, and right.
- Each mesh can support 4 device ports
- Each device port can be used to connect a cache-coherent master (RN-F) or slave device (SN-F).
Supports up to 8 RN-F interfaces for connecting CPU clusters, GPUs, accelerator cards, or other master devices with cache.
Supports up to 8 HN-F and up to 32MB of system cache
Supports up to 8 SN interfaces
Differences between ACE and CHI
- Similarities:
- Goal: hardware cache coherence solution
- Uses similar cache state transitions
- Understanding of snoop transactions is very similar
- Differences
- ACE uses a crossbar structure, CHI uses a mesh structure
- CHI uses a layered design: protocol layer, transport layer, link layer
- CHI uses packet-based communication
- CHI uses the concepts of request node, home node, and slave node to describe transactions
- CHI supports more snoop transactions
- CHI supports DCT, DMT, DWT, etc. to optimize transmission performance
- CHI supports atomic operations and cache stash
Nodes
RN (request node): request node
- RN-F: cache-coherent request node
- Built-in hardware cache-coherent cache
- Allows generation of all transactions
- Supports all snoop transactions
- RN-D: IO cache-coherent request node with built-in DVM (similar to ACE-Lite + DVM)
- Does not include hardware cache-coherent cache
- Receive DVM transfer transactions
- Can generate a portion of transactions
- RN-I: I/O request node
- Does not include hardware cache-coherent cache
- Does not receive DVM transfer transactions
- Can generate a portion of transactions
- Does not support snoop
HN (home node): the master node on the system bus, used to receive transfer transactions from request nodes
- HN-F: cache-coherent home node
- Receives all request types
- Manages snoop requests from RN-F using PoC
- Manages the ordering of memory requests using PoS
- Includes directory or snoop filter to reduce redundant snoops
- HN-I: home node that does not support cache coherence
- Can only handle a portion of requests
- Does not include PoC, so cannot handle snoopable requests
- Uses PoS to handle the ordering of I/O requests
SN: subordinate node
- **SN - F：**A subordinate device using normal memory, it can handle non-snoopable read/write, atomic requests (exclusive requests), and CMO requests
- SN - I: Similar to SN-F, used for peripherals or normal memory
Example
Cache state machine
Compared with ACE, UCE and UDP are added
Channel
- The channels defined by CHI are completely different from ACE
- The handshake protocol in Channel is different from AXI/ACE
- FLITV signal asserted high indicates that the transmitter is ready to send a packet, and the packet is valid
- LCDRV signal asserted high indicates that the receiver sends a credit to the transmitter: you may send now
Link layer
- The link layer provides a streamlined mechanism for packet-based communication between nodes and interconnect IP
- Provides a two-way link transmission mode
- Transmitter -> Receiver
- Receiver -> Transmitter
Packet format - Flits
- Flit = Flow control unIT, which is the smallest unit transmitted at the link layer. A packet contains multiple flits.
- Protocol Flit: used to transmit protocol information
- Link Flit: used to transmit link maintenance information
- CHI uses message packets (protocol messages) to convey information, including various IDs, opcodes, memory attributes, addresses, data, error responses, etc.
Flit packet format
Protocol flit
- CHI defines 4 types of protocol flits
- Request flit
- Response flit
- Snoop flit
- Data flit
- Each flit has its own format
ID
The CHI protocol defines many IDs:
- Source ID (SrcID): indicates the ID of the node sending the flit packet
- Target ID (TgtID): indicates the ID of the target node receiving the flit packet
- Transaction ID (TxnID): Each transaction has a unique ID, which can be used for outstanding requests, supporting up to 256 outstanding transaction IDs. Similar to the transaction ID in AXI.
- Request opcode (Opcode): used to specify the transaction type
- Data Buffer ID (DBID): used for responses and data packets, allowing the Completer of a transaction to provide its own identifier for the transaction
ID assignment and binding
- CHI uses the System Address Map (SAM) to convert the physical address in a transaction into a target Node ID
- Each RN and HN has a SAM
- The CHI specification does not specify how the SAM is implemented, including the format and structure of the SAM
- CHI’s requirements for SAM:
- Describe the system-wide address space; all SAMs must be globally consistent, e.g., address 0xFF00_0000 must map to the same HN.
- For unmapped addresses, an error response mechanism must be provided.
Completion acknowledgement
- Similar to the RACK and WACK signals in ACE, used to guarantee transaction ordering.
- CompACK guarantee: HN-F may process other snooping transactions only after receiving the completion CompACK.
- For read transactions:
- Except for ReadNoSnp and ReadOnce* transactions, all other read transactions require CompACK.
- RN-F sends CompACK only after receiving signals such as Comp, CompData, RespSepData.
- HN-F must wait for CompACK before sending snoops for other request transactions to the same address.
- For write transactions:
- Only WriteUnique and WriteNoSnp transactions require CompACK.
exclusive access
- The exclusive flow is similar to ACE.
- Exclusive access flow:
- Execute exclusive load
- Calculate
- Execute exclusive store
- If another master writes to this address, the operation fails.
- If no other master writes to this address, success
- An LP (Logical Processor) monitor must be implemented at the RN-F (master) side.
- A PoC (Point of Coherence) monitor must be implemented at the HN-F node inside the CHI interconnect bus.
LP monitor is located at RN-F:
- Each RN-F must implement an exclusive monitor to observe and monitor the memory address being exclusively accessed.
- When the CPU starts an exclusive load, the LP monitor is set.
- The LP monitor is reset in the following cases:
- If this address is modified by another LP.
- If the LP performs another store operation to this address.
POC monitor is located at HN-F:
- The POC monitor records every snoop transaction for exclusive access performed by each LP.
- The monitor monitors exclusive accesses of all LPs in parallel.
- When HN-F receives an exclusive load or store operation, the monitor registers this information: a certain LP is attempting an exclusive access.
- When an LP fails to execute an exclusive store, the LP needs to restart the access sequence of exclusive load and store.
- When HN-F receives an exclusive store operation:
- If the exclusive access record for this address is already registered in the PoC monitor and has not been reset by other LPs, the exclusive store succeeds, and then all other attempted exclusive access records are reset.
- If an LP performs an exclusive access but it is not found in the PoC monitor, the exclusive store fails.
Atomic access
- Atomic accesses are added in the CHI.B protocol.
- Atomic accesses allow operations and computations to be performed close to the data.
- HN-F or SN contains an ALU.
- Benefits of atomic accesses:
- More accurate and predictable latency.
- No need to contend with other requesters for the cache, reducing blocking and cache thrashing when accessing memory.
- Fairness. When multiple requesters access the same memory address simultaneously, arbitration is performed through PoS or PoC.
- There are four types of atomic transactions.
- AtomicStore
- AtomicLoad
- AtomicSwap
- AtomicCompare
Comparison of atomic access and exclusive access.
Atomic types.
New features.
cache stash
- I/O devices directly write data into the cache of the target RN-F.
- Similar to Intel DDIO technology.
- Cache stash supports four types of transactions.
- WriteUniquePtlStash
- WriteUniqueFullStash
- StashOnceUnique
- StashOnceShared
DMT and DCT.
- In the CHI.A protocol, read data and snoop data must first be sent to the home node before being sent to the receiver node.
- Disadvantages: increased transmission length and latency
