Linux Concurrency and Race Conditions

---

Timeline

Timeline

2025-11-12

init

2026-05-21

add rwlock, rw_semaphore , rcu and per_cpu

This article introduces the basic concepts of concurrency and race conditions in the Linux kernel and their causes, including typical scenarios such as multi-threaded access, interrupt access, preemptive access, and multi-core concurrent access (SMP). The article points out that in a concurrent execution environment, multiple tasks accessing shared resources simultaneously may lead to abnormal execution or data errors, i.e., race condition problems. To address this issue, the Linux kernel provides various synchronization mechanisms such as atomic operations, spinlocks, mutexes, and semaphores. The article focuses on integer atomic operations in atomic operations, detailing atomic_t and atomic64_t definitions of two atomic variable types, and lists common atomic operation functions through a table, such as initialization, read, set, add/subtract, increment/decrement, fetch-and-add, and test result functions, including their prototypes, functions, parameters, and return values. These contents provide a basic reference for Linux driver developers to understand and solve concurrency and race condition problems.

---

Linux Driver Notes

Table of Contents	Links
1. Linux Driver Framework	Linux Driver Framework https://even629.com/posts/2511093/
2. Linux Driver Loading Logic	Linux Driver Loading Logic https://even629.com/posts/2511100/
3. Character Device Basics	Character Device Basics https://even629.com/posts/2511113/
4. Concurrency and Race Conditions	Concurrency and Race Conditions https://even629.com/posts/2511123/
5. Advanced Character Device Topics	Advanced Character Device Topics https://even629.com/posts/2511133/
6. Interrupts	interrupt https://even629.com/posts/2511143/
7. Platform Bus	Platform Bus https://even629.com/posts/2511153/
8. Device Tree	Device Tree https://even629.com/posts/2511300/
9. Device Model	Device Model https://even629.com/posts/2512013/
10. Hotplug	Hotplug https://even629.com/posts/2512023/
11. pinctrl Subsystem	pinctrl subsystem https://even629.com/posts/2512160/
12. GPIO subsystem	gpio subsystem https://even629.com/posts/2512173/
13. Input subsystem	Input subsystem https://even629.com/posts/2512183/
14. 1-Wire	1-Wire https://even629.com/posts/2512233/
15. I2C	I2C https://even629.com/posts/2512283/
16. SPI	SPI https://even629.com/posts/2512303/
17. UART	UART https://even629.com/posts/2512313/
18. PWM	PWM https://even629.com/posts/2601043/
19. RTC	RTC https://even629.com/posts/2601053/
20. Watchdog	Watchdog https://even629.com/posts/2601063/
21. CAN	CAN https://even629.com/posts/2601093/
22. Network devices	Network devices https://even629.com/posts/2601133/
23. ADC	ADC https://even629.com/posts/2601143/
24. IIO	IIO https://even629.com/posts/2602153
25. USB	USB https://even629.com/posts/2602163
26. LCD	LCD https://even629.com/posts/2602173

---

Concurrency and Race Condition Concepts

In the following, concurrency and parallelism are collectively referred to as concurrency.

In a concurrent execution environment, multiple programs may access the same shared resource simultaneously. When multiple tasks attempt to operate on such resources at the same time, abnormal execution or data errors may occur, and this type of problem is called a race condition.

Common causes of race conditions include:

1. Multi-threaded access. Since Linux is a multitasking operating system, multiple threads may access the same shared resource at the same time, which is the fundamental cause of race conditions.
2. Interrupt access. When a process is accessing a shared resource and an interrupt interrupts the executing process, a race condition may also occur between the process that issued the interrupt and the interrupted process.
3. Preemptive access. Linux 2.6 and later versions introduced a preemptive kernel, where high-priority tasks can interrupt low-priority tasks. If a task accessing a shared resource is interrupted and another task then accesses the same resource, a race condition may be triggered.
4. Multi-core concurrent access (SMP). In a multi-core processor system, different CPU cores may concurrently access the same shared resource, resulting in inter-core race conditions.

The Linux kernel provides various mechanisms to address this problem. Common methods includeAtomic operations、spinlock、Mutex lock、Semaphoreetc.

Atomic operations

Atomic operations treat a read or write operation on an integer variable as a whole, ensuring that it is indivisible, thereby avoiding race conditions.

Atomic operations can be further subdivided into “integer atomic operations” and “Bit atomic operation”, and here we first explain integer atomic operations.

Integer atomic operations

Used in the Linux kernel atomic_t(32-bit counter) and atomic64_t(64-bit counter) structures to define atomic variables; both types are available on 32-bit and 64-bit systems

include/linux/types.h

typedef struct {
	int counter;
} atomic_t;

#define ATOMIC_INIT(i) { (i) }

#ifdef CONFIG_64BIT
typedef struct {
	s64 counter;
} atomic64_t;
#endif

32-bit atomic_t

include/linux/atomic.h

Function prototype	Function description	Parameters	Return value
#define ATOMIC_INIT(i)	Initialize atomic_t variable toi	i: initial integer value	None
int atomic_read(const atomic_t *v)	Read the value of the atomic variable	v: atomic_t pointer	Returnvcurrent value of
void atomic_set(atomic_t *v, int i)	Set the atomic variable toi	v: variable pointer;i: value to write	None
void atomic_add(int i, atomic_t *v)	Atomicallyv += i	i: value to add;v: variable pointer	None
void atomic_sub(int i, atomic_t *v)	Atomicallyv -= i	i: value to subtract;v: variable pointer	None
void atomic_inc(atomic_t *v)	Atomicallyv++	v: variable pointer	None
void atomic_dec(atomic_t *v)	Atomicallyv--	v: variable pointer	None
int atomic_inc_return(atomic_t *v)	atomicv++and return the new value (increment first, then return)	v: variable pointer	incremented value
int atomic_dec_return(atomic_t *v)	atomicv--and return the new value (decrement first, then return)	v: variable pointer	decremented value
int atomic_sub_and_test(int i, atomic_t *v)	atomicv -= i, if the result is 0 returns true	i: the decrement value;v: variable pointer	if 0 → return 1, if not 0 → return 0
int atomic_dec_and_test(atomic_t *v)	atomicv--, if the result is 0 returns true	v: variable pointer	if 0 → return 1, if not 0 → return 0
int atomic_inc_and_test(atomic_t *v)	atomicv++, if the result is 0 returns true	v: variable pointer	if 0 → return 1, otherwise 0
int atomic_add_negative(int i, atomic_t *v)	atomicv += i, if the result < 0 returns true	i: value to add;v: variable pointer	If result < 0, return 1, otherwise 0

64-bit atomic64_t

Function prototype	Function description	Parameters	Return value
#define ATOMIC64_INIT(i)	Initialize atomic64_t toi	i: initial 64-bit integer	None
long long atomic64_read(const atomic64_t *v)	Read the value of the 64-bit atomic variable	v: variable pointer	Return the current 64-bit value
void atomic64_set(atomic64_t *v, long long i)	Set the atomic variable toi	v: variable pointer;i: 64-bit value to write	None
void atomic64_add(long long i, atomic64_t *v)	Atomically executev += i	i: value to add;v: variable pointer	None
void atomic64_sub(long long i, atomic64_t *v)	Atomically executev -= i	i: value to subtract;v: variable pointer	None
void atomic64_inc(atomic64_t *v)	Atomically executev++	v: variable pointer	None
void atomic64_dec(atomic64_t *v)	Atomically executev--	v: variable pointer	None
long long atomic64_add_return(long long i, atomic64_t *v)	Atomically executev += iand return the new value (increment first, then return)	i、v	Return the value after addition
long long atomic64_sub_return(long long i, atomic64_t *v)	Atomically executev -= iand return the new value (decrement first, then return)	i、v	Return the value after subtraction
long long atomic64_inc_return(atomic64_t *v)	atomicv++and return the new value	v	Return the value after increment
long long atomic64_dec_return(atomic64_t *v)	atomicv--and return the new value	v	Return the value after decrement
int atomic64_add_negative(long long i, atomic64_t *v)	atomicv += i, if the result < 0 returns true	i、v	result < 0 → 1, otherwise 0
int atomic64_add_and_test(long long i, atomic64_t *v)	atomicv += i, if the result == 0 returns true	i、v	result == 0 → 1, otherwise 0
int atomic64_sub_and_test(long long i, atomic64_t *v)	atomicv -= i, if the result == 0 returns true	i、v	result == 0 → 1, otherwise 0
int atomic64_inc_and_test(atomic64_t *v)	atomicv++, if the result == 0 returns true	v	if 0, then 1, otherwise 0
int atomic64_dec_and_test(atomic64_t *v)	atomicv--, if the result == 0 returns true	v	if 0, then 1, otherwise 0
long long atomic64_xchg(atomic64_t *v, long long i)	Atomic swap: setvset toiand return the old value	v、i	returns the old value before the swap
long long atomic64_cmpxchg(atomic64_t *v, long long old, long long new)	Atomic compare-and-swap: if *v == old, write new	v、old、new	returns the old value before the swap (can be used to determine success)

Incorrect usage

The following usage is incorrect:

static atomic_t atomic_key = ATOMIC_INIT(1);

int atomic_t_test_open(struct inode *inode, struct file *file)
{
        file->private_data = test_drv_data;
        if(atomic_read(&atomic_key) == 0){
                return -EBUSY;
        }
        atomic_dec(&atomic_key);

        return 0;
}

int atomic_t_test_release(struct inode *inode, struct file *file)
{
        atomic_inc(&atomic_key);
        return 0;
}

The reason isatomic_read(&atomic_key)andatomic_dec(&atomic_key)These two operations are separate, and two processes can simultaneouslyatomic_read(&atomic_key)thenatomic_dec(&atomic_key)succeed. These two operations can be combined into one

if (!atomic_dec_and_test(&atomic_key))
    return -EBUSY;

Example:

#include <linux/init.h>
#include <linux/module.h>
#include <linux/cdev.h>
#include <linux/slab.h>
#include <linux/atomic.h>

struct test_drv_data {
        dev_t dev_num;
        struct cdev cdev;
        struct class *class;
        struct device *dev;
};

static struct test_drv_data *test_drv_data;

static atomic_t atomic_key = ATOMIC_INIT(1);

int atomic_t_test_open(struct inode *inode, struct file *file)
{
        file->private_data = test_drv_data;
        if (atomic_dec_and_test(&atomic_key) == 0){
                pr_err("this device is opened by another process\n");
                return -EBUSY;
        }
        pr_info("atomic_t_test_open is called\n");
        return 0;
}

int atomic_t_test_release(struct inode *inode, struct file *file)
{
        atomic_inc(&atomic_key);
        return 0;
}

ssize_t atomic_t_test_read (struct file *file, char __user *buf, size_t size, loff_t *offset){
	pr_info("atomic_t_test_read is called\n");
	return 0;
}
ssize_t atomic_t_test_write (struct file *file, const char __user *buf, size_t size, loff_t *offset){
	pr_info("atomic_t_test_write is called\n");
	return 0;
}

struct file_operations fops = {
        .owner = THIS_MODULE,
        .open = atomic_t_test_open,
        .release = atomic_t_test_release,
};

static int __init atomic_t_test_init(void)
{
        int err;
        test_drv_data = (struct test_drv_data *)kzalloc(sizeof(struct test_drv_data), GFP_KERNEL);
        if (test_drv_data == NULL) {
                err = -ENOMEM;
                goto kzalloc_fail;
        }
        err = alloc_chrdev_region(&test_drv_data->dev_num, 0, 1, "atomic_t test chrdev region");
        if (err < 0)
                goto alloc_chrdev_region_fail;
        cdev_init(&test_drv_data->cdev, &fops);
        err = cdev_add(&test_drv_data->cdev, test_drv_data->dev_num, 1);
        if (err < 0)
                goto cdev_add_fail;
        test_drv_data->class = class_create(THIS_MODULE, "atomic_t_test");
        if (IS_ERR(test_drv_data->class)) {
                err = PTR_ERR(test_drv_data->class);
                goto class_create_fail;
        }
        test_drv_data->dev = device_create(test_drv_data->class, NULL, test_drv_data->dev_num, NULL,
                                           "atomic_t_test%d", 0);
        if (IS_ERR(test_drv_data->dev)) {
                err = PTR_ERR(test_drv_data->dev);
                goto device_create_fail;
        }

        return 0;
device_create_fail:
        class_destroy(test_drv_data->class);
class_create_fail:
        cdev_del(&test_drv_data->cdev);
cdev_add_fail:
        unregister_chrdev_region(test_drv_data->dev_num, 1);
alloc_chrdev_region_fail:
        kfree(test_drv_data);
kzalloc_fail:
        return err;
}

static void __exit atomic_t_test_exit(void)
{
        device_destroy(test_drv_data->class, test_drv_data->dev_num);
        class_destroy(test_drv_data->class);
        cdev_del(&test_drv_data->cdev);
        unregister_chrdev_region(test_drv_data->dev_num, 1);
        kfree(test_drv_data);
}

module_init(atomic_t_test_init);
module_exit(atomic_t_test_exit);

MODULE_LICENSE("GPL");
MODULE_AUTHOR("even629<asqwgo@163.com>");
MODULE_DESCRIPTION("This is a test sample for atomic_t");

Test code:

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <errno.h>
#include <string.h>
#include <fcntl.h>

int main(int argc, char **argv)
{
        pid_t pid;
        int fd;

        pid = fork();

        if (pid < 0) {
                goto fail;
        } else if (pid == 0) { // child process
                fd = open("/dev/atomic_t_test0", O_RDWR);
                if (fd < 0)
                        goto fail;

                printf("child open device success\n");
                // do something
                sleep(3);
                close(fd);

        } else { // parent process
                fd = open("/dev/atomic_t_test0", O_RDWR);
                if(fd < 0){
                        goto fail;
                }
                printf("parent open device success\n");
                // do something
                sleep(3);
                close(fd);
        }

        return 0;
fail:
        fprintf(stderr, "Error:%s[errno:%d]\n", strerror(errno), errno);
        exit(EXIT_FAILURE);
}

Test:

$ insmod atomic_t.ko
[   13.631548] atomic_t: loading out-of-tree module taints kernel.
$ ./test_atomic_t.o
[   20.232024] atomic_t_test_open is called
[   20.233129] this device is opened by another process
parent open device success
Error:Device or resource busy[errno:16]

atomic_cmpxchg

This function is somewhat difficult to understand; its purpose is:

• If the current value of the atomic variable equalsold
  • change it tonew
  • returns the “value before modification” (becausevthe value before modification equalsold, so the returned value equalsold)
• if the current value ≠old：
  • the atomic variable will not be modified
  • Directly return the ‘current value’

#include <linux/atomic.h>

long long atomic64_cmpxchg(atomic64_t *v, long long old, long long new);

Parameters

Parameters	Description
v	Pointer to theatomic64_tvariable’s pointer
old	Expected old value (expected value)
new	If the value of the atomic variable equalsold, then usenewreplace it

Return value

• Return The original value before the atomic variable operation。
• If the original value equalsold, replace the original value with new, return value = old.
• If the original value does not equalold, replacement fails, return value ≠ old.

Example

#include <linux/init.h>
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/moduleparam.h>
#include <linux/fs.h>
#include <linux/types.h>
#include <linux/cdev.h>
#include <linux/device.h>
#include <linux/uaccess.h>
#include <linux/err.h>
#include <linux/errno.h>
#include <linux/atomic.h>

struct device_test {
	// Device ID
	dev_t dev_id;
	// The class it belongs to
	struct class *class;
	// Device node under the class
	struct device *device;
	// cdev, character class device
	struct cdev cdev_test;
};

static struct device_test device1;

static atomic64_t cnt = ATOMIC64_INIT(1);

static int major_num;
static int minor_num;

module_param(major_num, int, S_IRUGO);
module_param(minor_num, int, S_IRUGO);

int cdev_test_open(struct inode *inode, struct file *file)
{
	if (atomic64_cmpxchg(&cnt, 1, 0) != 1) {
		return -EBUSY;
	}
	file->private_data = &device1;
	pr_info("cdev_test_open was called");
	return 0;
}

ssize_t cdev_test_read(struct file *file, char __user *buf, size_t size,
		       loff_t *off)
{
	//struct device_test *dev1 = (struct device_test *)file->private_data;

	pr_info("cdev_test_read was called");
	return 0;
}

ssize_t cdev_test_write(struct file *file, const char __user *buf, size_t size,
			loff_t *off)
{
	//struct device_test *dev1 = (struct device_test *)file->private_data;

	return 0;
}
int cdev_test_release(struct inode *inode, struct file *file)
{
	atomic64_set(&cnt, 1);
	pr_info("cdev_test_release was called");
	return 0;
}

static struct file_operations cdev_test_ops = { .owner = THIS_MODULE,
						.open = cdev_test_open,
						.read = cdev_test_read,
						.write = cdev_test_write,
						.release = cdev_test_release };

static int __init my_driver_init(void)
{
	int ret;

	// Allocate device number
	if (major_num) { //Driver parameters, statically allocate device number
		device1.dev_id = MKDEV(major_num, minor_num);

		pr_info("major from module_param: %d", MAJOR(device1.dev_id));
		pr_info("minor from module_param: %d", MINOR(device1.dev_id));

		ret = register_chrdev_region(device1.dev_id, 1,
					     "my_driver device");

		if (ret < 0) {
			pr_err("register_chrdev_region error\n");
			goto get_chrdev_region_err;
		}
		pr_info("register_chrdev_region ok\n");

	} else { //Dynamically allocate device number
		ret = alloc_chrdev_region(&device1.dev_id, 0, 1,
					  "my_driver device");
		if (ret < 0) {
			pr_err("alloc_chrdev_region error\n");
			goto get_chrdev_region_err;
		}
		pr_info("alloc_chrdev_region ok\n");

		pr_info("major allocated: %d", MAJOR(device1.dev_id));
		pr_info("minor allocated: %d", MINOR(device1.dev_id));
	}
	// cdev initialization, register character class device
	cdev_init(&device1.cdev_test, &cdev_test_ops);

	//Pointing the owner field to this module can prevent the module from being unloaded while its operations are in use.
	device1.cdev_test.owner = THIS_MODULE;
	// Add a character device
	ret = cdev_add(&device1.cdev_test, device1.dev_id, 1);

	if (ret < 0) {
		pr_err("cdev_add error\n");
		goto cdev_add_err;
	}
	pr_info("cdev_add ok\n");

	// Create a class
	device1.class = class_create(THIS_MODULE, "test");
	if (IS_ERR(device1.class)) {
		ret = PTR_ERR(device1.class);
		pr_err("class create error\n");
		goto class_create_err;
	}
	// Create a device node under the class, /dev/my_driver
	device1.device = device_create(device1.class, NULL, device1.dev_id,
				       NULL, "my_driver");
	if (IS_ERR(device1.device)) {
		ret = PTR_ERR(device1.device);
		pr_err("device create error\n");
		goto device_create_err;
	}
	pr_info("my_driver: Module loaded\n");

	return 0;

device_create_err:
	class_destroy(device1.class);
class_create_err:
	cdev_del(&device1.cdev_test);
cdev_add_err:
	unregister_chrdev_region(device1.dev_id, 1);
get_chrdev_region_err:
	return ret;
}

static void __exit my_driver_exit(void)
{
	// Delete device node
	device_destroy(device1.class, device1.dev_id);
	// Delete the class of this device node
	class_destroy(device1.class);
	// Delete cdev
	cdev_del(&device1.cdev_test);
	// Release device number
	unregister_chrdev_region(device1.dev_id, 1);

	pr_info("unregister_chrdev_region ok\n");
	pr_info("my_driver: Module unloaded\n");
}

module_init(my_driver_init);
module_exit(my_driver_exit);

MODULE_LICENSE("GPL");
MODULE_AUTHOR("Zhao Hang");
MODULE_DESCRIPTION("my_driver Kernel Module");

Bit atomic operation

Function prototype	Function description	Parameter description	Return value
void set_bit(int nr, void *p)	willpThe address’snrbit Set to 1	nr: bit number (0 = least significant bit)p: start address	None
void clear_bit(int nr, void *p)	willpThe address’snrbit clear	same as above	None
void change_bit(int nr, void *p)	willnrbit invert(0→1 or 1→0)	same as above	None
int test_bit(int nr, void *p)	Readnrvalue of the bit	same as above	returns the value of the bit: 0 or 1
int test_and_set_bit(int nr, void *p)	willnrsets the bit to 1 and returns the original value	same as above	returns the value of the bit before the operation (0 or 1)
int test_and_clear_bit(int nr, void *p)	willnrclears the bit and returns the original value	same as above	returns the value of the bit before the operation
int test_and_change_bit(int nr, void *p)	willnrtoggles the bit (1↔0) and returns the original value	same as above	returns the value of the bit before the operation

bit numbering starts from p points to the starting address of memorycounting from:

• nr = 0→ the least significant bit of the first byte (bit0)
• nr = 7→ the most significant bit of the first byte
• nr = 8→ bit 0 of the second byte
• and so on

Bit numbers are absolute bit indices, not offsets within a byte.

spinlock

A spinlock is a locking mechanism proposed to protect shared resources.When a thread attempts to acquire a spinlock and finds that the lock is already held by another thread, it does not go to sleep to wait, but instead continuously loops trying to acquire the lock until it succeeds.. This process is called “spinning”, hence the name “spinlock”.

Definition

include/linux/spinlock_types.h

typedef struct spinlock {
	union {
		struct raw_spinlock rlock;

#ifdef CONFIG_DEBUG_LOCK_ALLOC
# define LOCK_PADSIZE (offsetof(struct raw_spinlock, dep_map))
		struct {
			u8 __padding[LOCK_PADSIZE];
			struct lockdep_map dep_map;
		};
#endif
	};
} spinlock_t;

offsetof is defined ininclude/linux/stddef.h. It is used to compute the offset of a field within a structure.

#ifdef __compiler_offsetof
#define offsetof(TYPE, MEMBER)	__compiler_offsetof(TYPE, MEMBER)
#else
#define offsetof(TYPE, MEMBER)	((size_t)&((TYPE *)0)->MEMBER)
#endif

The aforementioned spinlock field uses offsetof to calculate and set the padding field, ensuring that the two union members’dep_mapaddresses are exactly the same, thus ensuring that regardless of whetherCONFIG_DEBUG_LOCK_ALLOC，spinlock_tthe size and alignment are consistent

There is also another purpose:

spinlock_tYes generic lock type

• do not want to exposeraw_spinlockthe internal structure
• butlockdepNeeddep_map

This design allows kernel code to be written like this:

lockdep_init_map(&lock->dep_map, ...);

rather than:

lock->rlock.dep_map   // ❌ Not desired

The spinlock-related API functions are defined in the kernel sourceinclude/linux/spinlock.hfile (the spinlock.h header file includes spinlock_types.h, etc., so you only need to include spinlock.h)

API

API / Macro	Return value type	Function description	Whether it blocks	Interrupt status	Return value / Remarks
DEFINE_SPINLOCK(name)	void	Statically define and initialize a spinlock	-	-	None
spin_lock_init(spinlock_t *lock)	void	Dynamically initialize a spinlock (only initializes, does not allocate memory)	-	-	None
spin_lock(spinlock_t *lock)	void	Acquire spinlock	Blocking (spinning)	Unchanged	None
spin_lock_bh(spinlock_t *lock)	void	Acquire spinlock and disable softirq	Blocking	Disable softirq	None
spin_lock_irq(spinlock_t *lock)	void	Acquire spinlock and disable local CPU interrupts	Blocking	Disable local interrupts	None
spin_lock_irqsave(spinlock_t *lock, unsigned long flags)	void	Acquire spinlock and save local interrupt state	Blocking	Disable interrupts, save flags	None
spin_trylock(spinlock_t *lock)	int	Try to acquire spinlock	Non-blocking	Unchanged	Returns 1 on success, 0 on failure
spin_trylock_bh(spinlock_t *lock)	int	Try to acquire spinlock and disable softirq	Non-blocking	Disable softirq	Returns 1 on success, 0 on failure
spin_trylock_irq(spinlock_t *lock)	int	Try to acquire spinlock and disable interrupts	Non-blocking	Disable local interrupts	Returns 1 on success, 0 on failure
spin_trylock_irqsave(spinlock_t *lock, unsigned long flags)	int	Try to acquire the spinlock and save interrupt state	Non-blocking	Disable interrupts, save flags	Returns 1 on success, 0 on failure
spin_unlock(spinlock_t *lock)	void	Release spinlock	-	-	None
spin_unlock_bh(spinlock_t *lock)	void	Release spinlock and restore softirq state	-	Restore softirq	None
spin_unlock_irq(spinlock_t *lock)	void	Release spinlock and restore interrupts	-	Restore interrupts	None
spin_unlock_irqrestore(spinlock_t *lock, unsigned long flags)	void	Release spinlock and restore interrupt state	-	Restore interrupt state flags	None
spin_is_locked(spinlock_t *lock)	int	Check whether the lock is held	Non-blocking	-	Returns 1 if locked, otherwise 0
spin_is_contended(spinlock_t *lock)	int	Check whether the lock is contended	Non-blocking	-	Returns 1 if contended, otherwise 0

Using a spinlock is divided into the following 3 steps:

1. Acquire the spinlock before accessing critical resources
2. After acquiring the spinlock, enter the critical section; if the spinlock cannot be acquired, wait in place.
3. Release the spinlock when exiting the critical section.

Example

#include <linux/init.h>
#include <linux/module.h>
#include <linux/cdev.h>
#include <linux/slab.h>
#include <linux/spinlock.h>
#include <linux/sched.h>

struct test_drv_data {
        dev_t dev_num;
        struct cdev cdev;
        struct class *class;
        struct device *dev;
};

static struct test_drv_data *test_drv_dat;
static bool status = 1;
static spinlock_t lock;

int spinlock_test_open(struct inode *inode, struct file *file)
{
        file->private_data = test_drv_dat;
        spin_lock(&lock);
        if(status == 0){
                spin_unlock(&lock);
                return -EBUSY;
        }
        status = 0;
        spin_unlock(&lock);
        pr_info("spinlock_test_open is called by [pid: %d]\n",task_pid_nr(current));

        return 0;
}
ssize_t spinlock_test_read(struct file *file, char __user *buf, size_t size, loff_t *offset)
{
        pr_info("spinlock_test_read is called\n");
        return 0;
}
ssize_t spinlock_test_write(struct file *file, const char __user *buf, size_t size, loff_t *offset)
{
        pr_info("spinlock_test_write is called\n");
        return 0;
}
int spinlock_test_release(struct inode *inode, struct file *file)
{
        spin_lock(&lock);
        status = 1;
        spin_unlock(&lock);
        pr_info("spinlock_test_release is called by [pid: %d]\n", task_pid_nr(current));
        return 0;
}
static struct file_operations fops = {
        .owner = THIS_MODULE,
        .open = spinlock_test_open,
        .read = spinlock_test_read,
        .write = spinlock_test_write,
        .release = spinlock_test_release,
};

static int __init spinlock_test_init(void)
{
        int err;
        test_drv_dat = kzalloc(sizeof(struct test_drv_data), GFP_KERNEL);
        if (test_drv_dat == NULL) {
                err = -ENOMEM;
                goto kzalloc_fail;
        }
        err = alloc_chrdev_region(&test_drv_dat->dev_num, 0, 1, "spinlock test chrdev region\n");

        if (err < 0)
                goto alloc_chrdev_region_fail;
        cdev_init(&test_drv_dat->cdev, &fops);
        test_drv_dat->cdev.owner = THIS_MODULE;
        err = cdev_add(&test_drv_dat->cdev, test_drv_dat->dev_num, 1);
        if (err < 0)
                goto cdev_add_fail;
        test_drv_dat->class = class_create(THIS_MODULE, "chrdev");
        if (IS_ERR(test_drv_dat->class)) {
                err = PTR_ERR(test_drv_dat->class);
                goto class_create_fail;
        }
        test_drv_dat->dev = device_create(test_drv_dat->class, NULL, test_drv_dat->dev_num, NULL,
                                          "spinlock_test%d", 0);
        if (IS_ERR(test_drv_dat->dev)) {
                err = PTR_ERR(test_drv_dat->dev);
                goto device_create_fail;
        }
        // Initialize spinlock
        spin_lock_init(&lock);

        return 0;

device_create_fail:
        class_destroy(test_drv_dat->class);
class_create_fail:
        cdev_del(&test_drv_dat->cdev);
cdev_add_fail:
        unregister_chrdev_region(test_drv_dat->dev_num, 1);
alloc_chrdev_region_fail:
        kfree(test_drv_dat);
kzalloc_fail:
        return err;
}

static void __exit spinlock_test_exit(void)
{
        device_destroy(test_drv_dat->class, test_drv_dat->dev_num);
        class_destroy(test_drv_dat->class);
        cdev_del(&test_drv_dat->cdev);
        unregister_chrdev_region(test_drv_dat->dev_num, 1);
        kfree(test_drv_dat);
}

module_init(spinlock_test_init);
module_exit(spinlock_test_exit);

MODULE_LICENSE("GPL");
MODULE_AUTHOR("even629<asqwgo@163.com>");
MODULE_DESCRIPTION("This is a test sample for spinlock");

Test:

#include <stdio.h>
#include <stdlib.h>
#include <errno.h>
#include <unistd.h>
#include <fcntl.h>
#include <string.h>

int main(int argc, char **argv)
{
        pid_t pid;
        int fd = -1;

        pid = fork();
        if (pid < 0) {
                perror("fork error");
                fprintf(stderr, "Error: %s[errno: %d]\n", strerror(errno), errno);
                exit(EXIT_FAILURE);
        } else if (pid == 0) { // child process
                printf("[child: <pid: %d>] try to open /dev/spinlock_test0\n",getpid());
                fd = open("/dev/spinlock_test0", O_RDWR);
                if (fd < 0) {
                        perror("[child] open error");
                        fprintf(stderr, "[child] Error: %s[errno: %d]\n", strerror(errno), errno);
                        exit(EXIT_FAILURE);
                }
                printf("[child: <pid: %d>] open /dev/spinlock_test0 success\n", getpid());
                sleep(2);
                close(fd);

        } else { // parent process
                printf("[parent: <pid: %d>] try to open /dev/spinlock_test0\n", getpid());
                fd = open("/dev/spinlock_test0", O_RDWR);
                if (fd < 0) {
                        perror("[parent] open error");
                        fprintf(stderr, "[parent] Error: %s[errno: %d]\n", strerror(errno), errno);
                        exit(EXIT_FAILURE);
                }
                sleep(2);
                printf("[parent: <pid: %d>] open /dev/spinlock_test0 success\n", getpid());

                close(fd);
        }

        return 0;
}

Test case:

$ insmod spinlock_test.ko
[   10.948417] spinlock_test: loading out-of-tree module taints kernel.
$ ./test_spinlock.o
[child: <pid: 102>] try to open /dev/spinlock_test0
[parent: <pid: 101>] try to open /dev/spinlock_test0
[   16.331206] spinlock_test_open is called by [pid: 102]
[child: <pid: 102>] open /dev/spinlock_test0 success
[parent] open error: Device or resource busy
[parent] Error: Device or resource busy[errno: 16]
$ [   18.335915] spinlock_test_release is called by [pid: 102]

Notes.

1. SinceA spinlock will “wait in place” because waiting in place continues to occupy the CPU and consumes CPU resources, so the lock cannot be held for too long. In other words, the code in the critical section cannot be too much.
2. In the critical section protected by a spinlock,functions that may cause a thread to sleep cannot be called; otherwise, a deadlock may occur.
3. Spinlocks are mainly used inmulticore (SMP) systems; in single-core preemptive kernels, spinlocks are also useful (in conjunction with disabling preemption).
4. Only one task can hold a mutex at a time; this is actually not a rule, but a fact.
5. Unlocking more than once is not allowed.
6. They must be initialized via the API.
7. A task holding a mutex cannot exit, because the mutex will remain locked, and possible contenders will wait forever (and sleep).
8. The locked memory region cannot be freed.
9. A held mutex must not be reinitialized.
10. Since they involve rescheduling, mutexes cannot be used in atomic contexts, such as tasklets and timers.

Spinlock deadlock

Spinlock deadlock refers to a phenomenon in which multiple tasks or processes wait for each other to release resources, causing none of them to be able to continue executing.

For example, suppose there are two processes, A and B. Process A holds a spinlock on resource 1 and also wants to acquire resource 2. Process B holds a spinlock on resource 2 and also wants to acquire resource 1. In this case, process A and process B are both waiting for each other to release resources, thus causing a deadlock.

Another example: when an interrupt occurs while process A holds a spinlock, the CPU turns to execute the interrupt handler, and the interrupt handler also needs to acquire the same spinlock. At this point, because the lock is already occupied by A, the interrupt handler can only wait by spinning, causing the system to enter a deadlock state.

• such asWhen the interrupt handler also needs to acquire the spinlock, the driver must disable interrupts while holding the spinlock (spin_lock_irqsave, spin_lock_irqrestore), and enable interrupts when releasing the spinlock.。

• At the same time, try to hold the spinlock for as short a time as possible; holding the spinlock for a long time may exhaust system resources and thus cause a deadlock.

• Finally, alsoavoid a function that has acquired a spinlock calling other functions that also attempt to acquire this lock, otherwise it will also cause a deadlock.

• In the critical section, no function that can cause sleep or blocking may be called.

Read-Write Lock

Definition

usespinlockWhen protecting a critical section, multiple reads cannot be concurrent; they can only bespin, in order to improve the overall performance of the system, the kernel defines a type of lock:

• allowing multiple processor processes (or threads or interrupt contexts)to perform read operations concurrently（SMPon), which is safe and improvesSMPthe performance of the system.
• When writing, it ensures complete mutual exclusion of the critical section.

read/write spinlockis to protectSMPshared data structures under the system, and its introduction is to increase the kernel’s concurrency capability. As long as the kernel control pathdoes notmodify the data structure, the read/write spinlock allows multiple kernel control pathsto readthe same data structure simultaneously.

If a kernel control path wants to perform a write operation on this structure, it must first acquire the write lock of the read/write lock,The write lock grants exclusive access to this resource. The purpose of this design is that allowing concurrent reads of the data structure can improve system performance.

Example

/*
 * example_rwlock.c
 */
#include <linux/module.h>
#include <linux/printk.h>
#include <linux/rwlock.h>

static DEFINE_RWLOCK(myrwlock);

static void example_read_lock(void)
{
    unsigned long flags;

    read_lock_irqsave(&myrwlock, flags);
    pr_info("Read Locked\n");

    /* Read from something */

    read_unlock_irqrestore(&myrwlock, flags);
    pr_info("Read Unlocked\n");
}

static void example_write_lock(void)
{
    unsigned long flags;

    write_lock_irqsave(&myrwlock, flags);
    pr_info("Write Locked\n");

    /* Write to something */

    write_unlock_irqrestore(&myrwlock, flags);
    pr_info("Write Unlocked\n");
}

static int __init example_rwlock_init(void)
{
    pr_info("example_rwlock started\n");

    example_read_lock();
    example_write_lock();

    return 0;
}

static void __exit example_rwlock_exit(void)
{
    pr_info("example_rwlock exit\n");
}

module_init(example_rwlock_init);
module_exit(example_rwlock_exit);

MODULE_DESCRIPTION("Read/Write locks example");
MODULE_LICENSE("GPL");

API

InitializationAPI：

function	Description
DEFINE_RWLOCK(rwlock_t lock)	Define and initialize the read/write lock
void rwlock_init(rwlock_t *lock)	Initialize the read/write lock

Read operationAPI ：

function	Description
void read_lock(rwlock_t *lock)	Acquire the read lock
void read_unlock(rwlock_t *lock)	Release the read lock
void read_lock_irq(rwlock_t *lock)	Disable local interrupts and acquire the read lock
void read_unlock_irq(rwlock_t *lock)	Enable local interrupts and release the read lock
void read_lock_irqsave(rwlock_t *lock, unsigned long flags)	Save the interrupt state, disable local interrupts, and acquire a read lock
void read_unlock_irqrestore(rwlock_t *lock, unsigned long flags)	Restore the interrupt state to its previous state, enable local interrupts, and release the read lock
void read_lock_bh(rwlock_t *lock)	Disable the bottom half, and acquire a read lock
void read_unlock_bh(rwlock_t *lock)	Enable the bottom half, and release the read lock

Write operationAPI ：

function	Description
void write_lock(rwlock_t *lock)	Acquire write lock
void write_unlock(rwlock_t *lock)	Release write lock
void write_lock_irq(rwlock_t *lock)	Disable local interrupts, and acquire a write lock
void write_unlock_irq(rwlock_t *lock)	Enable local interrupts, and release the write lock
void write_lock_irqsave(rwlock_t *lock, unsigned long flags)	Save the interrupt state, disable local interrupts, and acquire a write lock
void write_unlock_irqrestore(rwlock_t *lock, unsigned long flags)	Restore the interrupt state to its previous state, enable local interrupts, and release the read lock
void write_lock_bh(rwlock_t *lock)	Disable the bottom half, and acquire a read lock
void write_unlock_bh(rwlock_t *lock)	Enable the bottom half, and release the read lock

Semaphore

Spinlocks handle concurrency and contention by “busy waiting”, so the protected critical section cannot be too long, to avoid wasting CPU resources. However, in some cases we inevitablyneed to protect some resources for a long time. At this point, semaphores can be used.

Semaphores cause the caller to sleep, so semaphores are also called sleep locks.

Semaphores have P operation(down) and V operation(up). The P operation decrements the semaphore value by one. If the value after decrementing is less than 0, it means the resource is occupied, and the caller will be added to the waiting queue and blocked; if it is greater than or equal to 0, the resource is available, and the caller continues execution.

Definition

include/linux/semaphore.h

/* SPDX-License-Identifier: GPL-2.0-only */
/*
 * Copyright (c) 2008 Intel Corporation
 * Author: Matthew Wilcox <willy@linux.intel.com>
 *
 * Please see kernel/locking/semaphore.c for documentation of these functions
 */
#ifndef __LINUX_SEMAPHORE_H
#define __LINUX_SEMAPHORE_H

#include <linux/list.h>
#include <linux/spinlock.h>

/* Please don't access any members of this structure directly */
struct semaphore {
	raw_spinlock_t		lock;
	unsigned int		count;
	struct list_head	wait_list;
};

#define __SEMAPHORE_INITIALIZER(name, n)				\
{									\
	.lock		= __RAW_SPIN_LOCK_UNLOCKED((name).lock),	\
	.count		= n,						\
	.wait_list	= LIST_HEAD_INIT((name).wait_list),		\
}

#define DEFINE_SEMAPHORE(name)	\
	struct semaphore name = __SEMAPHORE_INITIALIZER(name, 1)

static inline void sema_init(struct semaphore *sem, int val)
{
	static struct lock_class_key __key;
	*sem = (struct semaphore) __SEMAPHORE_INITIALIZER(*sem, val);
	lockdep_init_map(&sem->lock.dep_map, "semaphore->lock", &__key, 0);
}

extern void down(struct semaphore *sem);
extern int __must_check down_interruptible(struct semaphore *sem);
extern int __must_check down_killable(struct semaphore *sem);
extern int __must_check down_trylock(struct semaphore *sem);
extern int __must_check down_timeout(struct semaphore *sem, long jiffies);
extern void up(struct semaphore *sem);

#endif /* __LINUX_SEMAPHORE_H */

API

API / Macro	Return value type	Description
DEFINE_SEMAPHORE(name)	N/A	Define and initialize a semaphore with an initial value of 1 (mutex semaphore).
sema_init(struct semaphore *sem, int val)	void	Initialize a semaphoresem, with an initial value ofval。
down(struct semaphore *sem)	void	Acquire the semaphore. If the count > 0, decrement it by 1 and return immediately; if the count is 0, block until it can be acquired.
down_interruptible(struct semaphore *sem)	int	Acquire the semaphore. If interrupted by a signal, return non-zero; otherwise block until acquisition succeeds and return 0.
down_killable(struct semaphore *sem)	int	Acquire the semaphore. If the process receives a fatal signal, return non-zero; otherwise block until acquisition succeeds and return 0.
down_trylock(struct semaphore *sem)	int	Try to acquire the semaphore immediately. Return 0 on success, otherwise return non-zero immediately without blocking.
down_timeout(struct semaphore *sem, long jiffies)	int	Try to acquire the semaphore, blocking for at mostjiffiesclock ticks. Return value 0 indicates success, non-zero indicates timeout or interruption by a signal.
up(struct semaphore *sem)	void	Release the semaphore (increment count by 1) and wake up tasks in the waiting queue.

Difference between down and down_interruptible:

Features	down(struct semaphore *sem)	down_interruptible(struct semaphore *sem)
Blocking behavior	If the semaphorecountis 0, the calling process willblock indefinitelyuntil the semaphore becomes available	If the semaphorecountis 0, the calling process will block, butcan be interrupted by a signal
Signal interruption	Does not respond to signals; interruption will not causedown()Return	Responds to signals; if a signal is received while blocked, thenreturn non-zero immediately
Return value	void(always succeeds, unless kernel bug)	int, 0 indicates acquisition success, non-zero indicates interrupted by a signal, acquisition failed
Use case	No need to respond to interrupts, ensure acquisition of the semaphore	Needs interruptible blocking, e.g., user processes can respond to Ctrl+C or other signals

example

#include <linux/init.h>
#include <linux/module.h>
#include <linux/cdev.h>
#include <linux/slab.h>
#include <linux/semaphore.h>
#include <linux/sched.h>

struct test_drv_data {
        dev_t dev_num;
        struct cdev cdev;
        struct class *class;
        struct device *dev;
};

static struct test_drv_data *drv_dat;
static struct semaphore sema;

int semaphore_test_open(struct inode *inode, struct file *file)
{
        int ret;
        file->private_data = drv_dat;
        ret = down_interruptible(&sema);
        if (ret != 0) {
                pr_info("semaphore_test_open is called by [pid: %d] but is interrupted\n",
                        current->pid);
                return -EINTR;
        }
        pr_info("semaphore_test_open is called by [pid: %d]\n", current->pid);
        return 0;
}

int semaphore_test_release(struct inode *inode, struct file *file)
{
        up(&sema);
        pr_info("semaphore_test_release is called by [pid: %d]\n", current->pid);
        return 0;
}
struct file_operations fops = {
        .owner = THIS_MODULE,
        .open = semaphore_test_open,
        .release = semaphore_test_release,
};

static int __init semaphore_test_init(void)
{
        int err;

        drv_dat = (struct test_drv_data *)kzalloc(sizeof(struct test_drv_data), GFP_KERNEL);
        if (drv_dat == NULL) {
                err = -ENOMEM;
                goto kzalloc_fail;
        }

        err = alloc_chrdev_region(&drv_dat->dev_num, 0, 1, "semaphore_test_chrdev_region");
        if (err < 0)
                goto alloc_chrdev_region_fail;
        cdev_init(&drv_dat->cdev, &fops);
        drv_dat->cdev.owner = THIS_MODULE;
        err = cdev_add(&drv_dat->cdev, drv_dat->dev_num, 1);
        if (err < 0)
                goto cdev_add_fail;
        drv_dat->class = class_create(THIS_MODULE, "chrdev_test");
        if (IS_ERR(drv_dat->class)) {
                err = PTR_ERR(drv_dat->class);
                goto class_create_fail;
        }

        drv_dat->dev =
                device_create(drv_dat->class, NULL, drv_dat->dev_num, NULL, "semaphore_test%d", 0);
        if (IS_ERR(drv_dat->dev)) {
                err = PTR_ERR(drv_dat->dev);
                goto device_create_fail;
        }
        // Initialize the semaphore
        sema_init(&sema, 1);

        return 0;
device_create_fail:
        class_destroy(drv_dat->class);
class_create_fail:
        cdev_del(&drv_dat->cdev);
cdev_add_fail:
        unregister_chrdev_region(drv_dat->dev_num, 1);
alloc_chrdev_region_fail:
        kfree(drv_dat);
kzalloc_fail:
        return err;
}

static void __exit semaphore_test_exit(void)
{
        device_destroy(drv_dat->class, drv_dat->dev_num);
        class_destroy(drv_dat->class);
        cdev_del(&drv_dat->cdev);
        unregister_chrdev_region(drv_dat->dev_num, 1);
        kfree(drv_dat);
}

module_init(semaphore_test_init);
module_exit(semaphore_test_exit);

MODULE_LICENSE("GPL");
MODULE_AUTHOR("even629<asqwgo@163.com>");
MODULE_DESCRIPTION("This is test sample for semaphore");

Test case:

#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
#include <unistd.h>

int main(int argc, char **argv)
{
        pid_t pid;
        int fd;

        pid = fork();
        if (pid < 0) {
                perror("fork error");
                exit(EXIT_FAILURE);
        } else if (pid == 0) { // child
                fd = open("/dev/semaphore_test0", O_RDWR);
                if (fd < 0) {
                        perror("child open /dev/semaphore_test0 error");
                        exit(EXIT_FAILURE);
                }
                printf("[child: pid<%d>] open /dev/semaphore_test0 success\n", getpid());
                sleep(2);
                close(fd);
        } else { // parent
                fd = open("/dev/semaphore_test0", O_RDWR);
                if (fd < 0) {
                        perror("parent open /dev/semaphore_test0 error");
                        exit(EXIT_FAILURE);
                }
                printf("[parent: pid<%d>] open /dev/semaphore_test0 success\n", getpid());
                sleep(2);
                close(fd);
        }

        return 0;
}

$ insmod semaphore_test.ko
[   15.981179] semaphore_test: loading out-of-tree module taints kernel.
$ ./test_semaphore.o
[   20.535287] semaphore_test_open is called by [pid: 101]
[parent: pid<101>] open /dev/semaphore_test0 success
[   22.544668] semaphore_test_release is called by [pid: 101]
[   22.545409] semaphore_test_open is called by [pid: 102]
[child: pid<102>] open /dev/semaphore_test0 success
[   24.550326] semaphore_test_release is called by [pid: 102]

Note

1. The value of a semaphore cannot be less than 0.
2. When accessing a shared resource, the semaphore performs a “decrement by one” operation, and after the access is completed, it performs an “increment by one” operation.
3. **When the semaphore value is 0, threads that want to access the shared resource must wait.**When the semaphore is greater than 0, the waiting threads can access.
4. BecauseSemaphores can cause sleeping, so semaphores cannot be used in interrupts.
5. If the shared resource is held for a relatively long time, semaphores are generally used instead of spinlocks.
6. When using both semaphores and spinlocks, you must acquire the spinlock first, then acquire the semaphore. This is because semaphores can cause sleeping, and you cannot sleep while holding a spinlock.

Read-write semaphore

The difference from a read-write lock is that a read-write semaphore is a sleeping lock, while a read-write lock is a spinlock. Selection:

Can the critical section sleep?
├─ Yes → Use a read-write semaphore (rw_semaphore) or a mutex (mutex)
│ └─ Are read operations far more numerous than write operations? → Yes → Read-write semaphore
│ └─ Are read operations not significantly more numerous than write operations? → Mutex is simpler and has better performance
│
└─ No → Is the critical section very short (a few integer operations)?
├─ Yes → A normal spinlock (spinlock_t) may be better than a read-write spinlock
└─ No → Are reads far more numerous than writes? → Yes → Read-write spinlock (rwlock_t)
└─ No → Normal spinlock

Functions that may cause sleeping cannot be used in interrupt context:

**Because interrupt context does not belong to any process, it has no process control block (task_struct), so it fundamentally lacks the infrastructure required for “sleeping.”**When a process is about to enter the sleeping state, it actively marks itself as “sleeping,” removes itself from the run queue, and tells the scheduler: “Please wake me up when the lock is available.” This operation depends on the process context.

API

API function definitions	Description
DECLARE_RWSEM(name)	Declare a read-write semaphore named name and initialize it.
void init_rwsem(struct rw_semaphore *sem);	Initialize the read-write semaphore sem.
void down_read(struct rw_semaphore *sem);	Readers use it to acquire sem; if not acquired, the caller sleeps and waits.
void up_read(struct rw_semaphore *sem);	Reader releases sem.
int down_read_trylock(struct rw_semaphore *sem);	Reader tries to acquire sem, returns 1 if acquired, returns 0 if not. Can be used in interrupt context.
void down_write(struct rw_semaphore *sem);	Writer uses to acquire sem; if not acquired, the caller sleeps waiting.
int down_write_trylock(struct rw_semaphore *sem);	Writer tries to acquire sem, returns 1 if acquired, returns 0 if not. Can be used in interrupt context.
void up_write(struct rw_semaphore *sem);	Writer releases sem.
void downgrade_write(struct rw_semaphore *sem);	Downgrade the writer to a reader.

example

Initialization:

#include <linux/rwsem.h>

/* Statically declare and initialize */
DECLARE_RWSEM(my_rwsem);

/* Dynamic initialization (structure member) */
struct rw_semaphore my_rwsem;
init_rwsem(&my_rwsem);

Using read-write lock:

/* Reader: read configuration and copy to user space */
ssize_t my_read(struct file *filp, char __user *buf, size_t size, loff_t *off)
{
    int ret;
    
    down_read(&my_rwsem);                    /* Acquire read lock, can sleep */
    ret = copy_to_user(buf, my_data, size);  /* Allowed, because holding a semaphore that may sleep */
    up_read(&my_rwsem);                      /* Release the read lock */
    
    return ret;
}

/* Writer: update configuration */
ssize_t my_write(struct file *filp, const char __user *buf, size_t size, loff_t *off)
{
    down_write(&my_rwsem);                   /* Acquire write lock, exclusive */
    copy_from_user(my_data, buf, size);      /* Sleep allowed */
    up_write(&my_rwsem);                     /* Release write lock */
    
    return size;
}

Mutex lock

Only one visitor can access the same resource at the same timeOther visitors can only access the resource after the current one finishes. This is mutual exclusion.

Mutex is very similar to a semaphore with a count of 1but mutex is simpler and more efficient. However, there are more things to be careful about.

Definition

include/linux/mutex.h

struct mutex {
	atomic_long_t		owner;
	spinlock_t		wait_lock;
#ifdef CONFIG_MUTEX_SPIN_ON_OWNER
	struct optimistic_spin_queue osq; /* Spinner MCS lock */
#endif
	struct list_head	wait_list;
#ifdef CONFIG_DEBUG_MUTEXES
	void			*magic;
#endif
#ifdef CONFIG_DEBUG_LOCK_ALLOC
	struct lockdep_map	dep_map;
#endif
};

API

Function/Macro	Return value type	Description
DEFINE_MUTEX(name)	struct mutex	Statically define and initialize a mutex, initial state is unlocked
mutex_init(struct mutex *lock)	void	Initialize mutex to unlocked state
mutex_destroy(struct mutex *lock)	void	Destroy mutex (only actually works under DEBUG_MUTEXES)
mutex_lock(struct mutex *lock)	void	Acquire the mutex, blocking until the lock is available if it is held.
mutex_lock_interruptible(struct mutex *lock)	int	Acquire the mutex, blocking if the lock is held; interruptible by signals. Returns 0 on success, -EINTR if interrupted by a signal.
mutex_lock_killable(struct mutex *lock)	int	Acquire the mutex, blocking if the lock is held; interruptible by fatal signals. Return value is the same as above.
mutex_lock_io(struct mutex *lock)	void	Acquire the mutex, allowed in I/O context (uncommon).
mutex_lock_nested(struct mutex *lock, unsigned int subclass)	void	Nested lock acquisition, used for Lockdep analysis, actually equivalent tomutex_lock()
mutex_lock_interruptible_nested(struct mutex *lock, unsigned int subclass)	int	Interruptible nested lock acquisition, used for Lockdep analysis.
mutex_lock_killable_nested(struct mutex *lock, unsigned int subclass)	int	Nested lock acquisition interruptible by fatal signals, used for Lockdep analysis.
mutex_lock_nest_lock(struct mutex *lock, struct lockdep_map *nest_lock)	void	Nested lock acquisition, used for Lockdep analysis.
mutex_trylock(struct mutex *lock)	int	Try to acquire the mutex without blocking; returns 1 on success, 0 on failure.
mutex_trylock_recursive(struct mutex *lock)	enum mutex_trylock_recursive_enum	Try to acquire the mutex, allowing recursion; returns 0/1/recursion flag.
mutex_unlock(struct mutex *lock)	void	Release the mutex; must be called by the thread holding the lock.
atomic_dec_and_mutex_lock(atomic_t *cnt, struct mutex *lock)	int	Decrement the atomic counter by 1; if it reaches 0, acquire the mutex; otherwise return 0.
mutex_is_locked(struct mutex *lock)	bool	Query whether the mutex is held; true means the lock is held, false means it is not held.

Like the interruptible series of wait queue functions, it is recommended to usemutex_lock_interruptible(), which makes the driver interruptible by all signals, whereas formutex_lock_killable(), only signals that kill the process can interrupt the driver.

Callmutex_lock()you must be very careful; it can only be used if you can guarantee that the mutex will be released under all circumstances. In user context, it is recommended to always usemutex_lock_interruptible()to acquire the mutex, becausemutex_lock()it will not return even if a signal is received (even Ctrl+C).

example

#include <linux/init.h>
#include <linux/module.h>
#include <linux/slab.h>
#include <linux/cdev.h>
#include <linux/mutex.h>
#include <linux/sched.h>

struct test_drv_data {
        dev_t dev_num;
        struct cdev cdev;
        struct class *class;
        struct device *dev;
};

static struct test_drv_data *drv_dat;
static struct mutex drv_mutex;

int mutex_test_open(struct inode *inode, struct file *file)
{
        int err;
        file->private_data = drv_dat;
        err = mutex_lock_interruptible(&drv_mutex);
        if (err != 0) {
                pr_err("pid: %d is interrupted", current->pid);
                return err;
        }
        pr_info("mutex_test_open is called by pid: %d\n", current->pid);
        return 0;
}

int mutex_test_release(struct inode *inode, struct file *file)
{
        mutex_unlock(&drv_mutex);
        pr_info("mutex_test_release is called by pid: %d\n", current->pid);
        return 0;
}

static struct file_operations fops = {
        .owner = THIS_MODULE,
        .open = mutex_test_open,
        .release = mutex_test_release,
};

static int __init mutex_test_init(void)
{
        int err;
        drv_dat = (struct test_drv_data *)kzalloc(sizeof(struct test_drv_data), GFP_KERNEL);

        if (drv_dat == NULL) {
                err = -ENOMEM;
                goto kzalloc_fail;
        }

        err = alloc_chrdev_region(&drv_dat->dev_num, 0, 1, "mutex_test_chrdev_region");
        if (err < 0)
                goto alloc_chrdev_region_fail;

        cdev_init(&drv_dat->cdev, &fops);
        drv_dat->cdev.owner = THIS_MODULE;
        err = cdev_add(&drv_dat->cdev, drv_dat->dev_num, 1);
        if (err < 0)
                goto cdev_add_fail;

        drv_dat->class = class_create(THIS_MODULE, "chrdev_test");
        if (IS_ERR(drv_dat->class)) {
                err = PTR_ERR(drv_dat->class);
                goto class_create_fail;
        }

        drv_dat->dev =
                device_create(drv_dat->class, NULL, drv_dat->dev_num, NULL, "mutex_test%d", 0);
        if (IS_ERR(drv_dat->dev)) {
                err = PTR_ERR(drv_dat->dev);
                goto device_create_fail;
        }
        // Initialize the mutex lock
        mutex_init(&drv_mutex);
        return 0;
device_create_fail:
        class_destroy(drv_dat->class);
class_create_fail:
        cdev_del(&drv_dat->cdev);
cdev_add_fail:
        unregister_chrdev_region(drv_dat->dev_num, 1);
alloc_chrdev_region_fail:
        kfree(drv_dat);
kzalloc_fail:
        return err;
}

static void __exit mutex_test_exit(void)
{
        kfree(drv_dat);
}

module_init(mutex_test_init);
module_exit(mutex_test_exit);

MODULE_LICENSE("GPL");
MODULE_AUTHOR("even629<asqwgo@163.com>");
MODULE_DESCRIPTION("This is a test sample for mutex");

Test case:

#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
#include <unistd.h>

int main(int argc, char **argv)
{
        pid_t pid;
        int fd;
        char prefix[16];        

        pid = fork();
        if (pid < 0) {
                perror("fork error");
                exit(EXIT_FAILURE);
        } else if (pid == 0) {
                snprintf(prefix, sizeof(prefix), "[pid: %d]", getpid());

                fd = open("/dev/mutex_test0", O_RDWR);
                if (fd < 0) {
                        perror(prefix);
                        exit(EXIT_FAILURE);
                }
                printf("%s child open success\n", prefix);
                sleep(2);
                close(fd);

        } else {
                snprintf(prefix, sizeof(prefix), "[pid: %d]", getpid());

                fd = open("/dev/mutex_test0", O_RDWR);
                if (fd < 0) {
                        perror(prefix);
                        exit(EXIT_FAILURE);
                }
                printf("%s parent open success\n", prefix);
                sleep(2);
                close(fd);
        }

        return 0;
}

$ insmod mutex_test.ko
[   13.589343] mutex_test: loading out-of-tree module taints kernel.
$ ./test_mutex.o
[   17.141253] mutex_test_open is called by pid: 101
[pid: 101] parent open success
[   19.150144] mutex_test_release is called by pid: 101
[   19.151379] mutex_test_open is called by pid: 102
[pid: 102] child open success
[   21.156199] mutex_test_release is called by pid: 102

Notes.

1. Mutexes cause sleeping, therefore inmutexes cannot be used in interrupts
2. Only one thread can hold the mutex at a time, and only the holder can unlock it.
3. Recursive locking and unlocking are not allowed.

RCU

RCU（Read-Copy-Update, Read-Copy Update) as a high-performance concurrency control mechanism, specifically designed to solve the “read-heavy, write-light” scenario. By separating the synchronization logic of read and write operations, it makesread operations have almost no overhead(no locking, no atomic instructions, no blocking), while write operations ensure safety through “copy-then-update” and “deferred release”.

RCU has become an indispensable basic component in the Linux kernel, widely used in core modules such as process scheduling, memory management, and file systems.

RCU’s applicable scenariosIt mainly has the following characteristics:

1. Data structuremainly accessed through pointers;
2. read operations far outnumber update operations;
3. read-side code cannot tolerate lock overhead;
4. update operations are relatively infrequent;

The core idea of RCU can be summarized as**“read directly when reading, copy and modify when writing”**

Take a life scenario: There is a bulletin board at home that records the family members’ schedules. When the mother needs to update the schedule, she does not prevent others from viewing the bulletin board. Instead, she first copies the current content onto a new piece of paper, modifies it on the new paper, and then, at a moment when no one will notice, replaces the new paper onto the bulletin board. After the replacement, she does not immediately destroy the old paper, but waits for a period of time to ensure that everyone who might be looking at the old bulletin board has finished reading it, and only then discards the old paper.

In this metaphor, “replacement moment” corresponds to thepublishing (Publishing) operation, “waiting to ensure” Correspondinggrace period (Grace Period), and**“discarding the old paper”** then corresponds to**reclamation (Reclamation)**These three concepts form the core triangle of RCU.

Terminology

• Read-Side Critical Section: The code segment where read operations access RCU-protected data, which needs to be marked withrcu_read_lock()andrcu_read_unlock()marker. Within this interval, read operations can safely access old data (even if write operations have updated the data).

• Grace Period: The time period from when a write operation starts updating data until all read-side critical sections (for old data) end. The end of the grace period means “all read operations that might have accessed the old data have completed”, at which point the old data can be safely released.

• Write-Side Operation: Consists of three steps:

  1. Copy: Create a copy of the data (or new data).
  2. Update: Atomically switch the pointer from the old data to the new data.
  3. Deferred Free: After waiting for the grace period to end, free the old data.

How RCU Works

Read-Side

The lock-free access of read-side operations is the core source of RCU’s performance advantage. The process is as follows:

1. Through thercu_read_lock()Enter the read-side critical section (essentially disabling preemption, or tracking read-side state in Preemptible RCU).
2. usercu_dereference(p)Obtain a pointer to the shared data (including memory barriers to ensure atomicity and visibility of pointer reads).
3. Access the data pointed to by the pointer (read directly without locking).
4. Through thercu_read_unlock()Exit the read-side critical section.

rcu_read_lock();
/* Read protected data */
data = rcu_dereference(global_pointer);
/* Use data */
rcu_read_unlock();

Herercu_read_lock()andrcu_read_unlock()They do not perform actual lock and unlock operations like traditional locks. In non-preemptive kernels, they may be complete no-ops. Their real purpose isto tell the compiler not to reorder instructions, and to mark the beginning and end of critical sections in preemptible kernels

rcu_dereference()is a key macro that ensures read operations proceed in the expected order on weak-memory-ordering processors such as Alpha. It can be seen as a wrapper for memory barriers, ensuring that the pointer read occurs before fetching the actual data.

The key characteristic of the read side iszero overhead - no atomic operations, no memory barriers (on most architectures), no lock contention. This makes the RCU read path extremely efficient, and performance remains stable even under high-concurrency read scenarios.

Write-Side

The write-side operation must ensure that ‘old data is freed after all read operations complete.’ The process is as follows:

1. Copy/create new data: for example, allocate new memory for a linked-list node and initialize it.
2. Atomically update the pointer: Usercu_assign_pointer(p, new_ptr)Switch the shared pointer from old data to new data (including a memory barrier to ensure the new pointer is visible to all CPUs).
3. Wait for the grace period: viasynchronize_rcu()(blocking wait) orcall_rcu()(asynchronous callback) wait for the grace period to end.
4. Free the old data: after the grace period ends, safely free the old data (e.g.,kfree(old_ptr)）。

Example:

// Write-side operation: update global_ptr
struct data *new_data = kmalloc(sizeof(*new_data), GFP_KERNEL);
if (!new_data)
   	return -ENOMEM;
// Initialize new data...
new_data->value = new_value;

// Publish operation, atomically update the pointer, visible to the read side
rcu_assign_pointer(global_ptr, new_data);

// Wait for the grace period to end (all read operations on old data have completed)
synchronize_rcu();

// Free the old data
kfree(old_data);

Grace Period

How is safe freeing guaranteed?

The grace period is the core mechanism of RCU; its purpose is**Ensure that all read-side critical sections that started before the pointer update have completed.**How does RCU detect the end of a grace period?

• Core idea: Each CPU maintains an “RCU state” (such asrcu_seq), which records whether there is currently an active read-side critical section. When a write operation requests a grace period, RCU waits for all CPUs to experience a “context switch” (such as scheduling, interrupt return, etc.) — this means that the read-side critical section on that CPU has ended (because read-side critical sections disable preemption and cannot be interrupted by scheduling; if a context switch occurs, it indicates that the read-side section has ended).
• Implementation details: The Linux kernel usesrcu_nodea tree structure to track the RCU state of each CPU,synchronize_rcu()and blocks until all CPUs have completed a “grace period acknowledgment” (rcu_qs()）。

A grace period can last up to hundreds of milliseconds (depending on system load); the write side should not assume it will end quickly, and should avoid designing logic that depends on the duration of the grace period.

The Linux kernel usescontext switches、user-mode executionandidle loopsas indicators of read-side critical section exit. After a CPU has experienced one of these states, the kernel considers all RCU read-side critical sections on that CPU to have completed.

The implementation of RCU relies heavily onmemory barriersto ensure ordering consistency in multi-core environments, preventing compilers and CPUs from reordering instructions for optimization.

• On the read side,rcu_dereference()it includes a read memory barrier, ensuring that operations after the pointer fetch are not reordered before the pointer fetch.

• On the update side,rcu_assign_pointer()it includes a write memory barrier, ensuring that the pointer update itself is not visible to other CPUs until all store operations preceding the pointer update are visible to other CPUs.

This ordering guarantee is crucial to the correctness of RCU. Consider the data publication scenario: the updater must first initialize the new data, then publish the pointer. Without memory barriers, the CPU or compiler might reorder these two steps, causing readers to see not-fully-initialized data.

Grace periods extend to contain pre-existing RCU read-side critical sections.

RCU synchronization primitives

Read-Side

void rcu_read_lock(void);

void rcu_read_unlock(void);

The above two APIs are used to mark the beginning and end of a read-side critical section, ensuring that read operations can safely access old data within this interval.

• In Classic RCU (non-preemptible RCU),rcu_read_lock()Disable preemption (preempt_disable()），rcu_read_unlock()Enable preemption (preempt_enable()）。
• In Preemptible RCU (preemptible RCU), byrcu_read_lock_bh()disabling softirqs, or usingrcu_read_lock()in conjunction with the kernel preemption count.

you can usercu_dereferenceto dereference

typeof(p) rcu_dereference(p);

for safe read-side access to RCU-protected pointers, ensuring:

• Atomicity of pointer reads (preventing partial pointer loads caused by compiler optimizations).
• Memory barriers (ensuring new data is visible to the read side, avoiding CPU cache inconsistency).

Example:

// Read-side operation: accessing RCU-protected pointers
rcu_read_lock();
struct data *d = rcu_dereference(global_ptr); // Safely acquire the pointer
if (d)
   	printk("value: %d\n", d->value); // Access the data
rcu_read_unlock();

Write-Side

rcu_assign_pointer

void rcu_assign_pointer(p, typeof(p) v);

Write-side atomic update of RCU-protected pointers, ensuring:

• Atomicity of pointer updates.
• Memory barriers (ensuring new data is fully initialized before becoming visible to the read side).

synchronize_rcu()

• Function: blocks waiting for the grace period to end; after returning, old data can be safely freed.

• Note: cannot be used in atomic contexts (e.g., interrupt handlers) (will cause scheduling failure).

• example：

  // Deleting an RCU-protected linked list node
  list_del_rcu(node); // Remove the node from the linked list (RCU-safe version)
  synchronize_rcu();  // Wait for the grace period to end
  kfree(node);        // Free the old node

synchronize_rcu()It blocks waiting for the grace period; atomic contexts (such as interrupts, softirqs) cannot schedule, causing a kernel crash. In this case, you should usecall_rcu()。

call_rcu()

• Function: asynchronously waits for the grace period; after the grace period ends, the specified callback function is called to free the old data (non-blocking).

• Prototype：void call_rcu(struct rcu_head *head, void (*func)(struct rcu_head *head));

• example：

  // Define the callback function (to free data)
  void free_data(struct rcu_head *head) {
  	struct data *d = container_of(head, struct data, rcu);
  	kfree(d);
  } 
  
  // Write-side operation: update data and asynchronously free old data
  struct data *new_data = kmalloc(...);
  struct data *old_data = rcu_dereference_protected(global_ptr, ...);
  rcu_assign_pointer(global_ptr, new_data); // Update pointer
  call_rcu(&old_data->rcu, free_data);      // Call free_data after the grace period

• Read side must usercu_dereference(): Do not directly access the pointer (e.g.d = global_ptr), otherwise it may cause data inconsistency due to compiler optimization or CPU reordering.
• Write side must usercu_assign_pointer(): Do not directly assign (e.g.global_ptr = new_d), ensure the new pointer is visible to all CPUs.

RCU usage example

RCU-protected pointer update

Scenario: global pointerglobal_dataPoints to a structure; read operations access it frequently, write operations update it occasionally.

#include <linux/rcupdate.h>
#include <linux/sched.h>
 
// Define a shared data structure (containing an RCU head for call_rcu)
struct my_data {
    int value;
    struct rcu_head rcu; // Must be included for call_rcu callback
};
 
// RCU-protected global pointer
static struct my_data __rcu *global_data;
 
// Read-side function: access data
void read_data(void) {
    struct my_data *d;
 
    rcu_read_lock();
    d = rcu_dereference(global_data); // Safely acquire the pointer
    if (d)
        printk("Read value: %d\n", d->value);
    rcu_read_unlock();
}
 
// Write-side function: update data (use call_rcu for asynchronous release)
void update_data(int new_val) {
    struct my_data *new_d, *old_d;
 
    new_d = kmalloc(sizeof(*new_d), GFP_KERNEL);
    if (!new_d)
        return;
    new_d->value = new_val;
 
    // Atomically update the pointer (visible to the read side)
    old_d = rcu_dereference_protected(global_data, 1); // Write side safely acquires the old pointer
    rcu_assign_pointer(global_data, new_d);
 
    // Free old data after the grace period
    if (old_d)
        call_rcu(&old_d->rcu, (void (*)(struct rcu_head *))kfree);
}

RCU-protected linked list operations

RCU is often used to protect dynamic linked lists (such as process lists and network connection tables). The kernel provideslist_add_rcu()、list_del_rcu()and other RCU-safe linked list operation macros.

Scenario: maintain an RCU-protected doubly linked list; the read side traverses the list, and the write side adds/removes nodes.

#include <linux/list.h>
#include <linux/rculist.h> // RCU linked list macros
 
// Linked list node structure
struct my_node {
    int id;
    struct list_head list;
    struct rcu_head rcu;
};
 
// RCU-protected list head
static LIST_HEAD(my_list);
static DEFINE_SPINLOCK(list_lock); // Writer-side mutex (multiple writers need synchronization)
 
// Reader side: traverse the list (lock-free)
void traverse_list(void) {
    struct my_node *node;
 
    rcu_read_lock();
    // list_for_each_entry_rcu: RCU-safe list traversal macro
    list_for_each_entry_rcu(node, &my_list, list) {
        printk("Node id: %d\n", node->id);
    }
    rcu_read_unlock();
}
 
// Writer side: add node
void add_node(int id) {
    struct my_node *node = kmalloc(sizeof(*node), GFP_KERNEL);
    if (!node)
        return;
    node->id = id;
    INIT_LIST_HEAD(&node->list);
 
    // Writer side needs locking (to prevent multiple writers from modifying the list simultaneously)
    spin_lock(&list_lock);
    list_add_rcu(&node->list, &my_list); // RCU-safe addition
    spin_unlock(&list_lock);
}
 
// Writer side: delete node
void delete_node(int id) {
    struct my_node *node, *tmp;
 
    spin_lock(&list_lock);
    list_for_each_entry_safe(node, tmp, &my_list, list) {
        if (node->id == id) {
            list_del_rcu(&node->list); // RCU-safe deletion
            spin_unlock(&list_lock);
            // Wait for grace period before releasing the node
            call_rcu(&node->rcu, (void (*)(struct rcu_head *))kfree);
            return;
        }
    }
    spin_unlock(&list_lock);
}

RCU best practices

Minimize read-side critical section length

The longer the read-side critical section, the longer the grace period may be (writers have to wait longer), causing delayed release of old data and increasing memory pressure.

Principle: Only enter the critical section when accessing RCU data, and avoid time-consuming operations (such as I/O, complex calculations) inside the critical section.

// Bad example: critical section contains time-consuming operation
rcu_read_lock();
d = rcu_dereference(global_data);
if (d) {
    heavy_computation(); // Time-consuming operation, should not be inside the critical section
    printk("%d\n", d->value);
}
rcu_read_unlock();
 
// Good example: shrink the critical section
rcu_read_lock();
d = rcu_dereference(global_data);
if (d)
    value = d->value; // Only read necessary data
rcu_read_unlock();
heavy_computation(); // Execute outside the critical section
printk("%d\n", value);

RCU variants

The Linux kernel provides multiple RCU variants for different scenarios:

• Classic RCU（CONFIG_RCU_FANOUT）: Default variant, does not support read-side preemption, suitable for non-real-time kernels.
• Preemptible RCU（CONFIG_PREEMPT_RCU）: Supports read-side preemption, read-side critical sections can be scheduled, suitable for real-time kernels.
• Sleepable RCU（SRCU，CONFIG_SRCU）: The read side can block (e.g., callingmsleep()), you need to usesrcu_read_lock()/srcu_read_unlock(), suitable for scenarios where the read side needs to sleep.

PER_CPU

example

net/ipv6/af_inet6.c

static int __net_init ipv6_init_mibs(struct net *net)
{
	int i;

	net->mib.udp_stats_in6 = alloc_percpu(struct udp_mib);
	if (!net->mib.udp_stats_in6)
		return -ENOMEM;
	net->mib.udplite_stats_in6 = alloc_percpu(struct udp_mib);
	if (!net->mib.udplite_stats_in6)
		goto err_udplite_mib;
	net->mib.ipv6_statistics = alloc_percpu(struct ipstats_mib);
	if (!net->mib.ipv6_statistics)
		goto err_ip_mib;

	for_each_possible_cpu(i) {
		struct ipstats_mib *af_inet6_stats;
		af_inet6_stats = per_cpu_ptr(net->mib.ipv6_statistics, i);
		u64_stats_init(&af_inet6_stats->syncp);
	}


	net->mib.icmpv6_statistics = alloc_percpu(struct icmpv6_mib);
	if (!net->mib.icmpv6_statistics)
		goto err_icmp_mib;
	net->mib.icmpv6msg_statistics = kzalloc(sizeof(struct icmpv6msg_mib),
						GFP_KERNEL);
	if (!net->mib.icmpv6msg_statistics)
		goto err_icmpmsg_mib;
	return 0;

err_icmpmsg_mib:
	free_percpu(net->mib.icmpv6_statistics);
err_icmp_mib:
	free_percpu(net->mib.ipv6_statistics);
err_ip_mib:
	free_percpu(net->mib.udplite_stats_in6);
err_udplite_mib:
	free_percpu(net->mib.udp_stats_in6);
	return -ENOMEM;
}

struct netinstruct netns_mib mibThe members are defined as follows:

struct netns_mib {
	DEFINE_SNMP_STAT(struct tcp_mib, tcp_statistics);
	DEFINE_SNMP_STAT(struct ipstats_mib, ip_statistics);
	DEFINE_SNMP_STAT(struct linux_mib, net_statistics);
	DEFINE_SNMP_STAT(struct udp_mib, udp_statistics);
	DEFINE_SNMP_STAT(struct udp_mib, udplite_statistics);
	DEFINE_SNMP_STAT(struct icmp_mib, icmp_statistics);
	DEFINE_SNMP_STAT_ATOMIC(struct icmpmsg_mib, icmpmsg_statistics);

#if IS_ENABLED(CONFIG_IPV6)
	struct proc_dir_entry *proc_net_devsnmp6;
	DEFINE_SNMP_STAT(struct udp_mib, udp_stats_in6);
	DEFINE_SNMP_STAT(struct udp_mib, udplite_stats_in6);
	DEFINE_SNMP_STAT(struct ipstats_mib, ipv6_statistics);
	DEFINE_SNMP_STAT(struct icmpv6_mib, icmpv6_statistics);
	DEFINE_SNMP_STAT_ATOMIC(struct icmpv6msg_mib, icmpv6msg_statistics);
#endif
#ifdef CONFIG_XFRM_STATISTICS
	DEFINE_SNMP_STAT(struct linux_xfrm_mib, xfrm_statistics);
#endif
#if IS_ENABLED(CONFIG_TLS)
	DEFINE_SNMP_STAT(struct linux_tls_mib, tls_statistics);
#endif
#ifdef CONFIG_MPTCP
	DEFINE_SNMP_STAT(struct mptcp_mib, mptcp_statistics);
#endif
};

And the macroDEFINE_SNMP_STATdefined as follows:

#define DEFINE_SNMP_STAT(type, name)	\
	__typeof__(type) __percpu *name

References

[1]

Linux RCU In-depth Analysis：Principle、Practice and Best Practices

[2]

Linux RCU In-depth Mechanism Analysis：Principles and Practice

[3]

What is RCU

[4]

What is RCU Fundamentally?

[5]

Linux Driver Development Twenty-eight：Read-Write Lock

[6]

Per-cpuvariable