Personal Notes on Linux Virtual Memory Management

A few days ago, I simply went through the "Understanding Linux Kernel Memory Management". The text is very short and it is not easy to read. It is a frequently-read book.趁Impression is still hot to take a note first, the content is more personal, there is nothing to understand, forgive me: p

『Memory Management』 is actually a very broad word, physical page management, page table management, address space management , memory allocation, etc., each out of each piece can be called "memory management", but the difference between them is huge, with a word to cover the past, it is like a blind, not seeing the woods. To understand its full picture, it is better to discuss it separately. If you can figure out every part, you won't see the whole picture.

System Initialization and Memory Layout

In the boot phase (setup.o), the kernel will first map two temporary page tables to the first two entries of the page directory swapper_pg_dir^1, which will be 0~8m. The virtual address of 3g~3g+8m is mapped to the physical address of 0~8m at the same time.

The initialization of the kernel page table is in the kernel_physical_mapping_init() function in the kernel initialization phase. If possible, use PSE to set large pages; use PGE to set the global page, fix the memory map in the TLB, and do not need to refresh again when switching the address space.

Linux fixed mapping kernel address space. On x86 machines, the physical memory is divided into 3 zones: ◦ZONE_DMA: 0~16m◦ZONE_NORMAL: 16m~896m◦ZONE_HIGHMEM: 896m~

where ZONE_DMA and ZONE_NORMAL are fixedly mapped to 0xc000000, the rest The 128mb address space is used to map physical pages in ZONE_HIGHMEM, or FIXMAP.

In x64, the address space is sufficient to map all physical memory, ZONE_HIGHMEM is empty.

kmap

To use memory in ZONE_HIGHMEM in the kernel, you must first map it to the kernel address space via kmap().

But kmap() may go to sleep, and you must use kmap_atomic() in the interrupt context.

Buddy Allocator and Physical Page Management

In Linux, Buddy Allocator is used as a page-level memory allocator for contiguous physical pages of 2 powers. It acts as both a kernel memory allocator and a backend for Page Cache.

The related routine is alloc_pages(gfp_t gfp_mask, unsigned int order), which returns a linked list of page structures.

How to do limited resources bounce buffer: DMA memory is limited, space for life cycle; ◦kmap_atomic (): Reserved page table items are limited, as soon as possible, as soon as possible release.

NUMA and Node

On SMP systems, when CPUs increase, indiscriminate memory access becomes a bottleneck affecting scalability^2. The solution for this is to provide a separate piece of memory for each processor. In principle, the processor only accesses memory close to itself, thereby increasing the scalability of memory access. This is NUMA (Non-Uniform Memory Access), where each processor and memory is collectively called Node.

And what the kernel needs to do is to allocate only the memory in this Node when the CPU allocates memory. In Linux, the structure of Node is pg_data_t.

Slab and kernel memory allocation

To use memory in the kernel, use Slab. It uses buddy allocator as the back end, and the application for large blocks of memory is divided into small blocks, each of which is a large memory block. ◦kmem_cache_alloc() and kmem_cache_free(), a dedicated memory allocator for allocating structures commonly used in the kernel. ◦kmalloc() and kfree(), a generic memory allocator. Behind it is a set of slabs of different sizes.

If the allocation/release of structures is not particularly frequent, it is usually sufficient to use only kmalloc().

Page Table Management

The x86 linux three-level page table is actually prepared for PAE. If you have a 64-bit machine, you must have four levels.

misc notes◦struct address_space The name is misleading. In fact, the structure describing the user address space in the kernel is struct mm_struct. The struct address_space structure is more like a description of the Page Cache. ◦GFP is an abbreviation of Get Free Page.