# Linux Kernel Initialization

This blog is to understand how the Linux kernel is installed and initialized in the physical memory.

**Linux Boot Process:**

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1760731791478/dd576b14-a0b9-477c-86a3-e0ce660b2324.png align="left")

* When we start the system, initial control goes to the BIOS firmware, sitting on a chip on the motherboard.
    
* This software starts with checking if the hardware is working fine. This process is called Power-On-Self-Test(POST).
    
* Once POST completes, BIOS checks the boot order. BIOS tries to boot from the first available boot disk(USB, hard drive etc).
    
* Lets say, the system is trying to boot from a hard drive.
    
* BIOS tries to locate the Master Boot Record(MBR) from the hard drive. MBR is a 512 bytes(always) present in the Disk 0, Sector 1 and Head 0 location. This location is hard coded.
    
* BIOS moves MBR to the physical memory. MBR has the location of the boot loader. For example, GRUB, LILO etc. Let’s say we are using GRUB.
    
* MBR loads the GRUB files into the physical memory. This is when we see the menu options with all the available operating systems and versions. Users are expected to choose one in this list.
    
    ![](https://cdn.hashnode.com/res/hashnode/image/upload/v1760729928116/7715e5de-d7ee-43fe-9fe5-42a87377fb71.png align="center")
    
    We can access the GRUB files at ‘/boot/grub/grub.cfg’ location. We can modify the contents by changing the file ‘/etc/default/grub’.
    
* Boot loader has access to two files:
    
    1. first is the linux kernel executable, which is usually named, **vmlinuz-\*.** This is the file which controls the entire system and hardware once loaded and initialized later.
        
    2. Second is a file called initial RAM disk, usually named ‘**initrd-\***’. The purpose of this file will be explained further.
        
* Boot loader loads the linux kernel executable file first into the physical memory. But where does it load it?
    
    * This depends on the kernel configurations, like CONFIG\_PHYSICAL\_START, CONFIG\_PHYSICAL\_ALIGN, CONFIG\_RANDOMIZE\_BASE.
        
        On Linux Kernel-6.11, these values are as below:
        
    * ![](https://cdn.hashnode.com/res/hashnode/image/upload/v1760845807913/f4c80655-edf9-4adc-9a18-9b5dd8fcb7c2.png align="center")
        
    * These values show that the kernel is installed on the physical memory at 0x1000000 location( 100 MB) from the beginning of the physical memory. Since a static location is not safe, the latest implementations randomize this location. This is done by configuring CONFIG\_RANDOM\_BASE to ‘y’ during building the kernel.
        
        Also, the size of the memory allocation for Kernel depends on the build configurations used, but is usually 1 GB as shown below.
        
        ![](https://cdn.hashnode.com/res/hashnode/image/upload/v1760729226409/c58ce455-ad9b-4cce-9515-228124130158.png align="center")
        
        More about this will be discussed in another blog.
        
* Initially the kernel is in compressed format to ease the time to load to the memory. Once it's loaded, it de-compresses itself. Then the first activity it does is to mount the ‘initrd’ files on a file system called ‘initramfs’. This is an intermediary step before the kernel mounts the actual root file system.
    
* From the ‘initramfs’ file system, kernel executes some assembly code, which is usually present in location ./arch/x86/kernel/head\_{32 | 64}.S (depending on 32 vs 64 bit systems).
    
* This assembly code is architecture dependent. The goal of running this assembly code is to execute a C type method called ‘start\_kernel’.
    
* Start\_kernel completes the initialization of the kernel to its fullest form, upon which it mounts the root file system, starts the ‘init’ process and provides the user a login prompt.
    

---

**Kernel Memory Sections**

We noted earlier that the kernel occupies around 1 GB of memory in the physical memory during the above boot process.

This allocation consists of multiple sections as shown below:

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1760754412993/4096e395-a130-4b32-bc4f-e58fb3c269a9.png align="left")

* **22528K kernel code**: This is the executable machine code of the kernel itself, stored in the `.text` segment. This section is typically read-only.
    
* **4524K rwdata**: Short for read-write data, this memory contains global and static variables that have been initialized with a non-zero value. It is stored in the `.data` segment and can be modified at runtime.
    
* **15008K rodata**: Short for read-only data, this memory holds constant data that should not change during execution, such as string literals. It is stored in the `.rodata` segment.
    
* **4884K init**: This is temporary kernel initialization memory. After the system is booted, this memory is released back to the general memory pool to be reused.
    
* **4736K bss**: Short for Block Started by Symbol, this section contains uninitialized global and static variables. The kernel initializes this memory to zero at boot.
    
* **995728K reserved**: This is a large block of memory reserved by the firmware for hardware functions that the kernel does not manage. It is typically not available for general-purpose use.
    
* **0K cma-reserved**: This field refers to "Contiguous Memory Allocator" reserved memory, which is used for device drivers that require large, physically contiguous memory blocks. In this case, none is reserved.
    

These values are typical for a RAM of size 16 G. We can get these values for a system from ‘dmesg’ logs, as below(last line):

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1760762859734/070aaf5c-fccd-46dc-b441-f6667cc290e9.png align="center")

> The rest of the blog is to understand what happens in ‘start\_kernel’ code.

---

**Core** K**ernel Initialization**

* \[set\_task\_stack\_end\_magic()\] creates a kernel stack and adds a canary value **0x57AC6E9D** at the end of the stack to check for buffer overflows, As shown in the below picture.
    

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1760729763161/1fa1972d-adcf-43f4-92f5-18e40d520cea.png align="right")

* The size of this stack is architecture dependent. This stack is created in the kernel memory section ‘bss’ in the physical memory(RAM). The size is usually PAGE\_SIZE or 2 \* PAGE\_SIZE. A page size is usually 4096 or 8192 bytes.
    
    * How do you get this value on your system?
        
        To get the page size, you can use the command, ‘getconf PAGESIZE’ on the terminal, as below:
        

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1760819223429/73eba088-dfcc-45c2-b573-7237bce4e149.png align="right")

To get the kernel stack size, you can use the command ‘ulimit -s’ as below:

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1760819233863/3f081770-f358-414e-8028-9e7864bde6cd.png align="center")

* Whether the kernel stack grows upwards or downwards is based on the setting ‘CONFIG\_STACK\_GROWSUP’ and is platform dependent. For example, for x86 or ARM or PPC, this is not set and the stack grows downwards.
    

* \[smp\_setup\_processor\_id()\] Creates a CPU logical map for fetching logical processor id for a given physical CPU Id. This is required to manage and identify CPUs consistently across the system. For systems with multi-processor systems, where CONFIG\_SMP is set, the values range from 1…n
    
* \[local\_irq\_disable()\] Disable hardware interrupts on the CPU where the init process is running. These interrupts will be inactive until the init process is completed.
    
* \[boot\_cpu\_init()\] Decide which CPU Id to boot kernel from. In single CPU system, this would be 0.
    
* \[page\_address\_init()\] Create a 128 sized array of type ‘struct page\_address\_slot’ which is used to get the virtual address for a page in the highmem region. More about this will be discussed in a future blog.
    
* <div data-node-type="callout">
    <div data-node-type="callout-emoji">💡</div>
    <div data-node-type="callout-text">All the memory allocations from the below step are fulfilled through ‘memblock’. This is a boot time memory management system where memory is allocated in ‘regions’ directly in the physical memory. No virtual addressing happens here.</div>
    </div>
    
* \[setup\_arch()\] All the architecture specific setup here:
    
    * Load the kernel PGD (Page Global Descriptor) table to CPU’s CR3 register( only for 32 bit system)
        
    * Flush the TLB cache to remove any garbage entries(32 bit).
        
    * Set the MAX\_PHYSMEM\_BITS value which is used for addressing the virtual address space.
        
    * Reserve memblock region for the **.text** region of the kernel. Kernel memory sections have been described earlier.
        
    * Reserve memblock region for the **initrd** image.
        
    * Reserve memblock region for **data** part of the kernel.
        
    * Reserve memblock region for the **init** process. This memory is claimed back once the init process is started and moved to the user space.
        
    * Randomize the kernel base memory if CONFIG\_RANDOMIZE\_BASE is enabled.
        
    * Initialize the Interrupt Descriptor Table(IDT) which maps the interrupt or exception number to a memory address that handles it. This allows the CPU to avoid any unexpected events before the system is fully configured.
        
* \[setup\_nr\_cpu\_ids()\] Calculate the number of CPUs in the system and assign this value to the variable ‘nr\_cpu\_ids’. This value is used in all cases where CPU operations are involved.
    
* \[setup\_per\_cpu\_areas()\] Initialize memory for ‘per-CPU’ variables. This feature allows a copy of a variable to be made available(usually in L3 cache) for each CPU and thereby allow the CPUs to read the local copy instead of using locks to fetch its value from the physical memory.
    
* \[setup\_log\_buf()\] Setup a log buffer of size 128 KB. All the messages that ‘printk’ logs, are stored in this buffer. These messages can be accessed using the ‘dmesg’ shell command .
    

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1760835278147/e56ece8b-2297-4233-a8e1-90719d046e71.png align="right")

* \[sort\_main\_extable()\] Sort the kernel exception handlers table. This list is required to handle any exceptions seen by the kernel and make sure the system doesn’t crash.
    
* \[sched\_init()\] Initialize the scheduler, for task scheduling ahead. All the scheduler algorithms (like DL, RR, FAIR etc) are initialized here.
    
* \[trace\_init()\] Initialize tracing infrastructure, part of the ‘ftrace’ framework. Trace events are generated after this step.
    
* \[kfence\_init()\] Initialize Kernel Electric Fence(KFence) memory error detector. This tool is designed to detect common heap memory errors like out-of-bounds access, use-after-free and invalid-free errors.
    
* \[lockdep\_init()\] Initialize the lock dependency validator, which is a tool to detect the potential deadlocks and incorrect locking patterns within the kernel.
    
* \[anon\_vma\_init()\] Create slab cache called ‘anon\_vma’ for kernel objects of type ‘struct anon\_vma’, which is used to manage the anonymous virtual memory. This memory refers to the virtual memory which is not backed by a file or disk, like a process’s stack or heap.
    
* \[thread\_stack\_cache\_init()\] Create slab cache named ‘thread\_stack’ for kernel threads.
    
* \[fork\_init()\] Allocate slab cache named ‘task\_struct’ for kernel objects of type ‘struct task\_struct’. This structure holds the information about a single process.
    
* \[proc\_caches\_init()\] Create slab caches for the below kernel objects:
    
    * slab cache named ‘sighand\_cache’ for kernel objects ‘struct sighand\_struct’.
        
    * slab cache named ‘signal\_cache’ for kernel objects ‘struct signal\_struct’.
        
    * slab cache named ‘files\_cache’ for kernel objects ‘struct files\_struct’
        
    * slab cache named ‘fs\_cache’ for kernel objects ‘struct fs\_struct’.
        
    * slab cache named ‘vm\_area\_struct’ for kernel objects ‘struct vm\_area\_struct’.
        
* \[security\_init()\] Initialize the Linux Security Module (LSM) framework by loading all the LSM modules which were enabled in the boot config.
    
* \[net\_ns\_init()\] Create slab cache called ‘net\_namespace’ for kernel objects of type ‘struct net’ . Network namespaces allow for network isolation and each namespace can have its own network stack(ip addresses, firewalls, routing etc). This allows for launching multiple applications and services (isolated from each other) on a single Linux system. This is similar to container and virtual machine technologies which we use today.
    

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1760910779053/170f37de-8ce0-46c2-a32f-f0a3696652da.png align="left")

* \[vfs\_caches\_init()\] Initialize Virtual File System(VFS) by creating slab caches for kernel objects like directory entries, inodes, file objects, mount points, buffers for storing and accessing the files.
    
* \[pagecache\_init()\] Initialize page cache mechanism. As part of this setup, an array of size 256 is created to hold the information of the processes or threads which are waiting for a particular page to be made available. These are called wait queues. That means each page in the page cache, gets a wait queue of this size to hold information of the processes or threads waiting to access this page. This step also creates kernel threads and other data structures used to handle the ‘dirty’ pages, these are pages which have been modified by the process or thread, but not written back to the secondary storage.
    
* \[signals\_init()\] Create slab cache for kernel objects of type ‘struct sigqueue’.
    
* \[seq\_file\_init()\] Initialize seq\_file which is used to create virtual files later in the other file systems like ‘proc’. Virtual files are dynamically created when requested for. All the content in the /proc folder is created by this seq\_file mechanism.
    
* \[proc\_root\_init()\] Mount ‘proc’ file system. Mount ‘sys’, ‘fs’, ‘driver’, ‘’tty’, ‘sys’ file systems under ‘proc’ file system.
    
* \[nsfs\_init()\] Initialize and mount ‘nsfs’ file system. This file system helps with Linux namespaces, which are used to isolate the processes for optimizing the system resources. This file system cannot be mounted and usually is available through the proc file system. This file system helps to find which namespace is being used for a particular process.
    
* \[pidfs\_init()\] Initialize the ‘pidfs’ pseudo file system by creating the required data structures and mounting it. This file system is usually used within the kernel to manage the process Ids and related information.
    
* \[cpuset\_init()\] Initialize cpusets as part of the cgroup controllers. This step allows group of processes to limit access to cpusets.
    
* \[mem\_cgroup\_init()\] Create a slab cache called ‘mem\_cgroup’ to create kernel objects of type ‘struct mem\_cgroup’.
    
* \[cgroup\_init()\] Initialize control groups (cgroup). Mount the file system ‘/sys/fs’ as ‘cgroup’ and create cgroup files like ‘cgroup.procs’ etc, as shown below:
    

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1760848114691/3b9d2ca6-74c1-48bd-9543-9e7d74ee4315.png align="right")

* \[taskstats\_init\_early()\] Assign slab cache memory for the structure ‘taskstats’ which would be used for delay accounting subsystem.
    
* \[delayacct\_init()\] Initializes delay accounting subsystem, which tracks the and reports delays experienced by the tasks(processes and threads) waiting for kernel resources. This information is made available to user-space through the taskstats interface. This step involves assigning cache memory to store this information and then initiate this accounting for the initial ‘init’ process.
    
* \[acpi\_subsystem\_init()\] Initialize ACPI system. ACPI stands for Advanced Configuration and Power Interface which is an in-built power management mechanism which shuts down parts of the system which are not in use to save power. Usually helpful for laptops.
    
* \[kcsan\_init()\] Initializes KCSAN tool which can detect data races within the kernel code. We can enable this tool by using the setting CONFIG\_KCSAN=y and then compile the kernel. Initialization happens by adding a random value to per\_cpu\_data called ‘kcsan\_rand\_state’ variable.
    
* \[rest\_init()\] As a last step, spawns ‘init’ process as a user space process and assigns PID of 1 and spawns kernel thread ‘kthreadd’ and assigns it a PID of 2. A new ‘idle’ process is created (also called swapper process) with PID 0. This swapper process is scheduled by the scheduler when no other process is available to be scheduled on the CPU.
    
    * How do you identify kernel threads ? These are process which are reported in \[\] brackets when you run ‘ps’ command, as shown below:
        

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1760836418092/b3904664-0790-4dfc-83f6-d5530911fff5.png align="right")

* As you can see here, PID 1 is assigned to init process, PID 2 is assigned to first kernel thread ‘kthreadd’ and all the kernel threads are created by this thread and have a parent PID of 2 and are displayed in square brackets (\[\]).