Bastion OS: Phase I

Meshcheryakov Arseniy

Meshcheryakov Arseniy

— 8 min read
Bastion OS: Phase I

"My main task here is create bootable efi application for kernel, create skeleton of kernel and entry point of kernel for both archs."

Project skeleton

So, first of all let's look on project skeleton, it's very similar with Linux. Of course in future some moments here will be changed.

/ — Project Root

Top-level Makefile, config.mk (shared toolchain), .config

boot/ — UEFI Boot Loader

PE32+ .efi application that loads the kernel

  • boot/common/ — Architecture-independent code

    • ELF parser
    • Boot sequence: GPO, memory map, ACPI/FDT discovery
    • ExitBootServices()

  • boot/x86_64/ — x86_64-specific

    • EFI entry point and PE linker script

  • boot/aarch64/ — AArch64-specific

    • EFI entry point and PE linker script

configs/ — Kernel default configurations

doc/ - Kernel documentation

include/ — Shared Headers

  • include/boot/ — Single shared contract between loader and kernel and efi defenition
  • include/elf/ - ELF format defenition
  • include/kernel/ — Additional shared kernel headers

kernel/ — The Kernel (ELF64)

  • kernel/arch/ — Architecture-specific implementations

    • kernel/arch/x86_64/ — Assembly entry, linker script, arch-specific code
    • kernel/arch/aarch64/ — Assembly entry, linker script, arch-specific code

  • kernel/core/ — Architecture-independent kernel logic

    • Entry, banner, memory map dump, Hello world

  • kernel/lib/ — Freestanding support libraries (strings, cxxabi...)

  • kernel/include/ — Kernel-internal headers

  • kernel/drivers/ — Device drivers

  • kernel/fs/ — Filesystem implementations

  • kernel/mm/ — Memory management

    • PMM (Physical Memory Manager)
    • VMM (Virtual Memory Manager)
    • Heap

  • kernel/loader/ — Userspace ELF loader

scripts/ — Build & Run Helpers

  • configure.py — Linux-style config system

    • Converts .config → config.h + config_generated.mk

  • create_disk.sh — Disk image creation

    • Creates 64 MiB GPT/FAT32 disk image
    • Includes .efi and kernel.elf

  • run_qemu.sh — QEMU launcher

    • Finds OVMF/AAVMF firmware
    • Launches QEMU

result/ — Result Output (generated, ignored by git)

  • {arch}/bastion.img — Image for run in QEMU
  • {arch}/boot/ — directory with efi application
  • {arch}/kernel/ — directory with ELF kernel

BOOT Options

Legacy Boot

Legendary old method with magic number in MBR. It's very uncomfortable to use, i will add this method, but not today.

Limine

https://github.com/limine-bootloader/limine

Good bootloader, but i decided, that it is very easy and not funny to use ready bootloader.

U-BOOT

Great bootloader. In arm we almost haven't another options.

Roughly speaking u-boot have 3 ways to boot kernel:

  • U-Boot own protocol. Loads a flat binary or uImage. Passes dt to X0 and run it
  • booti/bootm commands. Loads Linux-format Image with header. Passes device tree pointer
  • U-Boot as UEFI firmware

It taker a very long time to add support of uImage (but in future i want to add it) and weird to use Linux format. So UEFI is great and universal option here.

GRUB

Same as Limine. Also can run UEFI application, so support grub format is not necessary at first stages.

UEFI Booting

Unified Extensible Firmware Interface. Absolutely best way to boot kernel. Easy, powerfull. I will use it for booting my kernel.

So, first of all, let's talk about how Linux boot with that.

Linux has EFI stub. When we build kernel with CONFIG_EFI_STUB=y, the final vmlinuz become PE32+ executable (https://docs.kernel.org/admin-guide/efi-stub.html). Linux writes hand-written PE32+ header to it and UEFI can successfully read it.

In two words, this efi stub do basic things, like getting memory map, loads initrd and after all jump to real kernel. With GRUB same situation: UEFI firmware -> GRUB (EFI app) -> Linux EFI stub -> real kernel.

With my kernel situation is a bit complex. It compiles to ELF format. I don't add pe32 header. And because of it i split booting into two files:

1) loader.efi - EFI application, that to add efi stuff and jump to kernel file

2) kernel.elf - kernel itself

With this logic i have next problems:

  • I need to use two files instead of one.
  • I need use clang, because GCC can't compile to pe32+ format (TODO: clarify this moment).

At first time, i will live with this, but in ideal world, be better if i will add creating of pe32+ header into code (like in Linux) and remove clang dependency.

Boot chain

Ok, i decided to use UEFI. Let's show boot chain.

1) UEFI firmware loads loader.efi

2) Get EFI_HANDLE and EFI_SYSTEM_TABLE pointers (passed from UEFI firmware).

3) Open the kernel ELF file from the EFI partition (via efi protocols)

4) Read, parse and validate ELF64 header of kernel file

5) Iterates via PT_LOAD segments, calculate total memory footprint for kernel

6) Allocate memory pages via UEFI for pt_load. And then copy all segment to new memory

6) Find firmware tables (RSDP or FDT)

7) Get Memory Map

8) Call ExitBootServices (UEFI Point of No Return)

9) Convert UEFI Memory Map to our format

10) Fill BootInfo structure (magic value, memory_map, addresses of kernel entry point)

11) Jump to real kernel

It was short description of boot chain, so now, I can go through all points and describe implementation.

Boot Implementation

UEFI Loads loader.efi

Very simple part. I just create linker.ld file, where setup ENTRY(efi_main), and setup segments (.text, .data, .bss .dynamic and etc) with 4KiB alignment. Linker scripts are same for both architectures.

All magic is done by clang with -target x86_64_unknown-windows (or aarch64-unknown-windows). Clang create PE32+ executable bin, i don't need to do anything.

Loader is running

UEFI run efi_main function. This function definition looks like this:

extern "C" EFI_STATUS EFIAPI efi_main(EFI_HANDLE image_handle, EFI_SYSTEM_TABLE* system_table)

And now very important moment. I need to somehow communicate with UEFI. This happens through system_table. It's just a pointer to memory. This memory have structs, fields and functions. I can write to it and read from it. UEFI also can write and read. Absolutely simple. But i need have definition of this structs and fields. Good way is use gnu-efi (gnu-efi is more than just header, but not now about it) , best way is rewrite whole UEFI spec to code, but i choose simplest way: i ask claude to read specification and write header for me. It's not hard task and he did it good.

Ok, we are in efi_main function (that located in entry.cpp). We have full UEFI functional. And we can go next.

Load kernel file

Go to efi_loader_main function. Here we can print some text to console (via TEXT_OUTPUT_PROTOCOL).

Next we call load_kernel_file, that via EFI_SIMPLE_FILE_SYSTEM_PROTOCOL open kernel.elf file, allocate memory and read this file to this memory.

Parse and validate kernel file

Run elf_load function. It get ELF header and validate it (check magic, class, architecture, type and other). Nothing hard.

From ELF header we get entry_point (e_entry field) value. It's address of place where kernel entry point located.

PT_LOAD segments

Here we iterates over all ELF Program PT_LOAD Headers. Get p_vaddr and p_memsz of it. First is Virtual address of the segment in memory, second is size of this segment. We need to find minimal virtual address and maximum virtual address(in other words we need total size of loadable segments).

When we get, we can allocate memory for it and copy segments from file to new memory.

In final of this stage we have result struct, that has phys_base (physical address from allocation), entry_point address, virt_base address (minimal virtual address) and total size of loadable segments.

Note: we can't use file memory, because of fixed addresses (kernel have hard-coded entry point address), same point in elf file memory will locates in other address.

Graphic Output Protocol

Kernel need somehow show something to user. Historically best way to do it is serial console. But claude suggested to use GPO framebuffer. It uses buffer to draw pixels on the display. Use it as main console instead of serial is worst idea ever. And of course i asked claude to write code for it. On my big surprise it works. So, yes, my earlyprintk is GPO framebuffer.

Note: at current phase of development i haven't any driver logic. Looking ahead, i will have EarlyDriver class, that will work before VMM (like Linux earlyconn). And will have different output technics, including normal GPO driver (not library code like now), UART console and other.

Firmware tables

Now we can iterates over ConfigurationTable and find rsdp address or fdt address.

For it we need ACPI2.0 GUID and DTB_GUID (https://uefi.org/htmlspecs/ACPI_Spec_6_4_html/05_ACPI_Software_Programming_Model/ACPI_Software_Programming_Model.html)

As i wrote before, claude create EFI header for me, including all needed GUIDs.

GetMemoryMap

The most important thing.

We need to know what memory and how much we have in system. So via UEFI GetMemoryMap we get all needed information.

Memory map is array of entries. Each entry is Base addr, Length and Type.

Interesting moment: UEFI will store memory map in new memory, that we need allocate. But allocation change this map itself. So, because of that we need to allocate more memory, that current size of map.

So, after getting memory map we can go to end.

ExitBootServices

Point of No Return for UEFI.

When we call it, UEFI lose control and we can't use it and its services.

After exit, we convert memory map from uefi format to our new format:

enum class MemoryRegionType : uint32_t {
    Usable                = 0,  // Free RAM - kernel can use
    Reserved              = 1,  // Firmware/hardware reserved
    AcpiReclaimable       = 2,  // ACPI tables - free after parsing
    AcpiNvs               = 3,  // ACPI non-volatile storage
    BootloaderReclaimable = 4,  // Loader code/data - free after kernel init
    KernelAndModules      = 5,  // Kernel image + any loaded modules
    Framebuffer           = 6,  // Framebuffer memory
};

Framebuffer isn't part of KernelAndModules type because of debug purposes.

Fill Boot Info

Ok, we sort all memory into special regions.

Now we can fill boot_info struct - the main data that that goes from loader to kernel.

struct BootInfo {
    uint64_t        magic;               // Must be BOOT_INFO_MAGIC

    // Framebuffer
    FramebufferInfo framebuffer;

    // Memory map (array of MemoryRegion)
    MemoryRegion*   memory_map;
    uint64_t        memory_map_count;

    // Platform-specific firmware tables (both always present to keep layout uniform)
    uint64_t        rsdp_address;        // ACPI RSDP (x86_64), 0 if absent
    uint64_t        fdt_address;         // Flattened Device Tree (aarch64), 0 if absent

    // Kernel load info
    uint64_t        kernel_phys_base;    // Where the kernel was loaded physically
    uint64_t        kernel_virt_base;    // Kernel's virtual base (from ELF)
    uint64_t        kernel_size;         // Total size of kernel in memory

    // Kernel entry point (virtual address from ELF e_entry)
    uint64_t        kernel_entry_point;

    // Higher-half direct map base (if set up by loader)
    uint64_t        hhdm_base;           // e.g., 0xFFFF800000000000
};

Jump to kernel

Let's back to entry.cpp code.

We can calc real kernel entry address:

uint64_t entry_offset = g_boot_info.kernel_entry_point - g_boot_info.kernel_virt_base;

uint64_t phys_entry   = g_boot_info.kernel_phys_base + entry_offset;

entry_offset is how far entry point from the start of kernel image. and phys_entry is phys_base address of ELF in memory + this "how far".

And now we can jump to real kernel address.

Real kernel

As i said many times our kernel is simple ELF binary, that has .text .rodata .data .bss segments. In linker.ld scripts i wrote all this segments.

And for each architecture i have base address, for x86 it is 0x100000 for arm is 0x40100000. As i remember 0x100000 is BIOS legacy value, but for QEMU testing i can use any address (for example 0x0). For arm this value connected with fact, that QEMU RAMs for arm start from 0x40000000.

In these linker scripts i set ENTRY(_start). kernel entry point. This point located in entry.S for each architecture.

Note: we must use asm code here, because after UEFI we can't jump to c/c++ code, because it need stack, asm can work without stack.

Asm code does very simple things: disable interrupts, set up kernel stack (16KiB) and jump to kernel_main (with pointer to BootInfo).

kernel_main is our c++ architecture-independent code, that get boot_info, validate it, read framebuffer data, parse memory map, print hello world and go to infinity halt.

So, here Phase I is done. We have bootable kernel.

Configuration

Two words about configuration.

I'm bad in Python and i asked claude to write me script, that will read configuration and create config_generated.mk and config.h.

Then in makefile i add -include $(PROJECT_ROOT)/include/kernel/config.h to compiler argument. Like in Linux.

Read more