Author: Li Rui
The author is actively looking for summer and winter internship opportunities, please contact the author through the contact information at the bottom of the article. Feel free to point out any mistakes or suggestions in the comments section.
Project address: github.com/KernelErr/M…
MoeOS on Nezha MQ development board (Quan Zhi D1s chip)
From watching “30 Days of Homemade OS” in middle school to learning Rcore-tutorial-book recently, I am more and more eager to build my own toy operating system. But when I watch these kinds of tutorials, I always have one feeling: “I don’t understand them, but I am shocked.” The moment I decided to start the project was when I saw the launch of the Sparrow development board and looked up some materials at home and abroad.
MoeOS has now completed the basic MMU setup, interrupt handling and clock, which took about a month to complete, with user mode to follow. The main reason for working on MoeOS is to learn about the operating system. I would like to thank the rCore team and other open source contributors for submitting my homework as an excellent tutorial.
This article will record the pits and thoughts I stepped on in the process of learning, and will be divided into sections according to the problems I encountered one by one.
In the memory
The first problem we encountered in developing MoeOS was to find a way to load the system into memory and run it. In the case of QEMU, we could easily specify the image to boot with the following command:
qemu-system-riscv64 \
-machine virt \
-nographic \
-bios sbi/fw_payload.bin \
-device loader,file=$(bin),addr=0x80200000
Copy the codeIn the early stages, I decided to skip the SD card image and use xFEL instead. Xfel is a tool made by Xboot, for the whole series of chips, in BootROM solidified a low-level subroutine named FEL, we can use FEL to complete memory initialization, load files into memory, execute binary operations.
In the MoeOS Makefile we can see the following command:
d1s: d1s_lds build
xfel ddr ddr2
xfel write 0x40000000 ./sbi/fw_jump_d1.bin
xfel write 0x40200000 $(bin)
xfel exec 0x40000000
Copy the codeHere we initialize the DDR2 memory (64M DDR2 memory built into the MQ development board), then write OpenSBI to address 0x40000000, MoeOS binary to address 0x40200000, and finally SBI. So here’s the question, how do we know which address to write?
For QEMU, we can look at the source code at hW/risCV /virt.c:
static const MemMapEntry virt_memmap[] = {
[VIRT_DEBUG] = { 0x0.0x100 },
[VIRT_MROM] = { 0x1000.0xf000 },
[VIRT_TEST] = { 0x100000.0x1000 },
[VIRT_RTC] = { 0x101000.0x1000 },
[VIRT_CLINT] = { 0x2000000.0x10000 },
[VIRT_PCIE_PIO] = { 0x3000000.0x10000 },
[VIRT_PLIC] = { 0xc000000, VIRT_PLIC_SIZE(VIRT_CPUS_MAX * 2) },
[VIRT_UART0] = { 0x10000000.0x100 },
[VIRT_VIRTIO] = { 0x10001000.0x1000 },
[VIRT_FW_CFG] = { 0x10100000.0x18 },
[VIRT_FLASH] = { 0x20000000.0x4000000 },
[VIRT_PCIE_ECAM] = { 0x30000000.0x10000000 },
[VIRT_PCIE_MMIO] = { 0x40000000.0x40000000 },
[VIRT_DRAM] = { 0x80000000.0x0}};Copy the codeWe can see that DRAM Memory starts at 0x80000000, and for our development board, we need to look at the Memory Mapping section of the chip manual of The Whole Chi D1s and find 0x40000000.
SBI
In RISC-V, there is the concept of SBI, the official description is as follows: SBI (Supervisor Binary Interface) is an interface between the Supervisor Execution Environment (SEE) and the supervisor. It allows the supervisor to execute some privileged operations by using the ecall instruction.
SBI will work under STATE M (Machine) and deal with hardware, while our kernel works in state S (Supervisor). The interface provided by SBI can be called through ECall. (If we write some User software, is in the User state, each state will have different permissions)
SBI is used as a startup phase of the system. In the Makefile, you can see that we execute SBI first, and when SBI completes its work, it jumps to the address we specified at compile time to execute the system.
MoeOS uses the interface provided by SBI to complete clock setting, input and output. At present, there are many SBI based on standard implementation, we use OpenSBI, but here we also recommend our domestic RustSBI written by Rust.
pub fn putchar(c: char) {
sbi_call(LegacyExt::ConsolePutchar.into(), 0, c as usize.0.0);
}
pub fn sbi_call(eid: usize, fid: usize, arg0: usize, arg1: usize, arg2: usize) - >usize {
let ret;
unsafe{ asm! ("ecall",
inout("x10") arg0 => ret,
in("x11") arg1,
in("x12") arg2,
in("x17") eid,
in("x16") fid,
);
}
ret
}
Copy the codeThe code shown above calls the SBI Interface by Ecall to output one character. See The Binary Interface Specification in Resources for definitions of these interfaces.
Inline asm
The good news is that Rust is stabilizing inline ASM. With the current Nightly edition, we can use inline ASM and Global ASM as follows:
usecore::arch::{asm, global_asm}; . global_asm! (include_str!("asm/boot.S")); .unsafe{ asm! ("",
out("x10") a0,
out("x11") a1,
);
}
Copy the codeAs a system, we need to use inline ASM to directly call the SBI interface, read register values, and so on.
Lit LED lights
SRC /device/ D1S /mq.rs
The MQ development board comes with a blue LED, which is shown on the schematic hanging above the PD22. Check the chip documentation and you can see that the register address controlling PD22 is 0x02000098, and for PD22 is bit 27 through 24, we can write 0001 to it to switch to the output mode. Here is the code for lighting the LED:
pub fn blue_led_on() {
let pd2_controller = 0x02000098 as *mut u32;
let mut pd2_controller_val: u32;
unsafe {
pd2_controller_val = ptr::read(pd2_controller);
}
pd2_controller_val &= 0xf0ffffff;
pd2_controller_val |= 0x01000000;
unsafe{ ptr::write_volatile(pd2_controller, pd2_controller_val); }}Copy the codeOut LED lights only need to change or to pd2_controller_val | = 0 x0f000000; Can.
Set the MMU
SRC /mem/ Mmu.rs
The chip we use supports THE Sv39 standard MMU, which is a piece of hardware responsible for processing CPU memory access requests and can help us complete the conversion from virtual address to physical address. Below, I will illustrate the working principle of MMU by combining codes and figures. Here our Page Size is 4KByte.
The above picture shows what our page table looks like. In the header bits are the contents of the vendor’s extension, indicating whether the page can be cached, etc. PPN is our physical address. The Bits at the end define the detailed status and permission identifier, as shown in the code. If the Bits at the end are 1, the page table is valid. If the Bits at the end are 1, the page table has write permission on the address corresponding to the page table.
The map function maps virtual addresses and their corresponding physical addresses to the page table, and we parse the code step by step. First, we set the address bits corresponding to VPN (virtual page number) and PPN (physical page number) in accordance with the standard, and then take the corresponding item of VPN[2] from the second level page table (there are 512 items). If it is invalid (that is, the end is 0), we call Zalloc to apply for memory and initialize the content to 0, and save the address in the page table. Then we restore the previously applied address, use this address space as the next level of page table, repeat the above operations to complete the initialization of the next level of page table. Here we apply 4096 bytes for a page entry, our VPN segment is 9 bits, that is, 512 numbers, 4096/512=8 bytes, 8*8=64 bits.
The last of the three page tables holds our physical address and corresponding permissions.
pub enum EntryBits {
None = 0,
Valid = 1 << 0,
Read = 1 << 1,
Write = 1 << 2,
Execute = 1 << 3,
User = 1 << 4,
Global = 1 << 5,
Access = 1 << 6,
Dirty = 1 << 7,
ReadWrite = 1 << 1 | 1 << 2,
ReadExecute = 1 << 1 | 1 << 3,
ReadWriteExecute = 1 << 1 | 1 << 2 | 1 << 3,
UserReadWrite = 1 << 1 | 1 << 2 | 1 << 4,
UserReadExecute = 1 << 1 | 1 << 3 | 1 << 4,
UserReadWriteExecute = 1 << 1 | 1 << 2 | 1 << 3 | 1 << 4.// T-HEAD Extend
Sec = 1 << 59,
Buffer = 1 << 61,
Cacheable = 1 << 62,
StrongOrder = 1 << 63,}pub fn map(root: &mut Table, vaddr: usize, paddr: usize, bits: u64, level: usize) {
if bits & 0xe= =0 {
panic!("Invalid PTE bits found");
}
let vpn = [
// VPN[0] = vaddr[20:12]
(vaddr >> 12) & 0x1ff.// VPN[1] = vaddr[29:21]
(vaddr >> 21) & 0x1ff.// VPN[2] = vaddr[38:30]
(vaddr >> 30) & 0x1ff,];let ppn = [
// PPN[0] = paddr[20:12]
(paddr >> 12) & 0x1ff.// PPN[1] = paddr[29:21]
(paddr >> 21) & 0x1ff.// PPN[2] = paddr[55:30]
(paddr >> 30) & 0x3ff_ffff,];let mut v = &mut root.entries[vpn[2]].for i in (level..2).rev() {
if! v.is_valid() {let page = zalloc(1);
v.set_entry(
// Set PTE's PPN
// As page size is 4kb, we have 12 zeros
// Set the address start from 10th bit
(page as u64 >> 2) | EntryBits::Valid.val(),
);
}
// Store next level PTE
let entry = ((v.entry & !0x3ff) < <2) as *mut Entry;
v = unsafe { entry.add(vpn[i]).as_mut().unwrap() };
}
// TODO: Complete C-SKY Extentions
let entry = (ppn[2] < <28) as u64 | // PPN[2] = [53:28]
(ppn[1] < <19) as u64 | // PPN[1] = [27:19]
(ppn[0] < <10) as u64 | // PPN[0] = [18:10]
bits |
EntryBits::Valid.val();
v.set_entry(entry);
}
Copy the codeSo if we get a virtual address, how do we get a physical address?
pub fn virt_to_phys(root: &Table, vaddr: usize) - >Option<usize> {
let vpn = [
// VPN[0] = vaddr[20:12]
(vaddr >> 12) & 0x1ff.// VPN[1] = vaddr[29:21]
(vaddr >> 21) & 0x1ff.// VPN[2] = vaddr[38:30]
(vaddr >> 30) & 0x1ff,];let mut v = &root.entries[vpn[2]].for i in (0..=2).rev() {
if! v.is_valid() {break;
} else if v.is_leaf() {
let off_mask = (1< < (12 + i * 9)) - 1;
let vaddr = vaddr & off_mask;
let addr = ((v.entry << 2) as usize) & !off_mask;
return Some(addr | vaddr);
}
let entry = ((v.entry & !0x3ff) < <2) as *mut Entry;
v = unsafe { entry.add(vpn[i - 1]).as_mut().unwrap() };
}
None
}
Copy the codeLet’s go level by level and make sure we find a valid page table for each level. In SRC /init.rs, we set the SATP register (MMU address translation register) to the address of our second-level page table after we initialized the MMU (kernel memory address mapping) and performed virtual memory synchronization via the sfence.vma instruction.
Note that we need to set the kernel pages to Access and Dirty, otherwise QEMU will work but will actually have problems.
interrupt
SRC /asm/trap.S SRC /trap.rs
For RISC-V, MoeOS sets STVEC (Superuser mode vector base Address register) to the address of the interrupt handler, with mode set to Direct.
Pattern interpretation:
-
00: Direct All interrupts use the same address as the exception entry
-
01: Vectored uses the address of BASE + 4 * Exception Code as the Exception entry
When an interrupt occurs, the code we write in assembly clears a space off the stack to store the scene (register values, etc.) and then calls the trap_handler function written by Rust. Resume the scene when the function returns.
#[no_mangle]
extern "C" fn trap_handler(cx: &mut TrapContext) -> &mut TrapContext {
let scause = scause::read();
let stval = stval::read();
match scause.cause() {
Trap::Exception(Exception::UserEnvCall) => {
cx.sepc += 4;
cx.x[10] = 0; }, Trap::Exception(Exception::StoreFault) | Trap::Exception(Exception::StorePageFault) => { error! ("{:? } va = {:#x} instruction = {:#x}", scause.cause(), stval::read(), sepc::read());
panic!("page fault!"); }, Trap::Exception(Exception::IllegalInstruction) => { error! ("IllegalInstruction"); }, Trap::Interrupt(Interrupt::SupervisorTimer) => { timer_next_triger(); } _ = > {panic!("Unhandled trap: {:? } stval = {:#x}", scause.cause(), stval);
}
}
cx
}
Copy the codeScause stores the type of interrupt, and stval stores the cause of the exception. We use the SEPC register to hold the PC value after exiting the exception handler function.
The clock
STC /timer.rs
The SBI is designed to provide us with an interface that we can use to set how many clock times have passed when the interrupt is triggered, and the interrupt handler that we’re handling is also visible in the container.
pub static mut CLOCK_FREQ: usize = 24000000;
const MICRO_PER_SEC: usize = 1 _000_000;
const TICKS_PER_SEC: usize = 1000;
pub fn get_time_us() - >usize {
unsafe {
time::read() / (CLOCK_FREQ / MICRO_PER_SEC)
}
}
pub fn get_time() - >usize {
time::read()
}
pub fn timer_next_triger() {
unsafe{ set_timer(get_time() + CLOCK_FREQ / TICKS_PER_SEC); }}Copy the code
The clock is 24Mhz, and we set 1000 ticks per second, meaning every 24,000 units of time we trigger an interrupt, so we know how much time has passed.
conclusion
There is a lot of work to be done on MoeOS, and I will continue to improve MoeOS in my spare time. I have stumbled a lot in the process of making OS, but I have also learned a lot. After going through several interviews, I realized that I wasn’t doing well in many of the basics, so I decided to do something fun to learn.
Let’s keep going in 2022!
Learning materials
- Ne Zha MQ Development Board (BOM)
- RCore Tutorial – Book the third edition
- The Adventures of OS: Making a RISC-V Operating System using Rust
- RISC-V SBI specification
- RISC-V Supervisor Binary Interface Specification
- Chip, core official documents
About the author
Bupt sophomore, open source enthusiast. Currently, I am working hard to learn the Linux kernel and Rust, and have a passion for technology. Welcome to add friends in Rust Chinese community.
GitHub：github.com/KernelErr
Personal blog: Lirui.tech /
Contact email: [email protected]