Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions build/kimage.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
//! The kernel is ours and linked by this build, so anything here that it
//! could not be is this build's bug, and panics naming it.

use toyos::elf::program::lay_out;
use toyos::elf::program::{lay_out, Source};
use toyos::elf::rela::{self, Op, RelaTable, Rules};
use toyos::elf::section::{symbol_tables, SectionTable};
use toyos::elf::sym::{SymTab, STT_FUNC};
Expand Down Expand Up @@ -80,7 +80,7 @@ pub fn section(kernel: &[u8], supervisor: &[u8], arch: Arch) -> Vec<u8> {
}
}

let program = lay_out(supervisor, machine, SUPERVISOR_OBJECT).unwrap_or_else(|why| panic!("supervisor: {why}"));
let program = lay_out(supervisor, machine, Source::Immutable(SUPERVISOR_OBJECT)).unwrap_or_else(|why| panic!("supervisor: {why}"));
Input {
machine: e_machine,
entry: min + layout.entry().get(),
Expand Down
11 changes: 8 additions & 3 deletions docs/src/architecture/memory.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,14 @@
- Each thread's stack capacity is committed before the thread starts, populated
lazily and guarded, with caller-chosen sizes and ordinary creation failure.
- Programs may generate executable code without a prompt; write and execute
stay separate and executable pages are shared safely.
- Shared memory may be installed at a requested address in a child before it
starts; conflicts are refused. There is no address-space cloning.
stay separate and executable pages are shared safely: shared memory is
executed only where nobody can write it.
- A range of a memory object is mapped at a requested address, in the caller
or in a child before it starts; conflicts in a child are refused. There is
no address-space cloning.
- A program's read-only segments are the pages of its image wherever nobody
can write the image, held once however many processes run it. Its writable
segments are private copies, taken as they are touched.

## The kernel's own memory

Expand Down
10 changes: 5 additions & 5 deletions docs/src/using/interface.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,11 +69,11 @@ userland library over the SDK.
environment (HOME, the starting directory, ...)
```

A parent reads the child's program and lays out the child's memory. What it
tells the child goes in the startup block, which the kernel copies as it
copies the program and never reads; the SDK, std and libc read it where it
lies. A child whose parent wrote none starts with no argument and an empty
environment.
A parent reads the child's program and lays out the child's memory, each
range the pages of a memory object or a private copy of them. What it tells
the child goes in the startup block, which the kernel copies and never reads;
the SDK, std and libc read it where it lies. A child whose parent wrote none
starts with no argument and an empty environment.

A program holds nothing it was not given. It asks for more only by connecting
to a service in its namespace, whose server decides.
Original file line number Diff line number Diff line change
Expand Up @@ -88,22 +88,59 @@ live split of a user leaf.
holds is refused at the call, not at a fault; its page tables are taken
then too. Kernel objects a process makes are charged to its job
(`kernel/src/mm/charge.rs`).
5. **Shared text.** A spawn copies each page of its program's segments into
the process at the page's first touch, out of the image object or out of
a snapshot of its `/tmp` file; no fault reads a file. A ROOT program's
image object is SYSTEM's subrange of ROOT's own pages (`fs::IMAGE`), so
no spawn copies the whole binary first. Mapping ROOT's read-only text
into each process without the per-page copy is the next step. Exit: two
processes of one binary hold its text once.
6. **`/apps` image sharing.** The design is the owner's decision, owed when
this stage is reached.
5. **Shared text.** Built for an image nobody can write: a ROOT program's
image object is SYSTEM's subrange of ROOT's own pages (`fs::IMAGE`), and
a spawn and the library loader map its read-only segments from it
(`SpawnBacking::Share`, `vm_map`), copying only a writable segment, a page
at its first touch. A program anywhere else is still read whole into
memory of the spawner's and copied a page at a time, until 7b's step 3.
A shared segment's mapping is never widened, so a program can no longer
make its own read-only segment writable (`mprotect`). Exit for that: 7b's
step 2 converts on protect, so a protect that adds write to such a range
commits its pages and makes it a copy range of the same object; only a
page shared until its first write waits for step 5's write faults. Exit:
two processes of one binary hold its text once, wherever it is served
from.
6. **`/apps` image sharing.** Decided by the orchestrator under the owner's
delegation of technical decisions (2026-11-11): 7b's step 3 maps images
from every pager-backed volume, `/apps`, `/tmp` and `/home` alike, sealed
against writes in place while they run. When a file server ends, every
mapping of its objects is revoked (7b's D1), so a program running from
DATA, like any process reading a mapping of a DATA file, dies at its next
access to that mapping.
7. **Large user leaves where they pay.** No user leaf is 2 MiB now, so every
user page costs a 4 KiB TLB entry and a table walk the 2 MiB windows did
not: a binary's text and data, a heap, a stack, a large buffer. Unpriced:
nothing measures the TLB misses of a workload against 2 MiB leaves. Exit:
whole aligned spans of one range get a 2 MiB leaf where a measurement
shows it pays, or the measurement shows it does not.

**7b lands in steps**, each a pull request: (1) a range of a memory object
mapped where the caller says, an immutable one shared and executable, which
is stage 5 for `/system`: built; (2) a paged object mapped read-only and
shared: faults that park on the pager, reverse records, revocation; (3) a
spawn maps its image from any pager-backed volume (stage 6), sealed against
writes in place while it runs; (4) the
storage track's attributes and names without a request; (5) writable shared
mappings and write-back; (6) the #1077 retry
(`issues/a-tcp-connect-zeroes-twelve-mebibytes.md`).

- **Owed on metal:** step 1 (#1259), on the T14 and the AMD bench: the whole
`testcases` boot, whose `vm_map_shares` member is the x86-64 verdict that a
program's read-only segments are its image's pages and take no private
frame (`spawn_cost` among its other members), and the `shared-debug` and
`proctreecase` boots. Main must show every job of each boot exit 0.
- **Owed on metal:** step 1 (#1259), on the T14 and the AMD bench: osbench's
`process.spawn_wait`, `proc.spawn_wait.*` and `proc.posix_spawn_wait.*`,
main before #1259
against main after, four boots a side. Main after must be no slower than
before; a spawn no longer copies a `/system` program's text, and whether
that makes it faster is not measured on metal.

Until step 2's revocation exists, an object a space maps must keep its frames
for the object's life: ROOT's pages stay non-reclaimable, and ROOT is not
served through a pager, until then.

**Paged objects** (memory design §7, `issues/toyos-builds-itself.md` item 13)
are split in two. **7a**, delivered by the storage track's file-server work:
a pager's objects, read through `SYS_PAGED_READ` and never mapped
Expand Down
27 changes: 11 additions & 16 deletions issues/root-in-memory-costs-its-whole-size.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,14 +22,14 @@ in turn, each carrying at most `SHARED_BOOT_BYTES` (64 MiB) of test binaries;
carries `tests/images/testcases.toml`'s own programs, 13 MiB (3328 blocks), where every test
boot carried the whole 619 MiB catalogue before.

**A running `/system` program's pages are in RAM twice.** Its spawn names
SYSTEM's object for the file, a subrange of ROOT's own pages (`fs::IMAGE`,
`SYS_SHM_SUB`), and `SharedImage::read_page` (`kernel/src/file_backing.rs`)
copies each page the child touches out of it: ROOT's copy and the page's are
held. Until SYSTEM handed that object out, each spawn also read the whole
binary into an image object of the spawner's, a third copy, and the
measurements below are of that design. Measured on QEMU's x86-64 machine booting
`tests/images/testcases.toml`, toybox's `free` after the boot: 330 MiB in use
**A running `/system` program's read-only pages are ROOT's own.** Its spawn
names SYSTEM's object for the file, a subrange of ROOT's own pages
(`fs::IMAGE`, `SYS_SHM_SUB`), and shares the segments the program cannot
write (`SpawnBacking::Share`); a writable segment's pages are copied as the
child touches them. Before that every touched page was copied, and until
SYSTEM handed the object out each spawn also read the whole binary into an
image object of the spawner's; the measurements below are of that design.
Measured on QEMU's x86-64 machine booting `tests/images/testcases.toml`, toybox's `free` after the boot: 330 MiB in use
at `b9e4c5862`, 384 MiB once ROOT left the kernel, one boot each, the same
session; the 54 MiB are the running services' images, each rounded up to
2 MiB, and the SYSTEM file server.
Expand Down Expand Up @@ -78,11 +78,6 @@ complete K ms`; a test boot on QEMU reads 1390, 108 (14) and 344.
Not measured: the T14. What that run owes is that line off the stick and off
NVMe, against the base's `Boot: complete`.

Exit: a program's pages mapped from ROOT's own read-only pages rather than
copied into each process, before the self-update's signature check makes the whole read a fixed
cost. The orchestrator ruled the mechanism (2026-10-22): the memory design's
stages 2, 4 and 6, namely 4 KiB pages, so an image is a program's size and not
2 MiB; memory-object subranges; and immutable file objects shared across
spawns, so that SYSTEM's object for a ROOT file is a subrange of the boot
image. No dedicated call over part of a boot image is to be built.
`spawn_children_cap` holds all 255 children again once that has landed.
Exit: the T14's boot line read off the stick and off NVMe, and the used
memory of 255 held children of one `/system` program read again now that
their read-only segments are ROOT's pages.
69 changes: 69 additions & 0 deletions kernel/pure/vm.rs
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,36 @@ impl Prot {
}
}

/// What a memory object is, for mapping its own pages.
#[derive(Clone, Copy, PartialEq, Eq, Debug)]
pub struct Shared {
/// Its bytes, whole pages.
pub size: u64,
/// Nobody can write it.
pub immutable: bool,
/// Memory the kernel does not own, taken back from where each mapper
/// said it mapped the whole.
pub aperture: bool,
}

impl Shared {
/// Whether `[offset, offset + len)` is whole pages of the object that a
/// space may map as `prot`: executable only where nobody can write it,
/// writable only where somebody can, and an aperture whole.
pub fn maps(self, prot: Prot, offset: u64, len: u64) -> bool {
let pages = len != 0 && offset.is_multiple_of(PAGE) && len.is_multiple_of(PAGE);
let within = offset.checked_add(len).is_some_and(|end| end <= self.size);
let allowed = match prot {
Prot::None => false,
Prot::Read => true,
Prot::ReadWrite => !self.immutable,
Prot::ReadExec => self.immutable,
};
let whole = offset == 0 && len == self.size;
pages && within && allowed && (whole || !self.aperture)
}
}

/// An object a range maps, as the kernel holds it; here only so that a range
/// split in two maps the same object twice and two neighbours mapping one
/// object in order join.
Expand Down Expand Up @@ -363,6 +393,45 @@ mod tests {
use std::vec;
use std::vec::Vec;

/// Shared memory is executed only where nobody can write it and
/// written only where somebody can; a range is whole pages inside the
/// object, and an aperture's is the whole of it.
#[test]
fn an_object_maps_as_what_it_is() {
let size = 4 * PAGE;
let writable = Shared { size, immutable: false, aperture: false };
let immutable = Shared { immutable: true, ..writable };
let aperture = Shared { aperture: true, ..writable };
for (object, prot, maps) in [
(writable, Prot::Read, true),
(writable, Prot::ReadWrite, true),
(writable, Prot::ReadExec, false),
(writable, Prot::None, false),
(immutable, Prot::Read, true),
(immutable, Prot::ReadExec, true),
(immutable, Prot::ReadWrite, false),
(immutable, Prot::None, false),
(aperture, Prot::ReadWrite, true),
(aperture, Prot::ReadExec, false),
] {
assert_eq!(object.maps(prot, 0, size), maps, "{object:?} whole as {prot:?}");
}
for (offset, len, maps) in [
(PAGE, 2 * PAGE, true),
(3 * PAGE, PAGE, true),
(0, 0, false),
(1, PAGE, false),
(0, PAGE + 1, false),
(3 * PAGE, 2 * PAGE, false),
(size, PAGE, false),
(u64::MAX - PAGE + 1, PAGE, false),
(PAGE, u64::MAX - PAGE + 1, false),
] {
assert_eq!(writable.maps(Prot::Read, offset, len), maps, "{offset:#x}+{len:#x}");
assert!(!aperture.maps(Prot::Read, offset, len), "an aperture's {offset:#x}+{len:#x}");
}
}

/// An object by number.
#[derive(Clone, Copy, PartialEq, Eq, Debug)]
struct Obj(u32);
Expand Down
2 changes: 1 addition & 1 deletion kernel/src/bootimage.rs
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
//! [`objects`] makes each one a [`SharedMemObject`] over its pages
//! ([`SharedMemObject::boot`]), whose handles never carry `WRITE`
//! (`ops::initial_rights`, and every part of one inherits its source's
//! rights), so every mapping of it is read-only; `sys_shm_map` asserts it.
//! rights), and no mapping of it is writable (`SharedMemObject::mappable`).

use alloc::sync::Arc;

Expand Down
74 changes: 51 additions & 23 deletions kernel/src/loader/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ use crate::sync::Lock;
use crate::{scheduler, UserAddr};
use toyos_abi::handle::Rights;
use crate::object::shm::SharedMemObject;
use toyos_abi::syscall::{SpawnMap, SyscallError};
use toyos_abi::syscall::{SpawnBacking, SpawnMap, SyscallError};
use toyos_symbols::frame::BuildId;

/// The main thread's stack: committed whole, populated as it is touched.
Expand All @@ -49,46 +49,73 @@ const USER_STACK_SIZE: usize = 8 * 1024 * 1024;
const STACK_GUARD: u64 = 1024 * 1024;

/// One range of a child's memory, its numbers checked where the request was
/// read (`syscall::proc::spawn_program`): whole pages, and a copy's bytes inside
/// an object of ordinary memory.
/// read (`syscall::proc::spawn_program`): whole pages, of ordinary memory
/// where an object backs it. What a shared range's object gives is checked as
/// it is mapped (`SharedMemObject::mappable`).
pub struct Map {
pub at: UserAddr,
pub len: u64,
pub prot: kernel::vm::Prot,
/// The object its pages start as a copy of, from which page boundary, and
/// how many bytes; zeros past them, and `None` for zeros alone.
pub copy: Option<(Arc<dyn CopySource>, u64, u64)>,
pub backing: Backing,
}

/// What a range's pages are.
pub enum Backing {
Zeros,
/// Private, each starting as a copy of the object's from this page
/// boundary for this many bytes, and zeros past them.
Copy(Arc<dyn CopySource>, u64, u64),
/// The object's own, from this page boundary.
Share(Arc<SharedMemObject>, u64),
}

impl Map {
/// The rights the handle naming `raw`'s object must carry: `MAP`, and
/// `WRITE` beside it for a writable share. `InvalidArgument` for a word
/// that names no protection or backing.
pub fn rights(raw: &SpawnMap) -> Result<Rights, SyscallError> {
let prot = crate::syscall::vm::vm_prot(raw.prot)?;
match SpawnBacking::from_u64(raw.backing).ok_or(SyscallError::InvalidArgument)? {
SpawnBacking::Copy => Ok(Rights::MAP),
SpawnBacking::Share => Ok(crate::syscall::vm::map_rights(prot)),
}
}

/// `raw`, checked as an ordinary mapping is: whole pages of the user half,
/// a protection, and a copy only of ordinary memory `object` holds from a
/// page boundary, `object` being `None` for zeros.
/// a protection, and an object of ordinary memory from a page boundary,
/// `None` for zeros.
pub fn checked(raw: &SpawnMap, object: Option<Arc<SharedMemObject>>) -> Result<Map, SyscallError> {
let page = crate::mm::PAGE_SIZE;
let prot = crate::syscall::vm::vm_prot(raw.prot)?;
let backing = SpawnBacking::from_u64(raw.backing).ok_or(SyscallError::InvalidArgument)?;
let at = UserAddr::checked(raw.addr).filter(|at| at.raw().is_multiple_of(page));
let ends = raw.addr.checked_add(raw.len).is_some_and(|end| end <= toyos_userbound::USER_TOP);
let (Some(at), true) = (at, ends && raw.len != 0 && raw.len.is_multiple_of(page) && raw.copied <= raw.len) else {
return Err(SyscallError::InvalidArgument);
};
let copy = match object {
None if raw.copied == 0 => None,
None => return Err(SyscallError::InvalidArgument),
Some(_) if !raw.offset.is_multiple_of(page) => return Err(SyscallError::InvalidArgument),
// Ordinary memory alone: a device aperture is no program, and a
// read of one is a device access. Each page is copied once, at
// its first touch, so a holder that writes the object later
// changes only pages not yet read: its own child's.
Some(object) => {
let backing = match (object, backing) {
(None, SpawnBacking::Copy) if raw.copied == 0 => Backing::Zeros,
(None, _) => return Err(SyscallError::InvalidArgument),
// Ordinary memory alone: a device aperture is no program, a read
// of one is a device access, and its last handle takes it back
// only from where its mapper said it is.
(Some(object), _) if !raw.offset.is_multiple_of(page) || !object.is_ordinary() => {
return Err(SyscallError::InvalidArgument);
}
// Each page is copied once, at its first touch, so a holder that
// writes the object later changes only pages not yet read: its
// own child's.
(Some(object), SpawnBacking::Copy) => {
let held = raw.offset.checked_add(raw.copied).ok_or(SyscallError::InvalidArgument)?;
if held > object.size() || !object.is_ordinary() {
if held > object.size() {
return Err(SyscallError::InvalidArgument);
}
Some((object as Arc<dyn CopySource>, raw.offset, raw.copied))
Backing::Copy(object as Arc<dyn CopySource>, raw.offset, raw.copied)
}
(Some(_), SpawnBacking::Share) if raw.copied != 0 => return Err(SyscallError::InvalidArgument),
(Some(object), SpawnBacking::Share) => Backing::Share(object, raw.offset),
};
Ok(Map { at, len: raw.len, prot, copy })
Ok(Map { at, len: raw.len, prot, backing })
}
}

Expand All @@ -112,9 +139,10 @@ fn lay(space: &mut crate::mm::space::AddressSpace, maps: &[Map]) -> Result<(), S
return Err(SyscallError::InvalidArgument);
}
let at = At::Fixed(map.at);
match &map.copy {
Some((source, offset, copied)) => space.copy(map.len, at, map.prot, Arc::clone(source), *offset, *copied)?,
None => space.reserve(map.len, at, map.prot)?,
match &map.backing {
Backing::Zeros => space.reserve(map.len, at, map.prot)?,
Backing::Copy(source, offset, copied) => space.copy(map.len, at, map.prot, Arc::clone(source), *offset, *copied)?,
Backing::Share(object, offset) => space.map(Arc::clone(object), *offset, map.len, map.prot, at)?,
};
}
Ok(())
Expand Down
Loading