memory.library — Virtual Memory on mmu.library¶

memory.library is the virtual-memory layer of Thomas Richter's MMULib. It sits on top of mmu.library and provides memory "that can be swapped to disk transparently to the application", so a program can allocate more memory than is physically present. Like mmu.library it is a third-party component, not part of AmigaOS 3.2; its developer documentation is the autodoc MuManual/Autodocs/memory.doc in the NDK 3.2 MuManual/ tree.

The approach is deliberately conservative. In the autodoc's words it is "some kind of a 'poor man's solution'": no exec function is patched, old programs keep working unchanged, and only programs that call the memory.library functions get virtual memory. A system-wide "VMM/GigaMem" built on it is described as planned and optional, because it "CAN'T be fully compatible to old programs".

Status and what the NDK contains¶

The MuManual/ReadMe history shows memory.library arriving late in the MMULib project: the "first release of the memory.library" is announced in 43.2, "a first version for the memory.library documentation" in 43.4.2, and the library "got tested now on the 030,040 and 060" in 43.3. The same file warns about its example programs (vmem, mmap): "DO NOT YET WRITE PROGRAMS FROM THESE EXAMPLES", because they do not yet build a private MMU context as recommended.

The autodoc refers to memory/memtags.h and memory/memerrors.h for tag values and error codes. Those headers, and prototypes or LVOs for memory.library, are not in the NDK's MuManual/Include tree. This article therefore gives tag names and semantics only. Take the numeric values from the headers of the MMULib distribution you build against.

The two objects: AddressSpace and VMPool¶

Both objects are opaque "magic cookies", like struct MMUContext.

AddressSpace (struct AdrSpace) — an extension of an MMUContext that gives addresses a meaning for every task that is part of it. As with mmu.library, an exec task corresponds to a Unix "thread" and an AddressSpace to a "process". Tasks enter and leave address spaces explicitly.

Each MMUContext holds at most one AddressSpace.
Do not attach an address space to the public context or to a supervisor context. Since 43.5.1 the library refuses both.
The autodoc "strongly encourage[s]" building a private MMUContext of the shared type (MCXTAG_SHARE) for a new address space.

VMPool (struct VMPool) — the counterpart of an exec memory pool, living inside an AddressSpace. An AddressSpace can hold any number of pools, each with its own memory flags, cache mode and optionally its own swap mechanism. A pool can either supply memory for AllocVMemory() / FreeVMemory(), or mirror a whole file in memory, which is "similar to the Un*x mmap() feature".

Access rules for virtual memory¶

These come from memory.doc --Background-- and are the main difference from AllocMem() memory:

No access under Forbid() or Disable(). Touching a swapped-out page then "may certainly cause a fatal 'Guru' of the Bus-Error type". AllocVMemory() and FreeVMemory() must not be called in that state either.
Not shareable. Only tasks that entered the pool's AddressSpace can access it, so never use virtual memory for OS structures passed to other tasks. Use AllocMem() for those.
No access when file I/O is impossible: while the swap partition's file system is inhibited, while the swap device is unavailable, or after taking over the hardware. The library tries to detect this and guru; otherwise the result is a deadlock.

LockMemory() and HoldMemory() relax rule 1 for specific ranges (see below).

Function overview¶

Prototypes and registers are as given in memory.doc; obvious typos in its synopses (a missing *, an untyped size) are corrected here.

Function	Registers	Purpose
`struct AdrSpace NewAdrSpaceA(struct TagItem )`	a0	create an address space (varargs `NewAdrSpace()`)
`void DeleteAdrSpace(struct AdrSpace *)`	a0	delete it, its pools and swap hooks
`struct MMUContext MMUContextOf(struct AdrSpace )`	a0	context of an address space; cannot fail
`struct AdrSpace AdrSpaceOfCtx(struct MMUContext )`	a0	address space of a context, or NULL
`struct VMPool NewVMPoolA(struct TagItem )`	a0	create a pool (varargs `NewVMPool()`)
`void DeleteVMPool(struct VMPool *)`	a0	delete a pool
`BOOL LockMemory(struct AdrSpace *, APTR, ULONG size)`	a0, a1, d0	swap in and pin a range; nests
`void UnlockMemory(struct AdrSpace *, APTR, ULONG size, BOOL force)`	a0, a1, d0, d1	release a lock
`APTR HoldMemory(struct AdrSpace *, APTR, ULONG size)`	a0, a1, d0	pin a range as one contiguous physical block; nests
`void UnholdMemory(struct AdrSpace *, APTR, ULONG size, BOOL force)`	a0, a1, d0, d1	release a hold
`BOOL SwapMemoryOut(struct AdrSpace *, APTR mem, ULONG size)`	a0, a1, d0	swap a range out on demand
`APTR AllocVMemory(struct VMPool *, ULONG)`	a0, d0	allocate virtual memory
`void FreeVMemory(struct VMPool *, APTR mem, ULONG size)`	a0, a1, d0	free it; size must match
`APTR PoolVBase(struct VMPool *)`	a0	logical base of a non-administrated pool
`ULONG PoolVSize(struct VMPool *)`	a0	size of a non-administrated pool
`BOOL EnterAddressSpace(struct AdrSpace adr, struct Task task)`	a0, a1	attach a task
`BOOL LeaveAddressSpace(struct Task *)`	a1	detach a task, back to the public context

Failing calls made from a dos.library process set IoErr() to a secondary error code (dos/dos.h or memory/memerrors.h).

Creating an address space: NewAdrSpaceA()¶

NewAdrSpaceA() neither attaches a task nor creates a pool. You must call NewVMPool() and EnterAddressSpace() yourself. Tags and their data must be in shared system memory.

General tags¶

Tag	Meaning / default
`MEMTAG_CONTEXT`	the `MMUContext` to extend; default: the context of `MEMTAG_TASK`
`MEMTAG_TASK`	pick the context through this task; default: current task
`MEMTAG_SWAPFAILHOOK`	`struct Hook *` called when swapping in or out fails
`MEMTAG_VMSIZE`	size of the virtual area (excluding shared system memory)
`MEMTAG_VMMAXSIZE`	upper limit for that size
`MEMTAG_MAXSYSMEM`	maximum physical memory held before pages are swapped out; default half the system memory minus a margin. Locks may exceed it temporarily
`MEMTAG_CACHESIZE`	write cache for swapped-out pages; default 16 pages (none with `MEMTAG_READONLY`)
`MEMTAG_WINDOWPTRPTR`	`struct Window **` for error requesters; `~0` suppresses them, NULL means Workbench
`MEMTAG_VMHOOK`	custom swap hook (`struct Hook *`)
`MEMTAG_VMSWAPTYPE`	built-in swap hook (preferred): see below

Either MEMTAG_VMHOOK or MEMTAG_VMSWAPTYPE must define the swap mechanism.

Swap types¶

`MEMTAG_VMSWAPTYPE` value	Swap target	Autodoc verdict
`MEMFLAG_SWAPFILE`	a dos.library file	"slow for the FFS, but is the least critical"
`MEMFLAG_SWAPPART`	a dos.library device (partition)	"best compromise between speed and safety"
`MEMFLAG_DEVICE`	an exec device and unit	trackdisk-like, random access

File swapping tags: MEMTAG_SWAPFILENAME (required), MEMTAG_SWAPFILELOCK (directory lock, default current directory), MEMTAG_KEEPFILE (default FALSE: the file is deleted on shutdown), MEMTAG_PREDEFINED (the file exists and its contents are mirrored), MEMTAG_READONLY (needs MEMTAG_PREDEFINED; changes are discarded).

Partition tags: MEMTAG_SWAPDEVICENODE (struct DeviceNode *) or MEMTAG_SWAPPARTNAME (e.g. "SWAP:", colon optional).

Device tags: MEMTAG_DEVICENAME (required), MEMTAG_DEVICEUNIT (default 0), MEMTAG_DEVICEFLAGS (default 0), MEMTAG_BUFMEMTYPE (bounce buffer type, default MEMF_24BITDMA), MEMTAG_MASK (default 0x00fffffc; "A well-written device driver will accept 0xffffffff"), MEMTAG_MAXTRANSFER (default 0x001fe000), MEMTAG_DEVICESIZE (limited to 2GB), MEMTAG_DEVICEORIGIN (byte offset of the swap area, as a pointer to a two-ULONG 64-bit value; large offsets need a TD64 or NSD compliant device, with NSDPatch running for NSD), MEMTAG_SECTORSIZE (default 512). The swap hooks always pass physical, never logical, addresses to the device.

Known limitation (BUGS): the device and partition hooks need a device block size that is a multiple of the page size. That rules out 256-byte pages (possible on the 68030 and 68020/68851) and block sizes that are not a power of two.

The autodoc's two reference setups:

/* mirror a file into memory ("mmap") */
adr = NewAdrSpace(MEMTAG_VMSWAPTYPE,   MEMFLAG_SWAPFILE,
                  MEMTAG_SWAPFILENAME, "MyMMAPTarget",
                  MEMTAG_KEEPFILE,     TRUE,
                  MEMTAG_PREDEFINED,   TRUE,
                  TAG_DONE);

/* classical swap partition */
adr = NewAdrSpace(MEMTAG_VMSWAPTYPE,   MEMFLAG_SWAPPART,
                  MEMTAG_SWAPPARTNAME, "SWAP:",
                  MEMTAG_VMSIZE,       128*1024*1024,   /* e.g. 128MB */
                  TAG_DONE);

DeleteAdrSpace() disposes all pools and closes the swap hooks (possibly deleting swap files), but does not remove tasks from the MMUContext or delete the context. Do that yourself.

Creating a pool: NewVMPoolA()¶

The pool's address space is selected by MEMTAG_ADRSPACE, MEMTAG_CONTEXT (a context with an address space) or MEMTAG_TASK (default: current task). Giving MEMTAG_VMHOOK or MEMTAG_VMSWAPTYPE gives the pool a private swap mechanism, and all swap tags of NewAdrSpaceA() apply. A private swap hook requires MEMTAG_FIXEDSIZE and MEMTAG_PREDEFINED set to TRUE.

Tag	Meaning / default
`MEMTAG_PREALLOC`	allocate the full size on setup; required to map a file. Default FALSE
`MEMTAG_MEMFLAGS`	exec memory type of the physical pages; default `MEMF_ANY`
`MEMTAG_CACHEFLAGS`	`mmu/context.h` properties: `MAPP_COPYBACK`, `MAPP_CACHEINHIBIT`, `MAPP_NONSERIALIZED` (68040), `MAPP_IMPRECISE` (68060), `MAPP_ROM`, `MAPP_WRITEPROTECTED`
`MEMTAG_CACHEMASK`	which cache flags to change; the library may override some
`MEMTAG_PUDDLESIZE`	puddle size, at least one MMU page; default eight pages
`MEMTAG_PUDDLETHRES`	allocations above this get their own puddle; default half the puddle size
`MEMTAG_POOLPRI`	higher-priority pools are swapped out later; default 0
`MEMTAG_FIXEDSIZE`	no new puddles; only useful with `MEMTAG_PREALLOC`
`MEMTAG_PROVIDESMEM`	TRUE (default): an `AllocVMemory()` pool. FALSE for an "mmap" mirror
`MEMTAG_PREDEFINED`	contents come from the swap space ("mmap")
`MEMTAG_READONLY`	changes are not written back; requires `MEMTAG_PROVIDESMEM` FALSE

The puddling "works not like that of exec.library. Do not expect any exec.library compatible memory pools." (see exec Memory Pools for the exec scheme). Since 43.3 the library uses its own "best fit" plus "buddy chunk" allocator instead of exec's.

The autodoc's two pool recipes:

/* "mmap": map the swap file of adr into memory */
vmpool = NewVMPool(MEMTAG_ADRSPACE,    adr,
                   MEMTAG_PREDEFINED,  TRUE,
                   /* MEMTAG_READONLY, TRUE, for read-only files */
                   MEMTAG_PREALLOC,    TRUE,
                   MEMTAG_PROVIDESMEM, FALSE,
                   MEMTAG_FIXEDSIZE,   TRUE,
                   TAG_DONE);
base = PoolVBase(vmpool);       /* file image starts here      */
size = PoolVSize(vmpool);       /* multiple of the MMU page size */

/* general virtual memory */
vmpool = NewVMPool(MEMTAG_ADRSPACE,    adr,
                   MEMTAG_PROVIDESMEM, TRUE,
                   TAG_DONE);

PoolVBase() and PoolVSize() work only on non-administrated pools (MEMTAG_PROVIDESMEM, FALSE plus MEMTAG_PREDEFINED, TRUE). For an mmap pool the size may exceed the file size because of page granularity. DeleteVMPool() on a predefined pool writes all changes back before closing the swap hook.

MEMF_CLEAR on a VM pool "might be very expensive", since clearing can force swapping.

Allocating and freeing¶

AllocVMemory(pool, bytesize) may reschedule and swap. It fails in Forbid()/Disable() and requires the caller to have entered the pool's AddressSpace. The result is at least long-word aligned. FreeVMemory(pool, mem, bytesize) accepts NULL; the size must match the allocation, and calling it under Forbid()/Disable() gives "a bus error 'Guru 80000002' or a MuForce hit".

Pinning memory: Lock vs. Hold¶

	`LockMemory()`	`HoldMemory()`
Swapped in and pinned	yes	yes
Accessible under `Forbid()`/`Disable()`	yes	yes
Physically contiguous	no, may be fragmented	yes
Returns	`BOOL`	physical address or NULL
Cost	lower	"relatively costy", may move data
Release	`UnlockMemory()`	`UnholdMemory()`

Both nest, round mem down and size up to page multiples, and may be used on an address space other than the caller's. Use HoldMemory() when the block must go to classic DMA I/O. Keep locks short, because the whole range must fit in main memory. The force argument of the release calls (TRUE) drops all nesting counts at once; use it "with care". SwapMemoryOut() leaves locked or held parts in memory; on failure the swap-fail hook has already been called.

Entering and leaving¶

if (EnterAddressSpace(adr, FindTask(NULL))) {
    APTR mem = AllocVMemory(vmpool, 4*1024*1024);
    if (mem) {
        /* use mem - never inside Forbid()/Disable() */
        FreeVMemory(vmpool, mem, 4*1024*1024);
    }
    LeaveAddressSpace(FindTask(NULL));
}

EnterAddressSpace() implicitly enters the AddressSpace's MMUContext, so it takes over the task's tc_Switch and tc_Launch vectors, like EnterMMUContext(). After LeaveAddressSpace() the task is back in the public context and must not touch virtual memory any more. Calling it for a task not in any address space returns FALSE and does nothing else.

How it uses mmu.library¶

From the mmu.library side:

Swapped-out pages are marked MAPP_SWAPPED with a MAPTAG_BLOCKID that tells the swapper where the page lives.
AddMessageHook() "is used by the memory.library to install its exception handler. The port will be in this case the port of the swapper daemon that loads swapped out pages from disk."
GetPageUsedModified() (V43) exists "to drive the state machine of a virtual memory system" by reading and clearing the used/modified bits atomically. It needs MAPP_SINGLEPAGE pages.
MGXTAG_ADDRESSSPACE in GetMMUContextData() is "Reserved for the memory lib".
AllocAligned() is "a service routine for the memory.library".
Swapping is ruled by the CPU caveats in exceptions.doc: on the 68040 a VM system "MUST NOT swap out a page previously swapped in. At least two pages must be available at once."

Keys	Action
`?`	Open this help
`n`	Next page
`p`	Previous page
`s`	Search