Thursday, February 14, 2008

DNA of xnu

Mach
Mach can considered as the core of xnu. Mach provides critical low-level services that are transparent to applications. It was used as a true microkernel in version 3. Before that it had monolithic implementation in which BSD and Mach resided in the same kernel address space. In Mac OS X, xnu does not not use Mach as a traditional microkernel - various subsystems that would be implemented as user-space servers in a true microkernel system are part of the kernel proper in Mac OS X. In particular, the BSD portion of xnu, the I/O Kit, and Mach are all reside in the same address space.

System aspects that Mach is responsible for include the following:
  • Hardware abstraction to some extent
  • Process management, including symmetric multiprocessing and scheduling
  • Preemptive multitasking, including support for tasks and threads
  • Virtual memory management, including low-level paging, memory protection, sharing, and inheritance
  • Low-level IPC mechanisms that are the basis for all messaging in the kernel
  • Real-time support that allows time-sensitive applications to have latency-bounded access to processor resources
  • Kernel debugging support
  • Console I/O
Started with Mac OS X 10.4, xnu added support for 64-bit processes on 64-bit hardware. The upper limit of the process virtual address spaces was 18 exabytes (10^18). Prior to 10.4, process virtual spaces was 32-bit.

BSD
The xnu kernel contains a substantial amount of BSD-derived code. But is is not the case that a well-defined BSD kernel runs within xnu. The BSD-derived portions were made to coexist with non-BSD entities such as I/O Kit and Mach. Some aspects that BSD-style code is responsible for include the following:
  • BSD-style process model
  • Signals
  • User IDs, permissions, and basic security policies
  • POSIX APIs
  • Asynchronous I/O APIs (AIO)
  • BSD-style system calls
  • TCP/IP stack, BSD sockets, and firewall
  • Network Kernel Extensions (NKEs) - type of kernel extension for making the BSD networking architecture fit into xnu
  • The virtual file system (VFS) layer and numerous file systems, including a file-system-independent VFS-level journaling mechanisms
  • System V and POSIX interprocess communication mechanisms
  • In-kernel cryptographic framework
  • A system notification mechanism based on FreeBSD's kqueue/kevent mechanism - a system-wide service enabling notifications between applications, and from the kernel to applications
  • The fsevents file system change notification mechanism that is used by the Spotlight search technology
  • Access Control Lists (ACLs) and the kauth authorization framework
  • Various synchronization primitives
In Mac OS X, a BSD process does not execute. It corresponds to a Mach task, which contains one or more Mach threads, and it is these threads execute. For example, the BSD-style fork() implementation in the kernel uses Mach calls to create a task. At the same time, it allocates and initializes a process structure that is associated with the task. Therefore, BSD process structure bind the UNIX and Mach together. Similarly, BSD's unified buffer cache (UBC) has a back-end that hooks into Mach's virtual memory subsystem. UBC allows the file system and the virtual memory subsystem to share kernel memory buffers. Unifying the buffer cache yields a single backing store for various entities, reducing disk accesses and the amount of wired memory used.

In addition to BSD system calls like sysctl() and ioctl(), Mac OS X uses Mach system calls (or Mach traps) when necessary. There are several ways to map memory, perform block-copy, and exchange information between user and kernel spaces.

I/O Kit
I/O Kit uses a restricted subset of C++ as it programming language. The C++ features that are not allowed include exceptions, multiple inheritance, templates, complicated constructors, initialization lists, and runtime type identification (RTTI). However, it does implement its own minimal RTTI system.

The I/O Kit's implementation consists kernel-resident C++ Libraries (libkern and IOKit) and a user-space framework (IOKit.framework). The Kernel.framework encapsulates the kernel-resident libraries in order to export their header files - the executable code for these libraries is contained in the kernel. The IOKit.framework is a conventional framework used for writing user-space programs that communicate with the I/O Kit. Other frameworks that form the basis of Mac OS X core are: Accelerate.framework, CoreFoundation.framework, CoreServices.framework.

I/O Kit presents abstractions of the underlying hardware to the rest of the system. The device driver model provided by the I/O Kit has the following features:
  • Extensive programming interfaces
  • Numerous device families such as ATA/ATAPI, FireWire, Graphics, HID, Network, PCI, and USB
  • Object-oriented abstractions of devices
  • Plug-and-play and dynamic device management ("hot-plugging")
  • Power management
  • Preemptive multitasking, threading, symmetric multiprocessing, memory protections, and data management
  • Dynamic matching and loading of drivers for multiple bur types
  • A database for tracking and maintaining detailed information on instantiated objects (the I/O Registry)
  • A database of all I/O Kit classes available on a system (the I/O Catalog)
  • Interfaces for applications and user-space drivers to communicate with the I/O Kit
  • Driver stacking
Therefore, standard devices such as mouse and keyboard that conform to well-defined and well-supported specifications do not require custom I/O Kit drivers.


libkern Library
The libkern library implements the runtime system for restricted subset of C++ used by the I/O Kit's programming model. It also defines the OSObject class which is the root base class for Mac OS X kernel. The following are examples of the functionality provided by libkern:
  • Dynamic allocation, construction, and destruction of objects with support for a variety of built-in object types such as Arrays, Booleans, and Dictionaries
  • Atomic operations and miscellaneous function such as bcmp(), memcmp(), and strlen()
  • Functions for byte-swapping
  • Provisions for tracking the number of current instances for each class
  • Mechanisms that help alleviate the C++ fragile base-class problem
libsa Library
libsa is an in-kernel support library which function like an in-kernel linker during early system startup for loading kernel extensions (standalone libraries exist on other operating system often with the name libstand). Mac OS X kernel extensions are normally loaded on demand through the kextd user-space daemon (/usr/libexec/kextd). libsa is removed from the kernel once kextd becomes available. Examples of specific functionality implemented by libsa include the following:
  • Simple memory allocation
  • Binary searching
  • Sorting
  • Miscellaneous string-handling functions
  • Symbol remangling
  • A dependency graph package used while determining kernel extension dependencies
  • Decompression of compressed kernels and verification of checksums