Darwin/x86

The x86 port of Darwin languished for many years before the release of Intel Macs. Even the OpenDarwin and GNU/Darwin projects never really made Darwin a first class citizen in the world of x86 operating systems.

Part, but not all, of the blame for this lies with Apple. It seems that Apple considers Darwin not to be a full-fledged operating environment but a way to be transparent about the inner-workings of OS X. In that respect, Darwin is a resounding success. For good developers, the availability of source code can make a big difference when fighting bugs and misfeatures in the core operating system.

That said, Apple traditionally did usable binary releases with every major Darwin version. Spinning a binary release of anything is very non-trivial and Apple's work on this should be commended. With the release of Darwin 9.0 (corresponding to OS X Leopard) Apple has stopped spinning binary releases. So for the time being, Darwin is, as shipped, only a way to examine the inner-workings of OS X.

But Darwin doesn't have to be that way. It is indisputable that the x86 port of Darwin is now the premier port. It should also be the premier OS for commodity hardware, not just Apple's hardware. Kudos to Apple for continuing to release a large portion of their work to the community under a very liberal source license.

Please note that I'm not trying to start another OpenDarwin project. I am not looking to maintain a never-ending abyss of software suitable for Darwin. If you want a working Darwin system, you can buy one from Apple. If you want open source software to run on it you should check out the OpenDarwin spinoff now known as MacPorts (formerly DarwinPorts)

My main motivation here is to learn about the OS I spend most of my time with. A secondary motivation is to introduce other developers to the Darwin system. The more I learn about Darwin, the more I like it. It all started with me just using OS X, then moving on to developing with Cocoa in Objective-C, then WebObjects in Java. Now I've familiarized myself with IOKit. All of these systems have a very important common thread. Namely, all of them are heavily object oriented and make heavy use of dynamically-typed property dictionaries.

Yes, even in the kernel, the Apple guys use OO design. It's almost like you're writing Python or Ruby except that you're writing Objective-C (Cocoa) or Java (WebObjects) or C++ (IOKit).

The State of the Software

At this point, it is impossible to run a recent release of Darwin on a non-Apple system using the source as shipped by Apple. A non-Apple system includes VMware, even when running on a real Mac. But that does not mean the source cannot be modified to support this.

It's no secret that not only Darwin but the full release of OS X has been modified to run on non-Apple hardware. So a good starting point for me was to analyze what had been done. It was a good learning experience but proper coding techniques do not mesh well with binary modification techniques.

The first observation was that most of the modifications focused on the kernel (xnu). But what modifications were really needed? Which could be done better? It turns out that xnu is highly dependent on its bootloader. More information can be found on the bootloader page but the short story is that the last x86 bootloader that Apple shipped (boot-132) is nearly up to modern xnu requirements, but not quite. With small modifications it gives xnu all of the information it needs to start the system, including an EFI system table, runtime services table, and stub implementations of the runtime services functions.

The bootloader modifications play a large role in the ability to run Darwin on non-Apple systems but it is not the end of the story, only the beginning. The modified bootloader will boot xnu on Apple and Apple-equivalent hardware. That is, the system must still have an ICH-based chipset and a Core-based processor.

Obviously not all systems are like that. For instance, VMware virtualizes a PIIX4 chipset. Aside from that, VMware doesn't fully virtualize the CPU. The IA32_PERF_STAT (or MSR_PERF_STAT) MSR which contains the maximum bus ratio of the processor simply returns 0. When this value is used as the denominator of an integer division, the kernel crashes. Oops.

Then there are things like PS/2 support. VMware does have USB support but it doesn't emulate a USB keyboard and does not provide a way to assign a real USB keyboard directly to the virtual machine. So clearly the guest OS must support a PS/2 keyboard if you'd like to be able to type anything. Ordinarily I'd suggest punting and using the serial port console except VMware Fusion, unlike other VMware products, doesn't emulate serial ports except for output to a text file.

Beyond that you might like to have a hard drive so either an IDE or a SCSI host bus and block storage driver is going to be a requirement. OS X includes AppleIntelPIIXATA.kext which, as the name implies, still supports older PIIX4 chipsets although it does require you to add the device/vendor IDs from an older version of it's Info.plist (which is part of the Darwin 8 binary release).

Right now my work on this is very preliminary and still requires Apple binaries to function because Apple has never released source for certain critical kernel extensions and they have chosen not to freely license binaries for use with Darwin 9. The older Darwin 8 binaries generally do not work because modern x86 Darwin has an incompatible ABI to older x86 Darwin.

However, if you do have an (additional) OS X license for your virtual machine, you can run Darwin (but not yet full-blown OS X) in VMware using the binary kernel extensions borrowed from OS X. To be compliant with the OS X license as I read it, you must do so on a real Mac. Obviously that limits the current developer base to people with an additional OS X license and a copy of VMware Fusion so I'm hoping to come up with a pure source build in the future. That should open up the developer pool to anyone with an x86 machine and VMware (even Player), then hopefully later to any x86 machine.

My overall intention here is to increase Darwin compatibility without breaking Darwin on Apple hardware. Obviously Apple hardware is Apple's primary focus so any code modifications need to be mindful of this. Hacks that remove hpet_init don't make for good patches.

The fruits of this labor will be made available on this website and Apple will be notified of progress.

Current Projects

boot - The Darwin/x86 Bootloader
xnu - The kernel
Extensions - Various useful kernel extensions