Building XNU for macOS Big Sur 11.0.1 (Intel)

 The macOS Big Sur 11.0.1 kernel (XNU) source has been released here: source ,  tarball.

A lot of things have changed in Big Sur, and the installation and running of a custom kernel is significantly different from previous OS releases. This post covers building a kernel for x86_64 (Intel) platforms. It is not currently possible to build open source XNU for Apple Silicon Macs.

Building XNU requires some patience, and some open source dependencies which are not pre-installed. This post walks through all the steps necessary to build the open source version of XNU and boot it on supported Apple Intel hardware.

TL;DR

UPDATE: A previous version of the Makefile incorrectly compiled libfirehose_kernel. The version available now has corrected this. The md5sum of the correct Makefile is: f6f2ec28eb09e00fdcba9c332b7fb91a

I have updated the makefile which automates the downloading and building of all prerequisites. The file now supports Big Sur  x86_64 kernels, and the new default macOS version is 11.0.1. You can manually grab it like:

• curl https://jeremya.com/sw/Makefile.xnudeps > Makefile.xnudeps

and invoke it like:

• make -f Makefile.xnudeps

This makefile will automatically detect the correct versions of source code to download based on the version of macOS you specify. By default, the version is 11.0.1, however you can select a different version like:

• make -f Makefile.xnudeps macos_version=10.15.3 xnudeps

You might need a different version of Xcode installed when building for previous OS versions. You can also see other features of the Makefile using the  help  target.

Setup Xcode

If you have not downloaded and installed Xcode, you will first need to do a small bit of setup to be able to use the command line tools. The steps to install and setup a specific version of Xcode are as follows:

1. Download the xip package from developer.apple.com/downloads/more.  The particular version you select may depend on what version of XNU you want to compile. You will need a developer account to download Xcode this way.
2. Unpack the xip package by double clicking on it, or using the command line:
   $ xip --expand Xcode_{ver}.xip
   
   
3. Select the new xcode:
   $ sudo xcode-select -s path/to/Xcode.app/Contents/Developer
   
   
4. Agree to the license:
   $ sudo xcodebuild -license
   
   
5. Make sure it works:
   $ xcrun -sdk macosx -show-sdk-path
   $ clang -v

Manual XNU Building

1. Download and Install Xcode
   
• Make sure you have at least Xcode 12.2 installed. You can install it via the App Store, or by manual download here:  https://developer.apple.com/download/more/
• NOTE: for older versions of macOS, you may need older versions of Xcode which are only available via download from developer.apple.com. You need a developer account to download from that site.
2. Download the source
• export TARBALLS=https://opensource.apple.com/tarballs
• curl -O ${TARBALLS}/dtrace/dtrace-370.40.1.tar.gz
• curl -O ${TARBALLS}/AvailabilityVersions/AvailabilityVersions-69.2.tar.gz
• curl -O ${TARBALLS}/libdispatch/libdispatch-1271.40.12.tar.gz
• curl -O ${TARBALLS}/xnu/xnu-7195.50.7.100.1.tar.gz
3. Build CTF tools from dtrace
• tar zxf dtrace-370.40.1.tar.gz
• cd dtrace-370.40.1
• xcodebuild install -sdk macosx -target ctfconvert \
  -target ctfdump -target ctfmerge \
  DSTROOT=$PWD/dst
• export TOOLCHAIN=`cd $(xcrun -sdk macosx -show-sdk-platform-path)/../../Toolchains/XcodeDefault.xctoolchain && pwd`
• sudo ditto "$PWD/dst/$TOOLCHAIN" "$TOOLCHAIN"
• cd ..
4. Install AvailabilityVersions
• tar zxf AvailabilityVersions-69.2.tar.gz
• cd AvailabilityVersions-69.2
• make install
• sudo ditto "$PWD/dst/usr/local/libexec" \
  "$(xcrun -sdk macosx -show-sdk-path)/usr/local/libexec"
• cd ..
5. Install XNU headers
• tar zxf xnu-7195.50.7.100.1.gz
• cd xnu-7195.50.7.100.1
• make SDKROOT=macosx ARCH_CONFIGS=X86_64 installhdrs
• sudo ditto "$PWD/BUILD/dst" "$(xcrun -sdk macosx -show-sdk-path)"
• cd ..
6. Build libfirehose from libdispatch
• tar zxf libdispatch-1271.40.12.tar.gz
• cd libdispatch-1271.40.12
• xcodebuild install -sdk macosx ARCHS=x86_64 VALID_ARCHS=x86_64 \
  -target libfirehose_kernel PRODUCT_NAME=firehose_kernel \
  DSTROOT=$PWD/dst
• sudo ditto "$PWD/dst/usr/local" \
  "$(xcrun -sdk macosx -show-sdk-path)/usr/local"
• cd ..
7. Build XNU
• cd xnu-7195.50.7.100.1
• make SDKROOT=macosx ARCH_CONFIGS=X86_64 KERNEL_CONFIGS=RELEASE
• (you can speed up the link by adding BUILD_LTO=0 to your xnu make invocation)
• (build the development kernel by setting KERNEL_CONFIGS=DEVELOPMENT in your xnu make invocation)
• (see colorful build output by adding LOGCOLORS=y to your xnu make invocation)
• (make the build output stay on a single line by adding CONCISE=1 to your xnu make invocation)

Check out the README.md  file at the top of the XNU source tree for more options to the build system.

Install and Run XNU

SECURITY WARNING: On Big Sur, you will need to  disable System Integrity Protection , set the machine's Secure Boot security setting to "No Security," and  disable the authenticated root volume in order to install and run a custom kernel.

After the final build step, you should have a new kernel built in {xnu}/BUILD/obj/kernel. In order to run this kernel on Big Sur, you will need to rebuild the kext collection (KC) artifacts (the new boot artifact format in Big Sur). The KCs can be generated using the kmutil command, specifically the create subcommand, which allows you to link custom KCs necessary to boot the system. For more information on the new boot artifact format, check out the kmutil man page:  man kmutil

Installing a kernel could potentially render your system un-bootable, so trying this out in a VM first is recommended!

Install and Run Your kernel

1. Build the KCs
• cd xnu-7195.50.7.100.1
• kmutil create -a x86_64 -Z -n boot sys \
  -B BUILD/BootKernelExtensions.kc \
  -S BUILD/SystemKernelExtensions.kc \
  -k BUILD/obj/kernel \
  --elide-identifier com.apple.driver.AppleIntelTGLGraphicsFramebuffer
  
• (the AppleIntelTGLGraphicsFramebuffer will not link against the open source xnu kernel)
2. Mount a live view of the filesystem
• mkdir BUILD/mnt
• sudo mount -o nobrowse -t apfs \
  /dev/diskMsN "$PWD/BUILD/mnt"
• (diskMsN can be found by running mount, looking for the root mount's device, and chopping off the last s, e.g. if your root is /dev/disk1s2s3, you'll mount /dev/disk1s2)
3. Place the kernel+KCs into the live volume
• (I recommend using a custom suffix in order to maintain a fallback from which you can recover your system)
• sudo ditto BUILD/BootKernelExtensions.kc "$PWD/BUILD/mnt/System/Library/KernelCollections/BootKernelExtensions.kc.development"
• sudo ditto BUILD/SystemKernelExtensions.kc "$PWD/BUILD/mnt/System/Library/KernelCollections/SystemKernelExtensions.kc.development"
• sudo ditto BUILD/obj/kernel "$PWD/BUILD/mnt/System/Library/Kernels/kernel.development"
• (note that "development" can be replaced with any short string, e.g., "usr")
4. Bless the new KCs (copy them to the appropriate Preboot volume)
• sudo bless --folder $PWD/BUILD/mnt/System/Library/CoreServices \
  --bootefi --create-snapshot
5. Setup boot-args to select the new KC
• sudo nvram boot-args="kcsuffix=development wlan.skywalk.enable=0"
• (replace development with your chosen suffix)
• (the wlan.skywalk.enable=0 boot-arg is necessary to disable the use of skywalk in the WLAN driver as Skywalk is not part of the open source kernel)
• If you run into panics related to Skywalk, you may also need to add the dk=0 boot-arg to disable DriverKit drivers. Networking DriverKit drivers are Skywalk clients.
6. Reboot!
• NOTE: due to missing network (Skywalk) and power management (XCPM) functionality, your machine will be missing some features. For example, sleep/wake will not work.

There are a few things to be aware of in this new kernel / kext management world.

1. If your machine becomes un-bootable, you can boot into Recovery Mode (by holding down the option key and pressing Cmd-r at the boot picker prompt), opening up terminal, and setting your kcsuffix boot-arg to release (or removing the kcsuffix= boot arg). If you used a suffix for your KCs, this will boot from the KCs originally supplied by Apple.
2. The KC filenames are important. The booter will load the file named BootKernelExtensions.kc{.suffix} into memory, and the kernelmanagerd user space daemon will attempt to mmap a file named SystemKernelExtensions.kc{.suffix} (where the suffix corresponds to the booted KC).
3. The boot and system KCs must be generated together because the system KC is linked against the boot KC. The system will panic if there is a mis-match.
4. The bless step is required because the booter only looks in the Preboot volume for KCs, and the bless invocation places the boot KC in the correct place in the Preboot volume.
5. By disabling the authenticated root volume, and booting from a new snapshot, you open your system up to evil maid attacks.
6. In order to re-enable all of the security on the machine, you will need to go through a software update (an update to the current system version should work).

It may also be useful to use the following boot-args to see serial output from the kernel as it boots:

• serial=3 -v