Wednesday, September 23, 2020

Building XNU for macOS Catalina 10.15.x

 The macOS Catalina 10.15 kernel (XNU) source has been released here: source ,  tarball.

It has been a while since I've updated this blog, so this post will cover building XNU for all current Catalina releases 10.15 - 10.15.6. Let's start with the standard disclaimer:
Building XNU requires some patience, and some open source dependencies which are not pre-installed. This post walks through all the steps necessary to build the open source version of XNU on supported Apple hardware.


Update

If you experience panics related to Skywalk, you may need to add wlan.skywalk.enable=0 to your boot-args.


TL;DR

I have updated the makefile which automates the downloading and building of all prerequisites. The file now supports all released 10.15.x kernels, and the new default macOS version is 10.15.6. You can manually grab it like:
  • curl https://jeremya.com/sw/Makefile.xnudeps > Makefile.xnudeps
and invoke it like:
  • make -f Makefile.xnudeps
This makefile will automatically detect the correct versions of source code to download based on the version of macOS you specify. By default, the version is 10.15.6, however you can select a different version like:
  • make -f Makefile.xnudeps macos_version=10.13.1 xnudeps
You can also see other features of the Makefile using the  help  target.

The default target of the makefile,  xnudeps, will perform all the necessary source and build/project fixups. However, if you want to re-download the source and perform the fixups, you can use the  download and xnu-fixups targets.


Setup Xcode

If you have not downloaded and installed Xcode, you will first need to do a small bit of setup to be able to use the command line tools. The steps to install and setup a specific version of Xcode are as follows:
  1. Download the xip package from developer.apple.com/downloads/more.  The particular version you select may depend on what version of XNU you want to compile. You will need a developer account to download Xcode this way.
  2. Unpack the xip package by double clicking on it, or using the command line:
    $ xip --expand Xcode_{ver}.xip
  3. Select the new xcode:
    $ sudo xcode-select -s path/to/Xcode.app/Contents/Developer
  4. Agree to the license:
    $ sudo xcodebuild -license
  5. Make sure it works:
    $ xcrun -sdk macosx -show-sdk-path
    $ clang -v


Manual XNU Building

All of the source for both XNU and required dependencies is available from  opensource.apple.com.  Each release of macOS open source code has a different set of dependencies. Please refer to the table below when downloading and building the source. The exact steps are given assuming you are building a kernel for macOS Catalina 10.15.6 - substitute different version numbers as appropriate (or use the makefile to automate the process). Unfortunately, this process does involve some code modifications to xnu to allow the build to succeed. If you are building manually, be careful to follow the instructions exactly!
Project 10.15 10.15.1 10.15.2 10.15.3 10.15.4 10.15.5 10.15.6
Xcode
(recommended)		 11.0
dtrace 338.0.1 338.40.5 338.100.1
AvailabilityVersions 45 45.3 45.5 45.7 45.9 45.11
libdispatch 1173.0.3 1173.40.5 1173.60.1 1173.100.2
libplatform 220 220.100.1
Libsystem 1281 1281.100.1
xnu 6153.11.26 6153.41.3 6153.61.1 6153.81.5 6153.101.6 6153.121.1 6153.141.1

Here are the manual steps necessary to build XNU:

  1. Download and Install Xcdoe
    • Make sure you have at least Xcode 11 installed. You can install it via the App Store, or by manual download here:  https://developer.apple.com/download/more/
    • NOTE: for older versions of macOS, you may need older versions of Xcode which are only available via download from developer.apple.com. You need a developer account to download from that site.
  2. Download the source
    • export TARBALLS=https://opensource.apple.com/tarballs
    • curl -O ${TARBALLS}/dtrace/dtrace-338.100.1.tar.gz
    • curl -O ${TARBALLS}/AvailabilityVersions/AvailabilityVersions-45.11.tar.gz
    • curl -O ${TARBALLS}/libplatform/libplatform-220.100.1.tar.gz
    • curl -O ${TARBALLS}/libdispatch/libdispatch-1173.100.2.tar.gz
    • curl -O ${TARBALLS}/xnu/xnu-6153.141.1.tar.gz
  3. Build CTF tools from dtrace
    • tar zxf dtrace-338.100.1.tar.gz
    • cd dtrace-338.100.1
    • mkdir obj sym dst
    • echo "#include <stdint.h>" > include/llvm-Support/DataTypes.h
    • sed -i -e 's,include "llvm/Support/DataTypes,include "llvm-Support/DataTypes,' include/llvm-Support/PointerLikeTypeTraits.h
    • xcodebuild install -sdk macosx -target ctfconvert \
      -target ctfdump -target ctfmerge ARCHS=x86_64 \
      SRCROOT=$PWD OBJROOT=$PWD/obj SYMROOT=$PWD/sym DSTROOT=$PWD/dst
    • export TOOLCHAIN=`cd $(xcrun -sdk macosx -show-sdk-platform-path)/../../Toolchains/XcodeDefault.xctoolchain && pwd`
    • sudo ditto $PWD/dst/$TOOLCHAIN /$TOOLCHAIN
    • cd ..
  4. Install AvailabilityVersions
    • tar zxf AvailabilityVersions-45.11.tar.gz
    • cd AvailabilityVersions-45.11
    • mkdir dst
    • make install SRCROOT=$PWD DSTROOT=$PWD/dst
    • sudo ditto \
      $PWD/dst/usr/local/libexec \
      $(xcrun -sdk macosx -show-sdk-path)/usr/local/libexec
    • cd ..
  5. Install libplatform headers
    • tar zxf libplatform-220.100.1.tar.gz
    • cd libplatform-220.100.1
    • sudo mkdir -p \
      $(xcrun -sdk macosx -show-sdk-path)/usr/local/include/os/internal
    • sudo ditto $PWD/private/os/internal \
      $(xcrun -sdk macosx -show-sdk-path)/usr/local/include/os/internal
    • cd ..
  6. Install XNU headers
    • tar zxf xnu-6153.11.26.tar.gz
    • cd xnu-6153.11.26
    • make SDKROOT=macosx ARCH_CONFIGS=X86_64 installhdrs
    • sudo ditto $PWD/BUILD/dst $(xcrun -sdk macosx -show-sdk-path)
    • cd ..
  7. Build firehose from libdispatch
    • tar zxf libdispatch-1173.100.2.tar.gz
    • cd libdispatch-1173.100.2
    • mkdir obj sym dst
    • awk '/include "<DEVELOPER/ {next;} /SDKROOT =/ {print "SDKROOT = macosx"; next;} {print $0}' xcodeconfig/libdispatch.xcconfig > .__tmp__ && mv -f .__tmp__ xcodeconfig/libdispatch.xcconfig
    • xcodebuild install -sdk macosx -target libfirehose_kernel \
      SRCROOT=$PWD OBJROOT=$PWD/obj SYMROOT=$PWD/sym DSTROOT=$PWD/dst
    • sudo ditto $PWD/dst/usr/local \
      $(xcrun -sdk macosx -show-sdk-path)/usr/local
    • cd ..
  8. Patch XNU Source  (these steps may vary slighly between 10.15.x releases)
    • cd xnu-6153.11.26
    • sed -i -e 's/outl(cfgAdr, XeonCapID5);//; s/uint32_t cap5reg = inl(cfgDat);/uint32_t cap5reg = 0;/' osfmk/i386/cpuid.h
    • sed -i -e 's/^notsup:$//' bsd/nfs/nfs_node.c
    • awk '/^subr_prof.o_CFLAGS/ { print "nfs_subs.o_CFLAGS_ADD += -Wno-unused-parameter"; print "nfs_vfsops.o_CFLAGS_ADD += -Wno-unused-parameter -Wno-unused-variable"; print "nfs_vnops.o_CFLAGS_ADD += -Wno-unused-function"; print "if_ipsec.o_CFLAGS_ADD += -Wno-unused-function"; } { print $0 }' bsd/conf/Makefile.template > .__tmp__ && mv .__tmp__ bsd/conf/Makefile.template
    • awk '/include <net\/if_utun.h>/ { print "#include <net/if_vlan_var.h>"; } { print $0 }' bsd/net/dlil.c > .__tmp__ && mv .__tmp__ bsd/net/dlil.c
    • sed -i -e 's/\(if_headless_init.*\)/\1;void \1/' bsd/net/if_headless.c
    • sed -i -e '/if (pcb->ipsec_kpipe_count == 0.*/{N;N;N;N;d;}' bsd/net/if_ipsec.c
    • sed -i -e '/nfs4_delegreturn_rpc(.*/{N;d;}' bsd/nfs/nfs_node.c
    • awk '/include <net\/net_api_stats.h>/ { print "#include <net/pfvar.h>"; } { print $0 }' bsd/net/if_bridge.c > .__tmp__ && mv .__tmp__ bsd/net/if_bridge.c
    • sed -i -e '/cwa_classifier_e wa_reqd;/d' osfmk/i386/cpuid.c
    • awk '/IOPMrootDomain.cpo_CXX/{ print "IODMACommand.iig.cpo_CXXWARNFLAGS_ADD += -Wno-duplicate-decl-specifier"; } { print $0 }' iokit/conf/Makefile.template > .__tmp__ && mv .__tmp__ iokit/conf/Makefile.template
    • echo -e "_pmap_in_ppl\n_pmap_free_reserved_ppl_page\n_pmap_claim_reserved_ppl_page" >> config/Private.exports
    • cd ..
  9. Build XNU (checkout the README.md for more options!)
    • cd xnu-6153.11.26
    • make SDKROOT=macosx ARCH_CONFIGS=X86_64 KERNEL_CONFIGS=RELEASE
    • cd ..

Check out the README.md  file at the top of the XNU source tree for more options to the build system. Some common and useful options include:  KERNEL_CONFIGS=DEVELOPMENT,  BUILD_LTO=0  and LOGCOLORS=y.

Install and Run XNU

SECURITY WARNING: You need to  disable System Integrity Protection and set the machine's Secure Boot security setting to "No Security" in order to install and run a custom kernel.

After the final build step, you should have a new kernel built in {xnu}/BUILD/obj/kernel. In order to run this kernel, you will need to install it, and rebuild the prelinkedkernel image. Installing a kernel could potentially render your system un-bootable, so trying this out in a VM first is recommended.

WARNING: In macOS Catalina the root partition is mounted read-only. In order to mount it read-write, you will need to first disable SIP, then remount using:
sudo mount -uw /
This will only succeed if you have first disabled SIP.

In order to successfully link the macOS Catalina open source kernel, you will also need to build and install the System.kext.
  • # make a backup copy of the existing System.kext!
  • sudo ditto /System/Library/Extensions/System.kext ~/System.kext.backup
  • cd xnu-6153.11.26
  • make SDKROOT=macosx KERNEL_CONFIGS=RELEASE \
    DSTROOT=$PWD/BUILD.syskext \
    install_config
  • sudo chown -R root:wheel BUILD.syskext
  • sudo ditto BUILD.syskext/ /
  • cd ..

To install and run your kernel:
  • cd xnu-6153.11.26
  • sudo ditto $PWD/BUILD/obj/kernel /System/Library/Kernels/kernel
  • sudo kextcache -v -invalidate /
    / locked; waiting for lock.
    Lock acquired; proceeding
    ...
  • sudo reboot
    ...
  • uname -a

If you build a different variant of XNU, you may need to ditto a different kernel name, e.g.,  kernel.development instead of just kernel.

Note that you can select different prelinkedkernel variants from which to boot using the kcsuffix boot-arg. For example, if you built a development kernel (using KERNEL_CONFIGS=DEVELOPMENT in the make invocation), you would install and run it like so:
  • sudo ditto $PWD/BUILD/obj/kernel.development \
    /System/Library/Kernels/kernel.development
  • sudo kextcache -v -invalidate /
  • sudo nvram boot-args="kcsuffix=development"
  • sudo reboot
If you have existing boot-args, you can, of course, preserve them in the nvram boot-args variable.

If you place a different kernel with a different suffix in /System/Library/Kernels, then the kextcache utility will automatically build a prelinked kernel with that variant, and it can be selected using the kcsuffix boot-arg. For example, instead of replacing the default kernel, you could:
  • sudo ditto $PWD/BUILD/obj/kernel \
    /System/Library/Kernels/kernel.test
  • sudo kextcache -v -invalidate /
  • sudo nvram boot-args="kcsuffix=test"
  • sudo reboot
It may also be useful to use the following boot-args to see serial output from the kernel as it boots:
  • sudo nvram boot-args="kcsuffix=development serial=3 -v
at
Labels: , , , , , , , , , , ,
Location: Cupertino, CA, USA

Building XNU for macOS 11.2 (Intel + Apple Silicon)

 The macOS Big Sur 11.2 kernel (XNU) source has been released here:  source ,  tarball . My previous post on building XNU for macOS 1...