[ARM64] Add QEMU testing environment setup guide for SVE testing

[omjavaid]

Summary

This patch adds a HowTo document to lldb docs which gives instruction on how to setup QEMU environment for modern Arm64 features not released in publicly available hardware till date.

More generally this howto can be applied to other architecture types supported by QEMU emulator. This may help setup a lldb test environment for all supported architecture/platform types.

[rovka]

Hi Omair,

If this is intended to be more generally useful, I wonder if we should rename it to something else (e.g. lldb-qemu-howto.txt) and only use AArch64 SVE as a case study.

In any case, thanks for writing this up!

lldb/docs/lldb-qemu-aarch64-sve-howto.txt

5 ↗

(On Diff #271536)

Nitpick: "in an emulation environment in the absence of actual [...]

7 ↗

(On Diff #271536)

Isn't this guide still useful after the patches are merged?

9 ↗

(On Diff #271536)

Typo: all the instructions

19 ↗

(On Diff #271536)

Would it be better for this to be an actual script in lldb/utils or somewhere similar?

78 ↗

(On Diff #271536)

Is this difficult to infer from QEMU's own docs? If not, maybe just add a link and mention only what's specific to this guide (I'm guessing the target-list).

96 ↗

(On Diff #271536)

Same as above, could this be just a link to the official docs?

124 ↗

(On Diff #271536)

Same as step 1: can this live in utils?

176 ↗

(On Diff #271536)

Nitpick: For Internet access [...]

217 ↗

(On Diff #271536)

For consistency, maybe there should be some commands here, or a link to the docs for cross-compiling.

[omjavaid]

In D82064#2113851, @rovka wrote:

Hi Omair,

If this is intended to be more generally useful, I wonder if we should rename it to something else (e.g. lldb-qemu-howto.txt) and only use AArch64 SVE as a case study.

In any case, thanks for writing this up!

Thanks Rovka, I ll generalize this doc and correct the pointed issues in next update.

[omjavaid]

Updated after incorporating review comments.

@rovka I gave your idea a thought about moving included scripts to lldb/utils. IMO these scripts have no relationship with LLDB and no such precedent exist where we uploaded such tools or utilities in LLDB source tree. Lets have other opinions and finalize.

[labath]

In D82064#2152807, @omjavaid wrote:

Updated after incorporating review comments.

@rovka I gave your idea a thought about moving included scripts to lldb/utils. IMO these scripts have no relationship with LLDB and no such precedent exist where we uploaded such tools or utilities in LLDB source tree. Lets have other opinions and finalize.

The relationship to lldb is that it can be used to setup lldb for cross-testing. And I personally don't see a difference between a standalone bash script stashed somewhere inside the repo, and having that script embedded inside a .txt file. The bar for stashing a script into the repo is fairly low, and this seems useful enough to cross it, particularly if it can handle architectures other than arm (I do like the idea of making the description generic, and then mention arm/sve specific in a separate section or file).

lldb/docs/lldb-qemu-aarch64-sve-howto.txt

39–40 ↗

(On Diff #278128)

Why not just RFS_IMG_SIZE_GB=$2, etc., without all the shifts?

[omjavaid]

In light of review comments I have come up with this new sceheme for setting up QEMU VM based testing environment for LLDB.

We now have three scripts:

1. setup.sh is used to build Linux kernel images for Arm and AArch64. Also it builds QEMU from latest sources.
2. rootfs.sh can generate Ubuntu rootf file system images for our QEMU VMs
3. run-qemu.sh puts it all together and uses kernel image, rootfs image and arch information to run a QEMU VM. We currently support Arm and AArch64 architectures and more architectures can be added later.

This can generates a QEMU testing setup in less than 10 minutes on a host machine with reasonable resources.

@labath @rovka thoughts?

[tschuett]

You can buy a PRIMEHPC FX700 from Fujitsu. It has a A64FX CPU with SVE.

[labath]

I like this. Apart from the inline comments (which are mostly about removing excessive mentions of arm(64) -- I want to make that text feel generic, since the file is in a generic place), I have one high-level comment. Given the current layout of the repo, it seems like these scripts would be better placed under lldb/scripts than lldb/tools. E.g., this scripts are fairly similar to the macos-setup-codesign.sh script, which also lives in that folder.

lldb/tools/lldb-test-qemu/README.txt

2 ↗

(On Diff #286161)

9–14 ↗

(On Diff #286161)

16 ↗

(On Diff #286161)

20 ↗

(On Diff #286161)

22 ↗

(On Diff #286161)

lldb/tools/lldb-test-qemu/rootfs.sh

57–65 ↗

(On Diff #286161)

You're only passing these to qemu-debootstrap. It sounds like things could "just work" for a bunch of other distro/arch combinations if we just removed these checks.

lldb/tools/lldb-test-qemu/run-qemu.sh

83–87 ↗

(On Diff #286161)

It would be nice if this searched for the qemu binary on the PATH if it was not explicitly provided. Something like:

if [ -d qemu.git ]; then
  QEMU_BIN=$(pwd)/qemu.git/arm-softmmu/qemu-system-arm
else
  QEMU_BIN=qemu-system-arm
fi

99–101 ↗

(On Diff #286161)

How about we check for this in the parsing while loop? That would mean the --arch argument has to come before --sve, but that does not seem like a bad thing

[JDevlieghere]

I didn't look at the scripts in detail but I have a few general (high level) comments:

• I also feel this should go under lldb/scripts.
• Can we put the documentations as RST under lldb/docs so it's rendered on the website? We already have bunch of developer resources there. That will certainly help with discovery.
• LLVM generally encourages scripts to be written in Python. While it doesn't look like this will work out of the box on Windows, I can imagine someone might be interested in that down the line. I also think the bar is lower for new contributors to write Python than bash. I'm kind of on the fence here because there's a reasonable balance between just running commands and control flow, but then I've been scarred by the build-script-impl disaster in Swift so I lean towards better safe than sorry.

[omjavaid]

In D82064#2223991, @JDevlieghere wrote:

I didn't look at the scripts in detail but I have a few general (high level) comments:

• I also feel this should go under lldb/scripts.
• Can we put the documentations as RST under lldb/docs so it's rendered on the website? We already have bunch of developer resources there. That will certainly help with discovery.
• LLVM generally encourages scripts to be written in Python. While it doesn't look like this will work out of the box on Windows, I can imagine someone might be interested in that down the line. I also think the bar is lower for new contributors to write Python than bash. I'm kind of on the fence here because there's a reasonable balance between just running commands and control flow, but then I've been scarred by the build-script-impl disaster in Swift so I lean towards better safe than sorry.

These scripts fairly optional and are more like developer help for someone who wants to give a quick test to any of LLVM tools on some target in absence of hardware and I agree someone may wanna use them on windows and quite likely I might be that someone in very near future. But I ll hold moving to python for now. I ll write an rst doc and arrange the scripts under lldb/scripts as suggested.

[omjavaid]

This diff addresses issues highlighted in review.

@labath any further thoughts?

[JDevlieghere]

lldb/docs/resources/test.rst

436

What's the reason you chose to refer to the README in the source tree instead of converting it to an RST page and linking to that page from here? I think it's good to have an entry here rather than the whole document, but I still think this should be a hyperlink to its own page.

[omjavaid]

lldb/docs/resources/test.rst

436

Well honestly this was more of me going lazy I ll make it a separate rst page if that sounds more appropriate.

[omjavaid]

This adds a rst page qemu-testing.rst that describes content which was previously written in README.txt.

[omjavaid]

Ping!

LGTM or any further comments or suggestions?

PS: I have been on holidays so apologies for late responses in past few week.

[labath]

This looks fine to me. Please wait a bit to give Jonas the opportunity to comment. (FWIW, I think these are fine as shell scripts right now, but that may change if windows comes into play.)

lldb/docs/index.rst

.. toctree::
    use/map
    use/formatting
    use/variable
    use/symbolication
    use/symbols
    use/python
    use/python-reference
    use/remote
+   use/qemu-testing
    use/troubleshooting
 
 .. toctree::
    :hidden:
    :maxdepth: 1
    :caption: Development
 
    resources/contributing

lldb/docs/resources/test.rst

2. You must tell the test-suite how to connect to the remote system. This is
    correspond to the platform select and platform connect LLDB commands. You
    will usually also need to specify the compiler and architecture for the
    remote system.
 
 Currently, running the remote test suite is supported only with ``dotest.py`` (or
 dosep.py with a single thread), but we expect this issue to be addressed in the
 near future.
 
+Running tests in QEMU System Emulation Environment
+``````````````````````````````````````````````````
+
+QEMU can be used to test LLDB in an emulation environment in the absence of
+actual hardware. `QEMU based testing <https://lldb.llvm.org/use/qemu-testing.html>`_
+page describes how to setup a emulation environment using QEMU helper scripts
+found under llvm-project/lldb/scripts/lldb-test-qemu. These scripts currently
+work with Arm or AArch64, but support for other architectures can be added easily.
+
 Debugging Test Failures
 -----------------------
 
 On non-Windows platforms, you can use the ``-d`` option to ``dotest.py`` which
 will cause the script to wait for a while until a debugger is attached.
 
 Debugging Test Failures on Windows
 ``````````````````````````````````
::
    -p TestPaths.py
    # Root of test tree
    D:\src\llvm\tools\lldb\packages\Python\lldbsuite\test
 
 ::
 
    --arch=i686 --executable D:/src/llvmbuild/ninja/bin/lldb.exe -s D:/src/llvmbuild/ninja/lldb-test-traces -u CXXFLAGS -u CFLAGS --enable-crash-dialog -C d:\src\llvmbuild\ninja_release\bin\clang.exe -p TestPaths.py D:\src\llvm\tools\lldb\packages\Python\lldbsuite\test --no-multiprocess
 
 .. [#] `https://lldb.llvm.org/python_reference/lldb.SBTarget-class.html#BreakpointCreateByName <https://lldb.llvm.org/python_reference/lldb.SBTarget-class.html#BreakpointCreateByName>`_

lldb/docs/use/qemu-testing.rst

+Testing LLDB using QEMU
+=======================
+
+.. contents::
+   :local:
+
+QEMU system mode emulation
+--------------------------
+
+QEMU can be used to test LLDB in an emulation environment in the absence of
+actual hardware. This page describes instructions to help setup a QEMU emulation
+environment for testing LLDB.
+
+The scripts under llvm-project/lldb/scripts/lldb-test-qemu can quickly help
+setup a virtual LLDB testing environment using QEMU. The scripts currently work
+with Arm or AArch64, but support for other architectures can be added easily.
+
+* **setup.sh** is used to build the Linux kernel image and QEMU system emulation executable(s) from source.
+* **rootfs.sh** is used to generate Ubuntu root file system images to be used for QEMU system mode emulation.
+* **run-qemu.sh** utilizes QEMU to boot a Linux kernel image with a root file system image.
+
+Once we have booted our kernel we can run lldb-server in emulation environment.
+Ubuntu Bionic/Focal x86_64 host was used to test these scripts instructions in this
+document. Please update it according to your host distribution/architecture.
+
+.. note::
+  Instructions on this page and QEMU helper scripts are verified on a Ubuntu Bionic/Focal (x86_64) host. Moreover, scripts require sudo/root permissions for installing dependencies and setting up QEMU host/guest network.
+
+Given below are some examples of common use-cases of LLDB QEMU testing
+helper scripts:
+
+Create Ubuntu root file system image for QEMU system emulation with rootfs.sh
+--------------------------------------------------------------------------------
+
+**Example:** generate Ubuntu Bionic (armhf) rootfs image of size 1 GB
+::
+
+  $ bash rootfs.sh --arch armhf --distro bionic --size 1G
+
+**Example:** generate Ubuntu Focal (arm64) rootfs image of size 2 GB
+::
+
+  $ bash rootfs.sh --arch arm64 --distro focal --size 2G
+
+rootfs.sh has been tested for generating Ubuntu Bionic and Focal images but they can be used to generate rootfs images of other Debian Linux distribution.
+
+rootfs.sh defaults username of generated image to your current username on host computer.
+
+
+Build QEMU or cross compile Linux kernel from source using setup.sh
+-----------------------------------------------------------------------
+
+**Example:** Build QEMU binaries and Arm/AArch64 Linux kernel image
+::
+
+$ bash setup.sh --qemu --kernel arm
+$ bash setup.sh --qemu --kernel arm64
+
+**Example:** Build Linux kernel image only
+::
+
+$ bash setup.sh --kernel arm
+$ bash setup.sh --kernel arm64
+
+**Example:** Build qemu-system-arm and qemu-system-aarch64 binaries.
+::
+
+$ bash setup.sh --qemu
+
+**Example:** Remove qemu.git, linux.git and linux.build from working directory
+::
+
+$ bash setup.sh --clean
+
+
+Run QEMU Arm or AArch64 system emulation using run-qemu.sh
+----------------------------------------------------------
+run-qemu.sh has following dependencies:
+
+* Follow https://wiki.qemu.org/Documentation/Networking/NAT and set up bridge
+  networking for QEMU.
+
+* Make sure /etc/qemu-ifup script is available with executable permissions.
+
+* QEMU binaries must be built from source using setup.sh or provided via --qemu
+  commandline argument.
+
+* Linux kernel image must be built from source using setup.sh or provided via
+  --kernel commandline argument.
+
+* linux.build and qemu.git folder must be present in current directory if
+  setup.sh was used to build Linux kernel and QEMU binaries.
+
+* --sve option will enable AArch64 SVE mode.
+
+
+**Example:** Run QEMU Arm or AArch64 system emulation using run-qemu.sh
+::
+
+  $ sudo bash run-qemu.sh --arch arm --rootfs <path of rootfs image>
+  $ sudo bash run-qemu.sh --arch arm64 --rootfs <path of rootfs image>
+
+**Example:** Run QEMU with kernel image and qemu binary provided using commandline
+::
+
+  $ sudo bash run-qemu.sh --arch arm64 --rootfs <path of rootfs image> \
+  --kernel <path of Linux kernel image> --qemu <path of QEMU binary>
+
+
+Steps for running lldb-server in QEMU system emulation environment
+------------------------------------------------------------------
+
+* Make sure bridge networking is enabled between host machine and QEMU VM
+
+* Find out ip address assigned to eth0 in emulation environment
+
+* Setup ssh access between host machine and emulation environment
+
+* Login emulation environment and install dependencies
+
+::
+
+  $ sudo apt install python-dev libedit-dev libncurses5-dev libexpat1-dev
+
+* Cross compile LLDB server for AArch64 Linux: Please visit https://lldb.llvm.org/resources/build.html for instructions on how to cross compile LLDB server.
+
+* Transfer LLDB server executable to emulation environment
+
+::
+
+  $ scp lldb-server username@ip-address-of-emulation-environment:/home/username
+
+* Run lldb-server inside QEMU VM
+
+* Try connecting to lldb-server running inside QEMU VM with selected ip:port

lldb/scripts/lldb-test-qemu/rootfs.sh

+#!/bin/bash
+
+set -e
+
+print_usage() {
+  echo "Usage:"
+  echo "Usage: $(basename $0) [options]"
+  echo -e "Creates a Ubuntu root file system image.\n"
+  echo -e "  --help\t\t\tDisplay this information."
+  echo -e "  --arch {armhf|arm64}\t\tSelects architecture of rootfs image."
+  echo -e "  --distro {bionic|focal}\tSelects Ubuntu distribution of rootfs image."
+  echo -e "  --size n{K|M|G}\t\tSets size of rootfs image to n Kilo, Mega or Giga bytes."
+  exit "$1"
+}
+
+invalid_arg() {
+  echo "ERROR: Unrecognized argument: $1" >&2
+  print_usage 1
+}
+
+update_repositories() {
+  echo -e "\nUpdating apt repositories. "
+  echo -e "\nPress 'y' to continue or any other key to exit..."
+  read -s -n 1 user_input
+  if [[ $user_input == 'Y' ]] || [[ $user_input == 'y' ]]; then
+    sudo apt update
+  else
+    exit
+  fi
+}
+
+# Parse options
+while [[ $# -gt 0 ]]; do
+  case "${END_OF_OPT}${1}" in
+    --help)     print_usage 0 ;;
+    --arch)     rfs_arch=$2;   shift;;
+    --distro)   rfs_distro=$2; shift;;
+    --size)     rfs_size=$2;   shift;;
+    *)          invalid_arg "$1" ;;
+  esac
+  shift
+done
+
+if [ -z "$rfs_arch" ]; then
+  echo "Missing architecture"
+  print_usage 1
+fi
+if [ -z "$rfs_distro" ]; then
+  echo "Missing distribution"
+  print_usage 1
+fi
+if [ -z "$rfs_size" ]; then
+  echo "Missing size"
+  print_usage 1
+fi
+
+if [[ "$rfs_arch" != "arm64" && "$rfs_arch" != "armhf" ]]; then
+  echo "Invalid architecture: $rfs_arch"
+  print_usage 1
+fi
+
+pat='^[0-9]+[K|M|G]$'
+if [[ ! $rfs_size =~ $pat ]]; then
+  echo "Invalid size: $rfs_size"
+  print_usage 1
+fi
+
+update_repositories
+
+echo "Installing build dependencies ..."
+sudo apt-get install debootstrap qemu-user-static schroot qemu-utils
+
+image_name=$rfs_distro-$rfs_arch-"rootfs"
+echo "Creating $rfs_distro ($rfs_arch) root file system ..."
+echo "Image name: $image_name.img"
+echo "Image size: $rfs_size"
+
+qemu-img create $image_name.img $rfs_size
+
+mkfs.ext4 $image_name.img
+mkdir $image_name.dir
+sudo mount -o loop $image_name.img $image_name.dir
+
+sudo qemu-debootstrap --arch $rfs_arch $rfs_distro $image_name.dir
+
+sudo chroot $image_name.dir locale-gen en_US.UTF-8
+
+sudo chroot $image_name.dir sed -i \
+'s/main/main restricted multiverse universe/g' /etc/apt/sources.list
+
+sudo chroot $image_name.dir sed -i '$ a\nameserver 8.8.8.8' /etc/resolv.conf
+
+sudo chroot $image_name.dir apt update
+sudo chroot $image_name.dir apt -y install ssh bash-completion
+sudo chroot $image_name.dir adduser --gecos "" $USER
+sudo chroot $image_name.dir adduser $USER sudo
+sudo umount $image_name.dir
+rmdir $image_name.dir

lldb/scripts/lldb-test-qemu/run-qemu.sh

+#!/bin/bash
+
+print_usage() {
+  echo "Usage: $(basename $0) --arch [arm|arm64] [options]"
+  echo -e "Starts QEMU system mode emulation for the architecture.\n"
+  echo -e "  --help\t\t\tDisplay this information."
+  echo -e "  --arch {arm|arm64}\t\tSelects architecture QEMU system emulation."
+  echo -e "  --sve {path}\t\t\tEnables AArch64 SVE mode.\n"
+  echo -e "  --rootfs {path}\t\tPath of root file system image."
+  echo -e "  --qemu {path}\t\t\tPath of pre-installed qemu-system-* executable."
+  echo -e "  --kernel {path}\t\tPath of Linux kernel prebuilt image.\n"
+  echo -e "By default this utility will use:"
+  echo -e "  QEMU image built from source in qemu.git directory"
+  echo -e "  Linux kernel image from linux.build/(arm or arm64) directory."
+  echo -e "Custom Linux kernel image or QEMU binary can be provided using commandline."
+  exit "$1"
+}
+
+invalid_arg() {
+  echo "ERROR: Unrecognized argument: $1" >&2
+  print_usage 1
+}
+
+run_qemu() {
+  QEMU_CORES=2
+  QEMU_MEMORY=1024
+
+  $QEMU_BIN \
+  -cpu $QEMU_CPU \
+  -m $QEMU_MEMORY \
+  -smp $QEMU_CORES \
+  -kernel $KERNEL_IMG \
+  -machine $QEMU_MACHINE \
+  -drive file=$ROOTFS_IMG,if=none,format=raw,id=hd0 \
+  -device virtio-blk-device,drive=hd0 \
+  -append "root=/dev/vda rw ip=dhcp mem=1024M raid=noautodetect  \
+  crashkernel=128M rootwait console=ttyAMA0 devtmpfs.mount=0" \
+  -netdev type=tap,id=net0 \
+  -device virtio-net-device,netdev=net0 \
+  -nographic
+}
+
+# Parse options
+while [[ $# -gt 0 ]]; do
+  case "${END_OF_OPT}${1}" in
+    --arch)     ARCH=$2; shift;;
+    --rootfs)   ROOTFS_IMG=$2; shift;;
+    --kernel)   KERNEL_IMG=$2; shift;;
+    --qemu)     QEMU_BIN=$2; shift;;
+    --sve)      SVE=1;;
+    --help)     print_usage 0 ;;
+    *)          invalid_arg "$1" ;;
+  esac
+  shift
+done
+
+if [ "$ARCH" == "arm64" ] && [ "$ARCH" == "arm" ]; then
+  echo "Invalid architecture: $ARCH"
+  print_usage 1
+fi
+
+if [[ ! -f "$ROOTFS_IMG" ]]; then
+  echo "No root file system image image available for emulation."
+  exit
+fi
+
+if [[ ! -f "$KERNEL_IMG" ]]; then
+  KERNEL_IMG_PATH=$(pwd)/linux.build/"$ARCH"/arch/"$ARCH"/boot/
+
+  if [[ ! -d "$KERNEL_IMG_PATH" ]]; then
+    echo "No Linux kernel image available for emulation."
+    exit
+  fi
+
+  if [[ "$ARCH" == "arm" ]]; then
+    KERNEL_IMG=$KERNEL_IMG_PATH/zImage
+  elif [[ "$ARCH" == "arm64" ]]; then
+    KERNEL_IMG=$KERNEL_IMG_PATH/Image
+  fi
+fi
+
+if [[ ! -f "$QEMU_BIN" ]]; then
+  if [[ "$ARCH" == "arm" ]]; then
+    QEMU_BIN=$(pwd)/qemu.git/arm-softmmu/qemu-system-arm
+  elif [[ "$ARCH" == "arm64" ]]; then
+    QEMU_BIN=$(pwd)/qemu.git/aarch64-softmmu/qemu-system-aarch64
+  fi
+
+  if [[ ! -f "$QEMU_BIN" ]]; then
+    echo "QEMU $ARCH system emulation executable not found."
+    exit
+  fi
+fi
+
+if [[ "$ARCH" == "arm" ]]; then
+  QEMU_MACHINE="virt,highmem=off"
+  QEMU_CPU="cortex-a15"
+
+  if [[ $SVE ]]; then
+    echo "warning: --sve is supported by AArch64 targets only"
+  fi
+elif [[ "$ARCH" == "arm64" ]]; then
+  QEMU_MACHINE=virt
+  QEMU_SVE_MAX_VQ=4
+  QEMU_CPU="cortex-a53"
+
+  if [[ $SVE ]]; then
+    QEMU_CPU="max,sve-max-vq=$QEMU_SVE_MAX_VQ"
+  fi
+fi
+
+run_qemu

lldb/scripts/lldb-test-qemu/setup.sh

+#!/bin/bash
+
+print_usage() {
+  echo "Usage: $(basename $0) [options]"
+  echo -e "Builds QEMU and Linux kernel from source.\n"
+  echo -e "  --help\t\t\tDisplay this information."
+  echo -e "  --kernel {arm|arm64}\t\tBuild Linux kernel for the architecture."
+  echo -e "  --qemu\t\t\tBuild QEMU from source."
+  echo -e "  --clean\t\t\tRemove qemu.git and linux.git directories in current directory."
+  exit "$1"
+}
+
+update_repositories() {
+  echo -e "\nUpdating apt repositories. "
+  echo -e "\nPress 'y' to continue or any other key to exit..."
+  read -s -n 1 user_input
+  if [[ $user_input == 'Y' ]] || [[ $user_input == 'y' ]]; then
+    sudo apt update
+  else
+    exit
+  fi
+}
+
+check_dir_exists() {
+  user_input=
+  if [ -d "$1" ]; then
+    echo -e "\n$1 already exists in working directory and will not be updated."
+    echo -e "\nPress 'y' to continue or any other key to exit..."
+    read -s -n 1 user_input
+    if [[ $user_input != 'Y' ]] && [[ $user_input != 'y' ]]; then
+      exit
+    fi
+  fi
+}
+
+invalid_arg() {
+  echo "ERROR: Unrecognized argument: $1" >&2
+  print_usage 1
+}
+
+build_qemu() {
+  echo "Installing QEMU build dependencies ..."
+  sudo apt install git python3-dev libsdl1.2-dev build-essential libpixman-1-dev
+
+  # Checkout source code
+  check_dir_exists "qemu.git"
+  if [ ! -d "qemu.git" ]; then
+    git clone --depth 1 git://git.qemu.org/qemu.git qemu.git
+  fi
+
+  cd qemu.git
+  # We are going to build QEMU Arm and AArch64 system mode emulation.
+  # ./configure --help emits a list of other possible targets supported by QEMU.
+  ./configure --target-list=arm-softmmu,aarch64-softmmu
+  make -j`getconf _NPROCESSORS_ONLN`
+}
+
+build_linux() {
+  echo "Installing Linux kernel build dependencies ..."
+  sudo apt install git bison flex build-essential libssl-dev bc
+
+  check_dir_exists "linux.git"
+
+  if [ ! -d "linux.git" ]; then
+    git clone --depth 1 \
+    https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git linux.git
+  fi
+
+  cd linux.git
+  make mrproper
+
+  if [[ "$1" == "arm" ]]; then
+    echo "Installing gcc-arm-linux-gnueabihf ..."
+    sudo apt install gcc-arm-linux-gnueabihf
+
+    # Configure kernel_branch=master arch=arm config=vexpress_defconfig
+    make O=../linux.build/arm ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- \
+    vexpress_defconfig
+
+    # Trigger Arm kernel build
+    make -j`getconf _NPROCESSORS_ONLN` O=../linux.build/arm ARCH=arm \
+    CROSS_COMPILE=arm-linux-gnueabihf-
+  elif [[ "$1" == "arm64" ]]; then
+    echo "Installing gcc-aarch64-linux-gnu ..."
+    sudo apt install gcc-aarch64-linux-gnu
+
+    # Configure kernel_branch=master arch=arm64 config=defconfig
+    make O=../linux.build/arm64 ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- \
+    defconfig
+
+    # Trigger AArch64 kernel build
+    make -j`getconf _NPROCESSORS_ONLN` O=../linux.build/arm64 ARCH=arm64 \
+    CROSS_COMPILE=aarch64-linux-gnu-
+  else
+    echo "ERROR: Unrecognized architecture: $1" >&2
+    print_usage 1
+    exit
+  fi
+}
+
+clean() {
+  if [ -d "linux.git" ]; then
+    echo "Removing linux.git ..."
+    rm -rf linux.git
+  fi
+
+  if [ -d "linux.build" ]; then
+    echo "Removing linux.build ..."
+    rm -rf linux.build
+  fi
+
+  if [ -d "qemu.git" ]; then
+    echo "Removing qemu.git ..."
+    rm -rf qemu.git
+  fi
+
+  exit
+}
+
+# Parse options
+while [[ $# -gt 0 ]]; do
+  case "${END_OF_OPT}${1}" in
+    -h|--help)   print_usage 0 ;;
+    -k|--kernel)
+      if [ "$2" == "arm64" ] || [ "$2" == "arm" ]; then
+      KERNEL_ARCH=$2
+      else
+        invalid_arg "$2"
+      fi
+      shift;;
+    -q|--qemu)
+        QEMU=1;;
+    -c|--clean)  clean ;;
+    *)           invalid_arg "$1" ;;
+  esac
+  shift
+done
+
+update_repositories
+
+if [ "$KERNEL_ARCH" != "" ]; then
+  pushd .
+  build_linux $KERNEL_ARCH
+  popd
+fi
+
+if [[ $QEMU -eq 1 ]]; then
+  pushd .
+  build_qemu
+  popd
+fi