How to create custom images
MAAS supports deploying custom OS images. Canonical provides both lp:maas-image-builder and gh:canonical/packer-maas to support creating custom images. These custom images can include static Ubuntu images, created with whatever tool you choose, as well as other OS images.
Even so, Canonical suggests customising machines using cloud-init user_data or Curtin preseed data whenever possible.
This article will help you learn:
- About static Ubuntu images
- About uploading hand-built Ubuntu images
- About how MAAS handles these images
- About how MAAS boots these images
- About configuring deployed machine networking
- About configuring deployed machine storage
- About static image metrics
- How to upload a custom Ubuntu image
- How to build MAAS images
About static Ubuntu images
MAAS provides the capability for you to build an Ubuntu OS image to deploy with MAAS, using any image-building method you choose. You can create the image once, with a fixed configuration,and deploy it to many machines. This fixed configuration can consist of anything that a normal image would contain: users, packages, etc.
About uploading hand-built Ubuntu images
You can upload hand-built Ubuntu images, containing a kernel, bootloader, and a fixed configuration, for deployment to multiple machines. The image can be built via a tool, such as packer, or build with scripts. You can upload these images to the boot-resources endpoint, where it will then be available for deployment to machines.
At a minimum, this image must contain a kernel, a bootloader, and a /curtin/curtin-hooks
script that configures the network. A sample can be found in the packer-maas repos. The image must be in raw img file format, since that is the format MAAS accepts for upload. This is the most portable format, and the format most builders support. Upon completing the image build, you will upload this img file to the boot-resources endpoint, specifying the architecture for the image.
About how MAAS handles these images
MAAS will save the image – in the same way it would save a tar.gz
file – in the database. MAAS can differentiate between custom Ubuntu images and custom non-Ubuntu images, generating appropriate pre-seed configurations for each image type.
MAAS will also recognise the base Ubuntu version, so it can apply the correct ephemeral OS version for installation. Custom images are always deployed with the ephemeral operating system. The base_image field is used to select the appropriate version of the ephemeral OS to avoid errors. This ensures a smooth deployment later.
About how MAAS boots these images
When you decide to deploy a machine with your uploaded, custom image, MAAS ensures that the machine receives the kernel, bootloader and root file system provided in the image. The initial boot loader takes over, and boots an ephemeral OS of the same Ubuntu version as the custom image, to reduce the chances of incompatibilities. Curtin then writes your entire custom image to disk. Once the custom image is written to disk, it is not modified by MAAS.
Note that custom non-Ubuntu images still use a standard Ubuntu ephemeral OS to boot, prior to installing the non-Ubuntu OS.
About configuring deployed machine networking
If you deploy a machine with a custom Ubuntu image, MAAS allows you to configure the deployed machine’s networks just like you would for any other MAAS machine. If you create an interface and assign it to a subnet or static address, this will be reflected in the deployed machine.
For this reason, MAAS also does some initial diagnostics while installing the custom image. MAAS will detect when a network configuration is not present and abort the installation with a warning. Essentially, MAAS checks to be sure that cloud-init
and netplan
are present in the images written by curtin
. If not, MAAS won’t deploy the machine with the image.
About configuring deployed machine storage
If you deploy a machine with a custom Ubuntu image, you will also want to be able to configure storage, just like you would do with any other machine. MAAS facilitates changes to the storage configuration. You can resize the root partition, as well as attaching and formatting any additional block devices you may desire.
About static image metrics
As a user, you want to keep track of how many static images are being used, and how many deployed machines are using static images. The standard MAAS dashboard reflects both of these metrics.
How to upload a custom Ubuntu image
Currently, custom Ubuntu images can be uploaded using the MAAS CLI,by creating a boot-resource, with a command similar to this one:
maas $PROFILE boot-resources create \
name='custom/ubuntu-custom' \
architecture=amd64/generic \
title=’custom ubuntu’ \
base_image=ubuntu/focal \
filetype=ddraw \
content@=./custom-ubuntu.img
When uploading a custom image, there is a new required field: base_image
. This is not required for non-custom images to be uploaded, but any image with the custom
prefix will require it.
MAAS supports deploying custom DD or TGZ images. Canonical provides both lp:maas-image-builder and gh:canonical/packer-maas to support creating custom images; however, these tools do not currently support Ubuntu. Instead Canonical suggests customising Ubuntu using cloud-init user_data or Curtin preseed data.
This article will help you learn:
- Why customised Ubuntu deployments aren’t supported
- Warnings on creating a custom Ubuntu image
- How to create a custom Ubuntu image for MAAS
Why customised Ubuntu deployments aren’t supported
When the MAAS stream is generated by lp:maas-images it starts by downloading the base SquashFS rootfs from cloud-images.ubuntu.com that is used for all clouds. The SquashFS does not contain a kernel so lp:maas-images mounts the SquashFS with an overlay and chroots in. It then installs a kernel and extra initramfs scripts from the cloud-initramfs-rooturl and cloud-initramfs-copymods packages to allow network booting. Once everything is installed the kernel and newly generated initramfs are pulled out of the overlay and everything is unmounted. lp:maas-imagesw provides the unmodified SquashFS, installed kernel, and generated initramfs as separate files on images.maas.io.
MAAS uses the kernel, initramfs, and SquashFS to perform network booting which allows commissioning, testing, and deployments. When deploying Ubuntu MAAS uses the same version of Ubuntu to network boot into and perform the deployment. This ensures there are no compatibility issues. Other operating systems use the Ubuntu version selected for the ephemeral environment for deployment.
Currently MAAS only allows custom images to be loaded as a single TGZ or DD.GZ, there is no way to upload a kernel, initrd
, and rootfs
as separate files. Even if MAAS was modified to allow the kernel, initrd
, and rootfs
as separate files MAAS requires cloud-initramfs-copymods
and cloud-initramfs-rooturl
to be included in the initrd. It is difficult for MAAS to detect if these scripts are missing and even harder for users to debug if they are missing.
Warnings on creating a custom Ubuntu image
Note that:
- Custom images are always deployed with the ephemeral operating system. This can cause hard to debug errors. For example CentOS 6 can only be deployed by Ubuntu Xenial due to advances in the ext4 filesystem.
- MAAS will still install and configure the GA kernel. If your custom image contains a kernel it most likely won’t be used. MAAS will not allow you to select which kernel(GA, HWE, lowlatency, etc) when deploying a custom Ubuntu image.
- All GNU/Linux custom images should be created as a TGZ to enable storage customisation. When a DD.GZ is used MAAS/Curtin simply writes the file to the filesystem.
How to create a custom Ubuntu image for MAAS
Note: LXD may prevent device files from being created when extracting the rootfs, it is suggested to do this on metal or on a VM:
-
Download the rootfs from images.maas.io
wget http://cloud-images.ubuntu.com/focal/current/focal-server-cloudimg-amd64-root.tar.xz
-
Create a work directory
mkdir /tmp/work
-
Extract the rootfs
cd /tmp/work
sudo tar xf ../focal-server-cloudimg-amd64-root.tar.xz
or
unsquashfs ../focal-server-cloudimg-amd64-root.squash
-
Setup chroot
sudo mount -o bind /proc /tmp/work/proc
sudo mount -o bind /dev /tmp/work/dev
sudo mount -o bind /sys /tmp/work/sys
sudo mv /tmp/work/etc/resolv.conf /tmp/work/etc/resolv.conf.bak
sudo cp /etc/resolv.conf /tmp/work/etc/
-
Chroot in
sudo chroot /tmp/work /bin/bash
-
Customise image
apt update
apt dist-upgrade
apt install emacs-nox jq tree
apt autoremove
-
Exit chroot and unmount binds
exit
sudo umount /tmp/work/proc
sudo umount /tmp/work/dev
sudo umount /tmp/work/sys
sudo mv /tmp/work/etc/resolv.conf.bak /tmp/work/etc/resolv.conf
-
Create TGZ
sudo tar -czf ~/focal-custom.tgz -C /tmp/work .
-
Upload it to MAAS
Note: Ubuntu release names and versions are reserved
maas $PROFILE boot-resources create name='custom/focal-custom' title='Ubuntu 20.04 Custom Image' architecture='amd64/generic' filetype='tgz' content@=~/focal-custom.tgz
-
Configure and deploy as normal
How to build MAAS images
There are two methods for building custom images to be deployed to MAAS machines: MAAS Image Builder, and packer. This section will help you learn:
- About packer prequisites
- About packer deployment requirements
- About customising images
- About building images via a proxy
- How to use packer to build MAAS images
- How to use MAAS Image Builder to build MAAS images
About packer prequisites
The following are required to to create a packer MAAS image:
- A machine running Ubuntu 18.04+ with the ability to run KVM virtual machines.
- qemu-utils
- qemu-system
- ovmf
- cloud-image-utils
- Packer
About packer deployment requirements
The following are required to deploy a packer MAAS image:
About customising images
It is possible to customize the image either during the Ubuntu installation, or afterwards (before packing the final image). The former is done by providing autoinstall config, editing the user-data-flat and user-data-lvm files. The latter is performed by the install-custom-packages script.
About building images via a proxy
The Packer template downloads the Ubuntu net installer from the Internet. To tell Packer to use a proxy, set the HTTP_PROXY environment variable to your proxy server. Alternatively, you may redefine iso_url to a local file, set iso_checksum_type to none to disable the checksums, and remove iso_checksum_url.
How to use packer to build MAAS images
Packer can be used to build images to be deployed by MAAS. In order to use packer, you must have a packer template for the OS version you intend to build.
Currently, templates are available for:
- CentOS6
- CentOS7
- CentOS8
- RHEL7
- RHEL8
- Ubuntu custom images
- VMWare EXSi
This section will help you learn:
Note that additional templates will be made available from time to time.
How to install packer
Packer is easily installed from its Debian package:
sudo apt install packer
It should install without additional prompts.
How to install qemu-utils
You can install qemu-utils
from Debian packages as follows:
sudo apt install qemu-utils
How to install qemu-system
You can install qemu-system
from Debian packages as follows:
sudo apt install qemu-system
How to install ovmf
You can install ovmf
from Debian packages as follows:
sudo apt install ovmf
How to install cloud-image-utils
You can install cloud-image-utils
from Debian packages as follows:
sudo apt install cloud-image-utils
How to obtain templates
You can obtain the packer templates by cloning the packer-maas github repository, like this:
git clone https://github.com/canonical/packer-maas.git
Make sure to pay attention to where the repository is cloned. The Packer template in this cloned repository creates a Ubuntu AMD64 image for use with MAAS.
How to create and use packer images
This subsection will help you learn:
How to build a packer raw image
To build a packer image, you must change to the template repository directory, then to the subdirectory for the image you want to build. From that subdirectory, you can easily build a raw image with LVM, using the Makefile:
$ make custom-ubuntu-lvm.dd.gz
This makefile will run for a couple of minutes before attempting to boot the image. While waiting for the image to boot, packer will attempt to SSH into the machine repeatedly until it succeeds. You will see terminal messages similar to this one for upwards of three to five minutes:
2022/05/09 15:50:46 packer-builder-qemu plugin: [DEBUG] handshaking with SSH
2022/05/09 15:50:50 packer-builder-qemu plugin: [DEBUG] SSH handshake err: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none password], no supported methods remain
2022/05/09 15:50:50 packer-builder-qemu plugin: [DEBUG] Detected authentication error. Increasing handshake attempts.
Eventually, you should see a successful SSH connection:
2022/05/09 15:50:57 packer-builder-qemu plugin: [INFO] Attempting SSH connection to 127.0.0.1:2351...
2022/05/09 15:50:57 packer-builder-qemu plugin: [DEBUG] reconnecting to TCP connection for SSH
2022/05/09 15:50:57 packer-builder-qemu plugin: [DEBUG] handshaking with SSH
2022/05/09 15:51:17 packer-builder-qemu plugin: [DEBUG] handshake complete!
If the process seems to run for a long time, you can predict whether it’s going to work by doing a series of netstat -a
on the IP:port
given in the connection attempt. Attempts may fail repeatedly, but if you repeat the netstat -a
command frequently, you will see some tentative connections, like this one:
stormrider@neuromancer:~$ netstat -a | grep 2281
tcp 0 0 0.0.0.0:2281 0.0.0.0:* LISTEN
tcp 0 0 localhost:46142 localhost:2281 TIME_WAIT
tcp 0 0 localhost:46120 localhost:2281 TIME_WAIT
tcp 0 0 localhost:46138 localhost:2281 TIME_WAIT
tcp 0 0 localhost:46134 localhost:2281 TIME_WAIT
tcp 0 0 localhost:46130 localhost:2281 TIME_WAIT
tcp 0 0 localhost:46124 localhost:2281 TIME_WAIT
stormrider@neuromancer:~$ netstat -a | grep 2281
tcp 0 0 0.0.0.0:2281 0.0.0.0:* LISTEN
tcp 0 0 localhost:46142 localhost:2281 TIME_WAIT
tcp 0 0 localhost:46146 localhost:2281 ESTABLISHED
tcp 0 0 localhost:2281 localhost:46146 ESTABLISHED
tcp 0 0 localhost:46138 localhost:2281 TIME_WAIT
tcp 0 0 localhost:46134 localhost:2281 TIME_WAIT
tcp 0 0 localhost:46130 localhost:2281 TIME_WAIT
tcp 0 0 localhost:46124 localhost:2281 TIME_WAIT
This ESTABLISHED
connection may not hold the first few times, but eventually, the SSH connection will be made, and the provisioning process will finish. If you want to walk away and come back, be advised that the Makefile clears the terminal buffer at the end, but echoes one final instruction:
rm OVMF_VARS.fd
You can check the validity of the operation with ls
, like this:
stormrider@neuromancer:~/mnt/Dropbox/src/git/packer-maas/ubuntu$ ls
custom-ubuntu-lvm.dd.gz packages seeds-lvm.iso user-data-lvm
http packer_cache ubuntu-flat.json
Makefile README.md ubuntu-lvm.json
meta-data scripts user-data-flat
You can also manually run packer. Your current working directory must be in the subdirectory where the packer template is located. In the case of this example, that’s packer-maas/ubuntu
. Once in packer-maas/ubuntu
, you can generate an image with the following command sequence:
$ sudo PACKER_LOG=1 packer build ubuntu-lvm.json
ubuntu-lvm.json is configured to run Packer in headless mode. Only Packer output will be seen. If you wish to see the installation output connect to the VNC port given in the Packer output, or change the value of headless to false in the JSON file.
This process is non-interactive.
How to upload packer images to MAAS
You can upload a packer image with the following command:
$ maas admin boot-resources create \
name='custom/ubuntu-raw' \
title='Ubuntu Custom RAW' \
architecture='amd64/generic' \
filetype='ddgz' \
content@=custom-ubuntu-lvm.dd.gz
Before relying on it in production, you should test your custom image by deploying it to a test (virtual) machine. It’s the machine named open-gannet
in this listing:
maas admin machines read | jq -r '(["HOSTNAME","SYSID","POWER","STATUS",
"OWNER", "OS", "DISTRO"] | (., map(length*"-"))),
(.[] | [.hostname, .system_id, .power_state, .status_name, .owner // "-",
.osystem, .distro_series]) | @tsv' | column -t
HOSTNAME SYSID POWER STATUS OWNER OS DISTRO
-------- ----- ----- ------ ----- -- ------
valued-moth e86c7h on Deployed admin ubuntu focal
open-gannet nk7x8y on Deployed admin custom ubuntu-raw
About the default image username
The default username for packer-created images is ubuntu
, the same as the default username for other MAAS images.
How to use MAAS Image Builder to build MAAS images
MAAS Image Builder is an older tool, still required to build some images (e.g., Windows images). Wherever possible, we recommend you use packer
, as described above.
In order to use MAAS Image Builder, you must purchase Ubuntu Advantage for Infrastructure.
This article will help you learn:
- How to install MAAS Image Builder via a private Canonical PPA (which you can request)
- How to create custom CentOS images
- How to create custom RHEL images
- How to create Windows images
- How to create other kinds of custom images
You can customise most images as much or as little as you wish, then use them to commission machines with MAAS.
How to install MAAS Image Builder
To get MAAS Image Builder, you must be subscribed to a private PPA provided by Canonical Support to those customers who have purchased Ubuntu Advantage for Infrastructure. Note that the steps below will fail if you have not purchased Ubuntu Advantage and been subscribed to the private PPA by your Canonical support rep.
Once subscribed, you need to obtain your credentials at this external link:
Also, you must add the repository with the add-apt-repository
command. Note: Be sure to substitute your unique URL in the command below:
$ sudo add-apt-repository \
“https://LaunchpadID:Password@private-ppa.launchpad.net/maas-image-builder-partners/stable/ubuntu"
Once you have added the private PPA, you can install the Image Builder like this:
$ sudo apt-get install maas-image-builder
All done? Great! Now you can build and customise images for MAAS machines, as shown in the sections below.
How to create custom CentOS images
MAAS already provides the latest available CentOS 6, CentOS 7, and CentOS 8 for automatic download. If you need something else, though, MAAS Image Builder supports the ability to create various CentOS images.
Network Requirements
Access to the Internet is required, since you will need to start with one of these sites:
- http://mirror.centos.org - OS, updates, and extra repositories
- https://download.fedoraproject.org - EPEL
- https://copr-be.cloud.fedoraproject.org - Canonical maintained cloud-init repository
Creating images behind a proxy
MAAS Image Builder can create CentOS images behind a proxy – just set the ‘http_proxy’ environment variable to your particular proxy. Once deployed, yum
will use this MAAS-configured proxy.
Creating the images
maas-image-builder
is designed to automate the process of generating the images for MAAS and curtin
. Here are some specific examples:
$ sudo maas-image-builder -o centos6-amd64-root-tgz --arch amd64 centos --edition 6
$ sudo maas-image-builder -o centos6-i386-root-tgz --arch i386 centos --edition 6
$ sudo maas-image-builder -o centos7-amd64-root-tgz --arch amd64 centos --edition 7
Customising CentOS images
Starting from MAAS Image Builder 1.0.4, customisation of CentOS images is now supported. You can provide a custom kickstart, in addition to the kickstart that MAAS Image Builder uses to create the images. You can customise your image like this:
$ sudo maas-image-builder -o centos7-amd64-root-tgz --arch amd64 centos --edition 7 --custom-kickstart ./custom.ks
Uploading the image into MAAS
Custom CentOS images can be uploaded to MAAS as shown in the command below. Do note that the name must start with ‘centos’ and must be one line:
maas admin boot-resources create name=centos/centos6-custom architecture=amd64/generic content@=./build-output/centos-amd64-root-tgz
You can use the MAAS WebUI to check that your custom CentOS image is valid and selectable for deployment.
How to create custom RHEL images
Currently, MAAS only supports RHEL as a custom image. In future versions of MAAS, RHEL will be natively supported.
Some Requirements
In order to create RHEL images, you will need access to these sites:
- A RHEL DVD ISO - Contains all RHEL archives which are not available without a license
- https://download.fedoraproject.org - Access to the EPEL repository to install required deps
- https://copr-be.cloud.fedoraproject.org - Access to the Canonical maintained cloud-init copr repository
Creating images behind a proxy
MAAS image builder supports creating RHEL images behind a proxy. To use a proxy when building a RHEL image, just set the ‘http_proxy’ environment variable to your local proxy. Once deployed, yum
will use the MAAS-configured proxy.
Creating the images
To generate a usable RHEL image, maas-image-builder
automates image generation; these images can be used by MAAS and curtin
.
$ sudo maas-image-builder -a amd64 -o rhel7-amd64-root-tgz rhel --rhel-iso blah.iso
Install into MAAS
The custom RHEL image can be uploaded to MAAS, but note that the name must start with ‘rhel’ and must be expressed as a single line, like this:
maas admin boot-resources create name=rhel/7 title="RedHat Enterprise Linux 7" architecture=amd64/generic content@=rhel7-amd64-root-tgz
How to create Windows images
Since Windows is a proprietary operating system, MAAS can’t download these images. You need to manually generate images to use with MAAS. On the upside, the end result will be much simpler, since there are CLI and WebUI tools to upload a Windows ISO – which helps automate the process.
Hyper-V 2012 R2
In this example, Windows Hyper-V 2012 R2 is used, but rest assured that multiple versions are supported. Download the Hyper-V 2012 R2 ISO from Microsoft, so it can be used in the image generation process. You can obtain the download at:
http://technet.microsoft.com/en-US/evalcenter/dn205299.aspx
There are several other supported versions (for --windows-edition):
- win2012
- win2012r2
- win2012hv
- win2012hvr2
- win2016
- win2016hv
Image Builder
MAAS Image builder can automate the process of generating images for MAAS and curtin
. In this instance, though, you need Windows drivers for your specific hardware. You can obtain these windows drivers with the following command:
sudo maas-image-builder -o windows-win2012hvr2-amd64-root-dd windows \
--windows-iso ~/Downloads/2012hvr2.ISO \
--windows-edition win2012hvr2 [--windows-updates] \
[--windows-drivers <folder/>] [--windows-language en-US]
As an example, consider:
sudo maas-image-builder -o windows-win2012hvr2-amd64-root-dd windows \
--windows-iso win2012hvr2.iso --windows-edition win2012hvr2 \
--windows-updates --windows-drivers ~/Win2012hvr2_x64/DRIVERS/ --windows-language en-US
Please note that this will not install any Windows updates. In order to obtain an up-to-date image of Windows, be sure provide the –windows-updates
flag. This requires access to a bridged connection with a DHCP server, an interface that can be specified with -i
.
Also note that you may be required to have specific Windows drivers for this image to work in your hardware. Be sure you inject those drivers when installing them. Those drivers are the default *.inf
files.
Debug
You can debug the Windows installation process by connecting to localhost:5901
using a VNC client.
Install into MAAS
The generated images need to be placed into the correct directories so MAAS can deploy them onto a node:
maas admin boot-resources create name=windows/win2012hvr2 \
architecture=amd64/generic filetype=ddtgz \
content@=./build-output/windows-win2012hvr2-amd64-root-dd
Now, using the MAAS WebUI, a node can be selected to use Windows Hyper-V 2012 R2. This selection gets reset when a node is stopped, so make sure to set it before starting nodes. You can also set the default operating system (and release) in the settings menu, which removes the need to set it per-node.
How to create other kinds of custom images
To install other custom images, use the following command sequence:
maas <user> boot-resources create name=<custom-image-codename> title="Ubuntu Custom Image" \
architecture=amd64/generic content@=/location/of/custom/image/ubuntu-custom-root-tgz
As an example:
maas admin boot-resources create name=custom1 \
title=”Ubuntu Custom Image” architecture=amd64/generic \
content@=/home/ubuntu/ubuntu-custom-root-tgz
Last updated 6 days ago.