Category: netboot

Booting Linux ISOs with Memdisk and iPXE

There are a number of distributions out there that provide proper support for booting the distribution over the network. A lot of the more popular distributions usually provide a installer kernels that can be easily downloaded for use. You point at the vmlinuz and the initrd and can them immediately proceed with the install streaming down packages as needed. These distributions make it great for tools like netboot.xyz to install using iPXE.

There are some distributions out there that don’t have this functionality and typically only produce the ISO without any repositories that provide installer kernels or the rootfs.

In those cases, occasionally you can use memdisk and iPXE to boot those ISOs but they don’t always work. In doing some research, I ran across one of the major issues as to why.

Syslinux – Memdisk

The following was taken from syslinux – memdisk.

The majority of Linux based CD images will also fail to work with MEMDISK ISO emulation. Linux distributions require kernel and initrd files to be specified, as soon as these files are loaded the protected mode kernel driver(s) take control and the virtual CD will no longer be accessible. If any other files are required from the CD/DVD they will be missing, resulting in boot error(s). Linux distributions that only require kernel and initrd files function fully via ISO emulation, as no other data needs accessing from the virtual CD/DVD drive once they have been loaded. The boot loader has read all necessary files to memory by using INT 13h, before booting the kernel.

There is also another solution, which requires the phram and mtdblock kernel module and memdiskfind utility of the Syslinux package (utils/memdiskfind). memdiskfind will detect the MEMDISK mapped image and will print the start and length of the found MEMDISK mapped image in a format phram understands:

modprobe phram phram=memdisk,$(memdiskfind)
modprobe mtdblock

This will create a /dev/mtdblock0 device, which should be the .ISO image, and should be mountable.

If your image is bigger than 128MiB and you have a 32-bit OS, then you have to increase the maximum memory usage of vmalloc, by adding:

vmalloc=<at_least_size_of_your_image_in_mib>Mi</at_least_size_of_your_image_in_mib>

Example: vmalloc=256Mi to your kernel parameters.

memdiskfind can be compiled with the klibc instead of with the glibc C library to get a much smaller binary for use in the initramfs:

cd ./syslinux-4.04/utils/
make spotless
make CC=klcc memdiskfind

Implementations of phram and mtdblock

ArchLinux has implemented the above concept here and here.

Debian Live used it here.

It’s also been implemented in Clonezilla and GParted.

Antergos Linux based on Arch Linux works great with memdisk using the phram module.

Conclusion

I think it would be great for more distributions to attempt to implement something like this so that iPXE tools can be used to load the ISOs instead of actually having to burn or look for the location of the latest ISO every time.

Some of the distributions I’d love to see network support or better memdisk support are:

Linux Mint
Manjaro
Elementary
Solus Project

There are also many other new distributions being released all the time. I typically use DistroWatch to determine the most popular distributions to attempt to add to netboot.xyz. I’d love to get a lot of these added to make it really easy to install anything on the fly.

I’d also love to see some of the hypervisors out there crack open the ISOs, pull them outside of their paywalls, and host the bits on their servers so that it’s much easier to immediately boot an install to test something out without having to jump through many hoops. I have working installs for VMware ESX and Citrix XenServer but I’d need to have them host the bits or allow permission to do so for a public facing installer menu.

netboot.xyz

My newest project on the side is netboot.xyz. If you’ve seen boot.rackspace.com, this should look pretty familiar to you. I ran across cheap .xyz domains from Namecheap (one dollar at the time!), and figured the netboot.xyz name space was much easier to remember and was more neutral to the goal I was trying to accomplish. I forked boot.rackspace.com (still doing basic maintenance) and am now focusing my efforts on netboot.xyz.

My goal with the project is to make it easy as possible to boot many of the popular operating systems from bare metal, virtual machines, etc without the hassle of hunting down the latest ISO for the OS you want to download. Ideally it’s usable with any service provider or just someone who maintains their own servers.

I usually try and use operating systems that make their boot loaders available via mirrors, although there are occasionally some exceptions. I’m also experimenting with various new builds like WinPE, Live Booted OS, and I’d like to even pursue getting some hypervisors on there as well to make it as easy as possible to install everything.

It’s also a great place to just let people play around with new operating systems with just a menu and learn about the many many distributions out there.

Check it out when you get a chance and drop me some feedback or make a pull request if you see something I’m missing. I’ve added a really easy way to test your pull request from the utility menu, all you need to do is enter in your github username and branch or hash of the commit you want to test.

I’m still working on a bunch of documents for demonstrating how easy it is to plug the 1MB iPXE ISO into things like VMware Fusion, Virtual Box, Openstack, so bear with me while I try and get all of those available.

Enjoy!

Building and Booting Debian Live Over the Network

If you’ve ever downloaded a LiveCD, you know it’s a self contained distribution that can run in memory and usually serves some sort of purpose. It can act as the front end for an installer, run a series of tools like Kali Linux, or be used to access the internet anonymously. But lets face it, who uses CDs today? Today, I’m going to run through what Debian Live is, how to build it, and how you can potentially use it an appliance you can boot over the network with having to worry about local storage.

You’ll typically want to run the live-build utility on Debian. Live-build has active development currently so there are newer builds in testing (Jessie) but they can be unstable. The version in stable (wheezy) is currently 3.0.5-1 whereas the one in testing is 4.0~alpha36-1. You might want to start out with the stable version first as the testing one is in active development.

What is Debian Live

Debian Live is a Debian operating system that does not require a classical installer to use it. It comes on various media, including CD-ROM, USB sticks, or via netboot. Because it’s ephemeral, it configures itself every time on boot. There are several phases it goes through on boot before init is started:

  • live-boot handles the initial start up and retrieval of the squashfs image. This runs within the initramfs.
  • live-config which handles all the configuration of how the image will be booted, including the initial setup of networking. This is ran from within the squashfs image.
  • Custom hooks are ran near the end of live-config which allow you to manipulate the filesystem and take care of any custom setup.  You can also key off custom kernel command line key values pairs that you’ve put in place to do different actions.

Once those phases are completed init is started and the image will boot as normal into the appropriate runlevel.

Building the Debian Live Image

To install live-build, run:

apt-get install live-build

This will install the lb binary used to generate the live image. Create and change into a directory that will hold your image.

mkdir debian-live
cd debian-live
lb init
lb config \
--distribution wheezy \
--architectures amd64 \
--binary-images netboot \
--debconf-frontend dialog \
--chroot-filesystem squashfs \
--parent-mirror-bootstrap http://mirrors.kernel.org/debian/ \
--parent-mirror-chroot-security http://mirrors.kernel.org/debian-security/ \
--parent-mirror-binary http://mirrors.kernel.org/debian/ \
--parent-mirror-binary-security http://mirrors.kernel.org/debian-security/ \
--mirror-bootstrap http://mirrors.kernel.org/debian/ \
--mirror-chroot-security http://mirrors.kernel.org/debian-security/ \
--mirror-binary http://mirrors.kernel.org/debian/ \
--mirror-binary-security http://mirrors.kernel.org/debian-security/ \
--archive-areas "main non-free contrib" \
--apt-options "--force-yes --yes" \
--bootappend-live "keyboard-layouts=en"

This will create a default directory structure to generate your live build. Most of those options are optional, but they will give you a good head start. With that base configuration, you should be able to run the following to start generating the build:

lb build

This will take a while, but once completed you’ll have generated some files that you can use for your netbooted image. The files you’ll need are here:

debian-live/tftpboot/live/vmlinuz
debian-live/tftpboot/live/initrd.img
debian-live/binary/live/filesystem.squashfs

Those three files are all you need to netboot a Debian Live image. The vmlinuz and initrd.img are called first and then the filesystem.squashfs is retrieved during initrd.img bootup.

Regenerating a New Image

If you need to regenerate the image again, you’ll need to clean the previous build up first:

To reset the directory but leave the package cache:

lb clean

To reset the directory and clear the cache:

lb clean --cache

To clean everything and just leave the config directory:

lb clean --all

Setting up iPXE

Using iPXE is probably the easiest way to load up the new image. With the files hosted via HTTP, you can set up your iPXE config like this, add it to your iPXE menu and then use it to load your new image.

#!ipxe
kernel http://mywebserver.com/live/vmlinuz
module http://mywebserver.com/live/initrd.img
imgargs vmlinuz boot=live config hooks=filesystem ${conkern} username=live noeject fetch=http://mywebserver.com/live/filesystem.squashfs
boot

It will create a default user of live with the password of live.

In a future article, I’ll talk about how you can further customize the live distribution. For now, make sure to check out the Debian Live manual that gives a great run down on customization.

Citrix XenServer 6.1 Automated Installer for Openstack

I’ve put my Openstack XenServer 6.1 (Tampa) installer onto Github here: https://github.com/amesserl/xs-tampa-openstack.

It has all of the modifications to get it running with the XenAPI Openstack Nova code and also includes the latest hotfixes. All you need to do is snag the latest CD and drop it in. I’ll continue to publish repos for new versions as they come out (Clearwater should be released soon). You can also boot it from osimag.es as well.