A very long time ago, I set up and played around with diskless machines. These are basically PCs can boot up an operating system fully without hard disks. All the operating system files come from a server on the network. It was amazing (well, to me at least)!
Back then, Ethernet cards used to have a DIP/PLCC socket, which allowed you to insert an EEPROM on which you burn a boot ROM. Fortunately I didn’t have to do any of that because the network cards at that time already came with PXE ROMs built-in, just as they do today. To activate this, you just need to select the network card’s option ROM in the boot order, or make it higher up in the boot priority.
As part of the boot process, the network card will request an address from the DHCP server, which also tells the client where it can find the TFTP server with the next boot stage. The ROM will download this file from the TFTP server and start executing it.
That’s how Linux ultimately gets started from the network.
An announcement was made recently on the Raspberry Pi blog that you can achieve total network boot, just like on the PC, without any SD cards.
Network booting machines will simplify management, especially if you have a large cluster of them. Without hard disks (or in this case, SD cards), there will also be lesser parts that will wear out and require replacement.
Some examples of large installations are classrooms, labs, and perhaps networked dashboards or billboards. Mythic Beasts use Raspberry Pis in a data centre. Couple that with a PoE hat/addon and a managed switch and you now have the ability to power-cycle individual boards remotely.
On The Raspberry Pi 3
On the new Raspberry Pis, you can activate other “boot modes” that have been thus far dormant, tucked away in the boot ROM code. Once activated, the Pi 3 no longer reads
bootcode.bin from the SD card but instead initializes the network interface and attempts to load
start.elf from a remote server.
You can read more about how to activate this mode and setup the remote servers in the following guides:
- Network boot on the Pi: https://www.raspberrypi.org/documentation/hardware/raspberrypi/bootmodes/net.md
- Tutorial to setup network boot on the Raspberry Pi 3, using another Raspberry Pi as a server: https://www.raspberrypi.org/documentation/hardware/raspberrypi/bootmodes/net_tutorial.md
Does this work on all Pis? Unfortunately not.
I have a 2nd generation Pi 1 (the one with 512MB of RAM instead of 256MB) and tried putting
/boot friends onto the SD card but it didn’t work.
The announcement post did summarize the support for each Pi though:
- Pi 1: doesn’t work
- Pi 1 B+ & Pi 2: needs
bootcode.binon SD card
- Pi 3: no SD card required
[photo from Raspberry Pi announcement blog post]
However, this boot ROM code isn’t without its problems. Apparently there are some bugs in the code, and because it has already been burned into the ROM of each Pi, it can’t ever be updated (or fixed). If this built-in implementation is not working for you, read on for Plan B.
On Older Raspberry Pis
What you want to aim for is to not re-flash the SD cards every time you want to upgrade your distro, or when changing
config.txt for example. This means trying to push as much of the files out of the SD card and onto the network server.
The Pi boot process looks like this:
start.elf ➔ Linux kernel
The general idea is to replace
start.elf with something that does what a PXE boot ROM would do. This involves initializing the USB peripheral/stack, then the network card (SMSC, or now Microchip LAN9512/4), and finally, a working TCP/IP stack. That’s a lot of code for someone to write and maintain!
One idea might be to just let it boot into the Linux kernel, since it will perform all of the steps above. At this stage, we will have networking support and can do DHCP + TFTP to get more files. The kernel has a feature called kexec, which technically allows you to replace the kernel with another kernel. This feature might not work well with certain drivers and/or hardware; read the wiki for more details.
Another alternative is to use U-Boot, which is commonly used as a bootloader for embedded devices. In this case, U-Boot pretends to be the Linux kernel and is loaded in its place. From there, it can then load the real kernel from the network. This thread on the Raspberry Pi forums is a good starting point.
There’s still work to be done, but this seems like the best alternative for older Pis so far…