FreeNAS 9.2.0 Development

I’ve been diving into FreeNAS development lately. In this blog post, I’ll talk a little bit about what I did to make development life on the FreeNAS appliance more livable from a FreeBSD developer’s perspective.

The first thing you notice after booting up FreeNAS is that it has a console menu. This was my first stop to getting the box online. A couple of keystrokes and we had an IP address. Load up the IP in your browser and you’re ready to start configuring FreeNAS. We set a root password (via the initial WebUI), and you’re off to the races.

Now comes time to customize FreeNAS for development purposes. Most people won’t want to (and shouldn’t) make these adjustments as they are strictly for the purpose of developing enhancements to FreeNAS (right on the box itself). These changes are not supported by the FreeNAS support team (I cannot emphasize this enough).

As a pre-requisite to getting any customizations in-place, we’ll need to be able to log into the UNIX command-line. In the WebUI, select “Services”, scroll down to “SSH” and hit the toggle to turn it “ON”. Then click the wrench logo just off to the right of the SSH toggle. To log in with the root password using SSH, we need to first enable the security feature which comes disabled by default. Check the configuration option labeled “Login as Root with password” (otherwise you can only login as root with a private key).

In my case, I want to develop enhancements before I configure my storage. So it is rather inconvenient that the FreeNAS WebUI only allows you to create users with home directories within configured storage. For your average consumer, this restriction is more than acceptable considering the boot media is often very small and has little free space. However, I need a place to store the patches that I will generate while working on the Volume Manager code and despite what it lacks in free space, the thumb drive or disk that FreeNAS boots from is ideal for this.

The trick to adding a user with non-standard home directory appears to be: first add your user from the WebUI, initially selecting “/nonexistent” as the home-directory (only valid choice when you have yet to configure a storage volume); and then afterward use the following command from the command-line of your FreeNAS appliance to change the home-directory:

sqlite3 /data/freenas-v1.db "UPDATE account_bsdusers \
    SET bsdusr_home=\"/data/home/USER\" WHERE bsdusr_username=\"USER\""

Note that every time you edit this user from the WebUI, you will need to change the home-directory to “/nonexistent” to force the WebUI to accept/save the changes. Afterward, repeat the above command to restore the desired home directory.

The next issue that I run into is perhaps less common. This issue is specific to my ISP. I require the ability to add the following line to the file /etc/dhclient.conf:

supersede domain-name "mydomain";

The reason for this is because I use my ISPs DHCP and the offer contains a domain of XYZ(dot)com, while I prefer ABC(dot)com. By adding the above line to /etc/dhclient.conf, the underlying FreeBSD system will “do the right thing” and supersede the domain provided by my ISP with my desired one. In the end, /etc/resolv.conf gets auto-populated with what I need while still using WAN-based DHCP (which itself is uncommon and seldom available).

The gotcha with simply modifying /etc/dhclient.conf is that you’ll lose your changes upon reboot. You really need to put the directive into:


But before you can edit this file, you need to re-mount the root filesystem as read-write by executing:

mount -uw /

Then you can change the files in /conf and either reboot or execute the following to make the root filesystem read-only again:

mount -ur /

NOTE: This command may take a few moments as it needs to clear pending writes.

Next, I like to install a few packages like bash, vim-lite, pstree, iftop, and sudo (NB: root filesystem needs to be read-write before executing):

pkg_add -r bash
pkg_add -r vim-lite
pkg_add -r pstree
pkg_add -r iftop
pkg_add -r sudo

I’m sure that I install more than just the above, but since the FreeNAS appliance makes /var a memory filesystem, there’s no backup record (usually /var/db/pkg) of installed packages. Once you reboot, you’ll lose the record of what’s installed-versus-not.

ASIDE: It’s really convenient that FreeNAS allows you to specify that a group be allowed to execute sudo. Rather than checking the box for each individual user that you would like to allow sudo access, using a group means you can then store files as group-writable on the disk — simplifying team-based system administration.

After performing all these customizations, I think it’s time to make a backup. To make a backup of my FreeNAS thumb drive, I’ve had success in using version 0.9 of Win32DiskImager from Just stick the thumb drive into a Window system, launch Win32DiskImager, and use the “Read” option to make an image of your thumb drive. This image can then be stored for backup or imaged to another device.

I’m actually not a huge fan of running my OS from a thumb drive. So the first thing I do with my backup image is to write it to an internal Solid State Disk. I copied the backup image of my thumb drive to a FreeBSD server where the SSD was installed as an extra disk, and executed the following command to write the image to the drive:

dd if=backup.img of=/dev/ada3 bs=64k

NOTE: /dev/ada3 is the SSD installed as an extra disk.

After that has completed, I remove the SSD from my FreeBSD server and insert it into the 1U that used to boot FreeNAS from thumb drive.

All is dandy; the SSD boots up just like the thumb drive (except many times faster, shaving over 30 seconds from the boot process).

Next, I wish that the /data directory (mounted from /dev/ufs/FreeNASs4) was larger than the scant 20MB initially created by the installer. The SSD that I just cloned FreeNAS to is much larger than 2GB, leaving just over 80GB of unused free space. Being-one to never waste (and especially since I now have a user with a home directory of /data/home/user ready to utilize said space), I really want the /data partition to reach all the way to the end of the drive.

First, stick the drive into another FreeBSD or FreeNAS system (I used FreeBSD 9.2-STABLE @r260095M) where it is visible as an “extra” (unused) disk. Then execute the following steps to reclaim the unused free space (assigning it to the /data partition):

mount /dev/ufs/FreeNASs4 /mnt
mkdir /foo
rsync -avH /mnt/ /foo/
umount /mnt
gpart resize -i 4 ada3
newfs -b 32768 -f 4096 -L FreeNASs4 /dev/ada3s4
mount /dev/ufs/FreeNASs4 /mnt
rsync -avH --delete /foo/ /mnt/
umount /mnt
rm -Rf /foo

Now I remove the SSD, replace it into my FreeNAS chassis and boot it up. Voila, 82G of free space for /data.

FreeNAS is now ready for a collective team of multiple system administrators, able to treat the system less like an appliance and more like a sandbox for development. Cheers!