I work on FreeBSD as the current maintainer/author of the following [released] works:
- FreeBSD boot loader Forth/FICL code [since
- FreeBSD bsdconfig(8) [since
- FreeBSD sysrc(8) [since
- Several ports including:
- … and I’m probably the last committer to work on sysinstall(8).
Now that introductions are out of the way, let’s meet our topic:
The FreeBSD Installer
If you’ve been living under a rock (or in a cave without Internet access) for the past 3+ years, here’s the highlights for what you’ve been missing in the FreeBSD Installer arena:
+ sysinstall(8) had a good run from
9.0-RELEASE: Introduced bsdinstall(8) [new default installer]
9.1-RELEASE: No noticeable changes
+ Improved bsdinstall(8) scripting
+ Introduced bsdconfig(8) [bsdinstall helper]
+ Introduced sysrc(8)
Before continuing, let’s take a brief moment to give credit where credit is due:
~23,000 Lines: Legacy sysinstall(8) by Jordan Hubbard
NOTE: Including the hard work of countless others (including myself)
~6,000 Lines: New installer bsdinstall(8) by Nathan Whitehorn
NOTE: Including many patches from others (including myself)
Meanwhile, I proudly take credit for the following:
However, exactly what is the future with respect to our Installer? It’s often tempting to sit down and look at where we came from and immediately [with ease] point out any number of short-comings in the replacements. While hindsight is paramount to preserving backward compatibility, we should be ever mindful to avoid simply reproducing a new generation of the same thing.
That being said, and with so much effort being put into a complete rewrite, I believe some new high-level goals and ideas can pave a path avoiding the dreaded bikeshed situation (see bikeshed.com; or perhaps teal.bikeshed.com is your cup of tea).
This is a nuclear reactor style look at the FreeBSD Installer from the top-down. Note however, that as we perform a top-down view, design was performed from the bottom-up. We’ll start our top-down view with pretty pictures and end with the very words and
code that inspired this blog entry.
Starting at the top, let’s focus on the User Interface. The UI is still a fairly broad topic, so let’s further focus on a subset of the UI, realtime feedback. Our initial focal point will be the UI management for providing feedback during data input/output operations.
User Interface :: Data I/O Feedback
NOTE: To get proper box drawing characters on the system console under FreeBSD 9.0 or higher, you must change the default terminal type in ttys(5) from xterm to cons25. This is easy in FreeBSD 9.2 or higher — execute
bsdconfig syscons_ttys, select IBM437 (VGA default), execute
init q, then
exit, and log back into the system console.
Meanwhile, here’s what legacy sysinstall(8) looks like:
As you can see from the above two pictures, bsdinstall(8) (top) is an improvement over sysinstall(8) (bottom). Perhaps the most noticeable change is that bsdinstall(8) presents a single widget to the user, providing feedback on all requested distributions within a single screen; meanwhile sysinstall(8) presents only a single progress bar widget, multiple times in a series.
While this is indeed an improvement, a few things come to mind:
However, perhaps the most important observation should be that both applications operate serially. Both bsdinstall(8) and sysinstall(8) do not proceed to the next distribution until finishing the previous one. Since the data in each distribution is unique, it is not a requirement to unpack them serially but rather, how it’s simply been done (for over 15 years).
When bsdinstall(8) improved the UI to provide a multi-progress widget, the obvious question of concurrent parallelism should be raised. From a UI standpoint, once you have a widget that can display progress for multiple items, the next logical step is to allow the “Pending” items to be processed without delay. In other words, the UI we have today for distfetch, is a great starting point but we can take it further.
For example, view below a prototype that I am developing:
NOTE: Both dialog(1) and dialog(3) render ANSI bold as gray on the system console in 9.0 or higher. Compare to the following image, executing the same program via SSH in which bold is exactly what you’d expect.
NOTE: The above was taken with a value of en_US.ISO8859-1 for the LC_ALL environment variable, nicely resulting in comma-separators in the KBytes/sec status line entry. See localeconv(3).
The prototype that I am developing can also replace other [less than informative] areas of the system. For example, the following picture shows how [deprecated] sysinstall(8) told the user that it was downloading the 24MB INDEX file (commonly via FTP):
Neither deprecated sysinstall(8) nor new bsdconfig(8) provide any information when [down]loading packages/INDEX. There’s no progress bar, no indicator that data is moving, and no data/rate info. My prototype application (named fdpv(1)) can improve the user experience here by providing those things (pictured below).
However, perhaps the best use for such a tool is to improve the way packages are installed. Pictured below is deprecated sysinstall(8), installing a single package.
Not quite as blunt as the fetching of packages/INDEX; this time at least sporting data/rate information in the status line. However, moving forward and away from sysinstall(8), let’s see what replacement bsdconfig(8) has for us:
I plan to fix the regression by introducing fdpv(1) as a new back-end for package installation. Here’s what it could look like:
NOTE: The above is a Photoshop‘ed image. Concurrency has not yet been programmed.
To better understand the fdpv(1) prototype, here is a picture with annotations showing the layout of information and where it is drawn from (no pun intended).
As you can see, the fdpv(1) tool is just a conduit. As such, the tool can be used for both legacy package management as well as pkgng (new default package management tool in FreeBSD).
For the sake of discussion regarding package management, notice the re-introduction of the status line (at bottom) to provide data/rate info (enumerating total throughput) as well as concurrent parallel processing of multiple dependencies during the installation of xorg (pictured).
This should dramatically reduce the amount of time required to install packages because independent dependencies, whom already have their own dependencies satisfied, can be installed in-parallel rather than waiting.
Just how many concurrent workers are allowed? Well, that would be up to Grand Central Dispatch (GCD). Programmed to use GCD, the utility should allow transparent scaling, respective to hardware capability and machine load.
Of course, I have to admit to not knowing GCD yet, so I’ve cast a fellow FreeBSD committer to help with that end. I was originally planning on using POSIX Threads aka pthreads and fancy mutex signaling. But Grand Central Dispatch looks much more promising.
What I’ve accomplished thus far (as of tagged-version 0.5) on this project is the following:
What’s not done however, is the following:
- For package management, a command-line utility is inadequate. The
main()portion of the utility needs to be translated into a library (libfdpv) allowing the integration of libpkg.
- Concurrent handling of file arguments via Grand Central Dispatch.
- Actual handling of I/O (reads/writes on file paths).
Those things need to be finished before we can import this new tool to the FreeBSD base system. Once those things are finished and fdpv(1) is in the base system, we can then start rewriting bsdinstall(8) and bsdconfig(8) to use it, providing enhanced I/O data feedback where we need it most (the installation of the system and package maintenance, respectively).
Pure bonus is the ability to work in X11 (shown below).
This matches bsdconfig(8)‘s already built-in ability to work entirely in X11, meanwhile bsdinstall(8) has to slowly be reworked to support X11 (by introducing new tools such as fdpv(1) to replace the existing C components that currently support only dialog(3)).
The code repository for fdpv(1) is at the below address:
NOTE: There is a “Download GNU Tarball” link at the bottom as well as a pull-down menu for selecting a “Sticky Tag”, allowing easy download of tagged revisions.
We’ve had an in-depth look at Data I/O Feedback, in the next blog post we will be switching over to another User Interface issue, Internationalization.
Thank you for reading.
While FreeBSD-9.1 has recently been released, I find myself still working in FreeBSD-8 (and at the time of this writing, 8.4 is soon to be released). This article is based on work performed in FreeBSD-8.1 (only because we froze our development environment for several years to prove the technology that I’m going to talk about in this article).
The new optional feature is called “VNET” and it allows each jail to have a private networking stack. However useful and exciting this may sound, these VNET jails require you to “Bring Your Own Network Interface.”
The VNET feature allows you to move network interfaces in/out of the view of a jail. When a network interface is moved into the view of a jail, it is no longer visible to the host of said jail.
A VNET jail starts with an empty network stack.
For example, you can create an ad hoc persistent VNET jail using the following command (requires
VIMAGE option enabled in kernel — more on that later):
jail -c vnet name=vj1 host.hostname=vj1 path=/ persist
NOTE: If your kernel is not compiled with the
options VIMAGE line, then you’ll get an error of
jail: unknown parameter: vnet).
This jail is now a special jail, called a VNET jail. In this jail,
ifconfig shows a private network stack which is currently empty except for an unconfigured local-loopback interface (
You can now move network interfaces into that jail using the below ifconfig syntax from the host:
ifconfig <interface_name> vnet <jail_id>
Meanwhile, you can move the network interface back out-of that jail using the below syntax from the host:
ifconfig <interface_name> -vnet <jail_id>
-vnet to recover an interface versus
vnet to relinquish an interface to a VNET jail.
<interface_name> is something like
igb0, etc. and
<jail_id> is the value shown in the “JID” column of the output produced by the
jls utility (executed without arguments).
However, the usefulness of this is still very limiting. With the VNET feature alone, all one is afforded is the ability to move whole network interfaces into and out-of VNET jails. When one does this, the jail and host do not share the network interface but rather the host can no longer see (using
ifconfig for example) nor use the network interface.
To alleviate this limitation, a bridging technology is required to allow the host and VNET jail to share a network interface.
Enter netgraph(4), another optional component in FreeBSD — graph based kernel networking subsystem — publicly available since FreeBSD-3.4. By adding a few NETGRAPH options to the kernel we can provide the VNET jails a solid bridging layer able to provide the jails fresh/functional network interfaces.
However, we’re still not to a point of usefulness because the following gaps still exist:
- How to automate the bootup, shutdown, and management of VNET jails?
- How to automate the creation, destruction, and bridging of netgraph devices?
- How to make sure bridged interfaces have unique MAC addresses?
- How to allow multiple VNET jails to have the same root directory? (considering each sets their own IP from rc.conf(5))
Enter vimage, software I’ve written to tie VNET jails together using netgraph.
NOTE: My “vimage” boot/management script (which lives in /etc/rc.d and oft-executed via “service”) should not be confused with the “vimage” utility from the VirtNet project (imunes.tel.fer.hr/virtnet/).
My vimage is a fork of the jail rc.d script — rewritten to work only with VNET jails and help you harness the most flexibility in network topology and jail structure.
HINT: If you are familiar with the IMUNES project (http://www.imunes.tel.fer.hr), both their work and my own performance-test results were factored into the ultimate choice to settle on netgraph for our bridging needs.
My vimage boot/mgmt script is available here:
Let’s get started…
First, you’ll need a custom kernel for your FreeBSD host before you can run any VNET jails, let alone vimage jails (VNET jails with netgraph).
Add the following to a GENERIC FreeBSD kernel config as a minimum:
NOTE: In the future, when
VIMAGE is compatible with
SCTP, you will not need the
nooptions SCTP line.
UPDATE: A known issue is that the non-default
options IPFILTER conflicts with
VIMAGE in stable/9 (and possibly HEAD too) in that a kernel panic occurs during boot. A backtrace shows the trap originating from
sys/contrib/ipfilter/netinet/fil.c. My current recommendation for running a kernel with both Firewall abilities and
VIMAGE is to use the
IPFIREWALL family of options and ipfw(8).
Compile your kernel and boot it. Next, grab the FreeBSD package for vimage-1.4 and use the following command to install 2 files (
Now we’re ready to build a jail. Personally, I use my own “jail_build” script to build jails from binary releases. You can get
jail_build from my SourceForge page:
NOTE: If you are planning on running i386 jails under an amd64 kernel, be advised that you may need a patch to your amd64 kernel to solve a problem with applying a default gateway using the 32-bit
route command (as discussed here).
jail_build works with any release, 8.x and older (sorry, no 9.x support yet).
How you use
jail_build in a nut-shell:
mv jail_build.txt jail_build
chmod +x jail_build
mkdir -p /usr/repos
# For example
# Now pick a single base-URL from below:
# Download "base" directory from FTP to local "."
# Download "doc" directory from FTP to local "."
# Download "dict" directory from FTP to local "."
# Download "games" directory from FTP to local "."
# Download "info" directory from FTP to local "."
# Download "manpages" directory from FTP to local "."
# Download "proflibs" directory from FTP to local "."
# Download "kernels" directory from FTP to local "."
You should now have:
jail_build will automatically probe for
NOTE: You can certainly run older releases as a jail, but generally speaking you should not run newer releases than what your host is running. So if you build an 8.3-RELEASE jail, you should have at least an 8.3 kernel or higher.
jail_build brings up a menu of repositories to select from (living in
/usr/repos) and allows you to select one before prompting you to enter the directory you wish to unpack the contents to (creating a new jail root directory). The default directory is
<jail_hostname> to the fully qualified hostname of the jail — this is suggested for easy tracking but feel free to make this whatever you like).
NOTE: You don’t have to use my
jail_build technique to build jails, in fact there is a long-standing tradition of populating jails from source-code. For details on how to build a jail from source, see the Creating and Controlling Jails section of the FreeBSD Handbook.
After you’ve created a jail, it’s time to configure it as a vimage.
In the host’s
/etc/rc.conf file, add the following:
vimage_enable="YES" # Set to NO to disable starting of any vimages
vimage_list="vj1" # Space separated list of names of vimages
vimage_vj1_rootdir="/usr/jail/vj1" # vimage's root directory
vimage_vj1_hostname="vj1" # vimage's hostname
vimage_vj1_devfs_enable="YES" # mount devfs in the vimage
This configures your basic vimage (we’ll cover how to give it a network interface in a moment). You can compare this directly to the procedures documented in the Creating and Controlling Jails section of the FreeBSD Handbook.
The same rules that apply to
jail_list in the Handbook apply to
vimage_list. The methodology is the same because again,
/etc/rc.d/vimage is a fork of the documented
Next, if you only want to move whole network interfaces into the vimage when it is started (and implied, automatically move the interface back out when the vimage is stopped), add the following to the same
rc.conf file as above:
vimage_vj1_vnets="igb0" # list of interfaces to give to the vimage
However, the real flexibility comes from using the bridging option based on netgraph. To create a bridged interface for the vimage — leaving the original interface on the host unaffected — add the following instead of the above:
vimage_vj1_bridges="igb0 igb0" # list of interfaces to bridge
In the above example, a single port physical Intel Gigabit network adapter is listed twice — this will create two unique bridged interfaces from the same physical interface. This is not an error and is perfectly valid (imagine simulating a router that runs on two subnets but over the same physical wire).
If you start the vimage at this point (using
service vimage start) you can get a list of the bridged interfaces from the jail’s point of view by executing:
jexec vj1 ifconfig
You’ll see (in the bridging example) two network interfaces, one named
ng0_vj1 and the other
ng1_vj1. The naming convention for bridged interfaces is
# is a counter that starts at zero and increases by one for each bridge (regardless of the device being bridged) and
NAME is the vimage name as seen in
NOTE: Due to an internal limitation, the name of any network interface in FreeBSD cannot exceed 15 characters. For long vimage names, be aware that
ng#_NAME will be truncated to be less than 16 characters if necessary.
PRO-TIP: By placing the vimage name within the name of the bridged network interface, it makes it simple to configure multiple vimages to use the same root directory.
The network configuration for the above vimage is configured within the vimage’s own
/etc/rc.conf file, example below:
ifconfig_ng0_vj1="inet 192.168.1.100 netmask 255.255.255.0"
ifconfig_ng1_vj1="inet 10.0.0.1 netmask 255.255.255.0"
Let’s say you had a second vimage named “vj2” pointed at the same root directory (and therefore the same
/etc/rc.conf file). You would then add the following, for example:
ifconfig_ng0_vj2="inet 10.0.0.2 netmask 255.255.255.0"
vj1 vimage jail would ignore the
vj2 entries and vice-versa.
At this point, let’s recap — you should have done the following:
- Compiled, installed, and executed a custom kernel that enables the VNET and netgraph(4) requirements.
- Installed my vimage package from druidbsd.sf.net.
- Created a jail using either the source-method mentioned in the FreeBSD Handbook or using
jail_buildand a binary release.
PRO-TIP: You can actually get by in testing without building a discrete jail but instead use “
/” as the jail root path. This is perfectly valid and acceptable.
- Configured the
vj1test vimage in
vj1to bridge at least one physical interface using
vimage_vj1_bridges(in the example above, we use
/etc/rc.conffile within the
At this point, the vimage will boot with the rest of the machine and can be controlled after boot with the following syntaxes:
# Stop, start, or restart all vimages
service vimage stop
service vimage start
service vimage restart
# ... versus: Stop, start, or restart just the vj1 vimage
service vimage stop vj1
service vimage start vj1
service vimage restart vj1
vj1 vimage starts, the
ng*_vj1 network interfaces will automatically be created with unique MAC addresses and moved into the VNET jail before kickstarting the FreeBSD boot process within the jail. As the jail boots, it will automatically configure the network interfaces through rc.conf(5). You can use jls(8) to see the running vimages and you can use jexec(8) to execute processes inside them (like tcsh(1), ps(1), and ifconfig(8)).
PRO-TIP: ps(1) can produce the JID of all running processes with the syntax:
However, what if you want to SSH into the vimage jail? SSH is not automatically started inside the vimage (and in-fact, only the network services required to get the vimage talking to the net are started). The answer is to add a new configuration line to the host machine’s
/etc/rc.conf to configure
sshd to be started in each/every jail:
vimage_afterstart_services="sshd" # set new default for all vimages
The default for
vimage_after_services is NULL but can be set to a space separated list of services (names of scripts living in
sshd). If you wanted to leave this the default and only have the
vj1 vimage start
sshd, you can instead opt for the following line:
So now we have a usable framework for creating multiple vimages that can be SSH’d into as (let’s say) development environments. When you scale this out, netgraph starts to shine. Afterall, netgraph is based on graphs.
Without any additional work, simply by using vimage to produce VNET jails with netgraph bridged interfaces, we can produce graphs of the vimage network topology with the following syntax:
ngctl dot | dot -Tsvg -o vgraph.svg
PRO-TIP: The dot(1) utility can produce more than SVG (like PNG, JPEG, and GIF), but I find SVG to be the most scalable and informative (modern SVG viewers such as latent browsers display much information as tool-tips which I find helpful).
Here are some graphs of different topologies we’ve used over the years:
It’s worth noting that if an interface is not bridged, it is shown in the “disconnected” cluster. This does not imply that the network interface is unused within FreeBSD — just that it has not been connected to any nodes within the netgraph layer.
That’s it for now, thank you for reading. Comments welcome. Depending on comments, I may do another installment showing the more exotic things you can do with this configuration.
FreeBSD. It is Free (in the truest sense; with a capital-F) and is a descendant of Berkeley Systems Distribution (specifically BSD-Lite v4.2/4.4), which originated from the University of Berkeley, about 2.5 miles away from where I live (though I’m not a student, nor have I ever been). FreeBSD is an amazingly versatile and flexible UNIX operating system with ancestry reaching back decades.
This entry is for those persons already well versed in this well-edified system.
I would like to announce a completely new — written from scratch — boot menu for the FreeBSD operating system. It is today that I release the code that I’ve been developing for nearly 5 years. Code to enhance the level of options provided to the user at each and every boot of this venerable OS.
Historically, the original menu offered by FreeBSD featured our beloved helper-daemon mascot, named “Beastie.” But soon after, this menu was revamped to [a] lay-rest our triton-carrying/sneaker-wearing assistant in favor of the less-controversial textual “FreeBSD” banner and [b] imbue the menu with new options.
Currently, the default boot menu offers booting normally as well as options for the following:
- Booting with ACPI support disabled (if you’re running an i386 compatible system)
- Booting into “Safe Mode” (implies disabling ACPI)
- Booting verbose
- Booting into single-user mode
However, one of the things that is not currently possible is the ability to boot into any combination of the aforementioned options. That is to explain, you cannot boot into single-user mode with ACPI disabled but verbosity enabled. The reason for this is obvious as once a menu option has been selected, the system proceeds to boot with those options rather than allowing additional selections to be made. See below.
Enter my new menu (pictured below in the following paragraphs), first created in 2006 internally for a proprietary custom installer, now generalized for the rest of the world and being released for all to enjoy. This new loader does not proceed to boot the system after selecting a menu option but rather updates the menu to display the option status, allowing the user to toggle any number of multiple options before pressing ENTER to finally boot with the displayed options.
And for those like myself that balked when Beastie exited stage-left, I have a nice surprise. Some time back, the FreeBSD community held a contest for artists to create a new logo for the FreeBSD project. The results of which can be viewed here: http://www.freebsd.org/logo.html
When the contest ended, I had my fun in running the graphics through some Image-to-ASCII conversion programs without much luck. Months later, I doubled-down my efforts still to no avail, attempting to create an ASCII-art version of the new orb to display at boot time. Six months later, I succeeded in producing a bi-color ASCII-art representation of the “horny orb” logo easily recognizable as the FreeBSD logo. This gem (as well as a B&W copy) have been added to the latest “beastie.4th” module for your viewing pleasure and is enabled by default upon installation.
Here you can lay your eyes on the final edition of the first-offering being made to the public today:
Other additions that you will notice are that the replacement menu does not load right away. Another enhancement is that there is a 2 second delay before launching the menu where the user can press Ctrl-C to escape the menu, allowing immediate booting or escape to the loader prompt. Something that the current menu does not allow for. See below.
The new menu is a system of ANS/FICL Forth (a reverse-polish stack-based language) modules that extend (but do not replace) the standard FreeBSD boot loader (/boot/loader). An interesting note about this package is that it will backup any files that will be replaced during installation to the file /var/backups/loader_menu.backup.tgz. Should you decide that you do not like the new and improved boot loader, you can uninstall it, returning your system back to the original boot loader by using the pkg_delete(1) utility (the backup will automatically be restored upon pkg_delete(1)).
With this initial offering, there are only two short-comings which should certainly be livable for most if not all:
1. The menu that we are replacing only conditionally displays the option to disable ACPI support (based on whether you are running an i386 compatible system), whereas the new loader always presents this option. This should be of no concern unless you’re running a non-i386/amd64 system (such as sparc, pc98, ppc, etc.).
2. The ability to boot into single-user mode by pressing “s” and boot verbose by pressing “v”, etc. is not enabed. The proponents for these options on the FreeBSD-Questions mailing list were clear in that they liked these options because they allowed “one-key” booting into a special mode. However — with the new menu — even if I enabled the use of those keys as shortcuts for toggling these options in the new stateful menu, the user would still have to press an additional key to boot (‘1’ or ENTER). So the primary principle use — one-key booting into a special mode — is no longer capable, unless I add these keys not as togglers for the visible menu items but invisible hot-keys for immediately booting into single/verbose mode. I’ll have to sleep on that (maybe as a concession for power-users as invisible options are obviously less than intuitive).
However, don’t let those misgivings dissuade you. All but the most ardent and stringent of surveyors will find those to be shortcomings and overall I predict your experience will be a rather pleasing one.
Some other nice additions are that if/when you select the option to escape to the loader prompt, you are given a way to get back to the menu (simply type “menu” and press ENTER), allowing you to perform work under the hood and then return to the menu as desired (without losing track of the status of the menu options even). As shown below:
Overall, the new stateful nature of this menu over the old makes it far less likely that a user will need to reboot if they hit the wrong key. The only mistake that could result in wasted cycles would be if they prematurely hit #1 to boot or hit #7 to reboot. All other options would simply toggle a menu item (except for #6 to escape to the loader prompt, to which they can immediately return to the menu if desired).
Last, but certainly not least, I’m including a completely rewritten password-checking module to replace the existing one (I couldn’t get the standard one to work). See below.
I hope that you like what I’ve developed, and your feedback is certainly welcome. I’ll be cross-posting this to the FreeBSD mailing lists, announcing in RSS feeds, and updating http://druidbsd.sourceforge.net/ all simultaneously to coincide with this blog-post.