I recently found my FreeNAS box rebooting itself in the middle of the night (every night) and I wanted to know what was going on. I first tried enabling the debug kernel (see below picture) but that didn’t work nor did it get me any closer to understanding the issue at-hand.
The idea behind enabling the debug kernel was that it would drop me into the kernel debugger rather than rebooting 15 seconds after an unattended fatal error (leaving zero logs to analyze). But much to my surprise, the system rebooted anyway.
The next step was to set up remote syslog to allow FreeNAS to log to an external box. The first thing I did was connect my FreeNAS server directly to a FreeBSD system. The FreeNAS system will be configured to log everything to the FreeBSD system; so when FreeNAS reboots I will have a log of what lead up to that event.
I configured the secondary network interface on each server to talk on the same subnet. On the FreeNAS side, I went into the WebUI and configured the second interface of
em1 to have IPv4 address
192.168.1.2 with a
/24 subnet (
255.255.255.0). When that was done, it looked like the below picture.
After that, I logged into my FreeBSD server and prepared it to receive remote syslog data.
ifconfig em1 inet 192.168.1.1/24
echo 192.168.1.2 freenas >> /etc/hosts
sysrc syslogd_flags="-a freenas -v -v"
echo "+freenas" >> /etc/syslog.conf
echo "*.* /var/log/freenas.log" >> /etc/syslog.conf
service syslogd restart
Now SSH into your FreeNAS box and execute the following:
mount -uw /
echo 192.168.1.1 freebsd >> /conf/base/etc/hosts
echo 192.168.1.1 freebsd >> /etc/hosts
mount -ur /
Then perform the last final bit, which is to go into the FreeNAS WebUI and enter a value of “freebsd” for the syslog server (see below picture).
At this point, you should be able to execute via SSH on the FreeNAS system:
And get the following entry in
/var/log/freenas.log on your FreeBSD system:
Jan 26 04:42:37
freenas root: foo
Of course, the jury is still out on exactly why my system is crashing, but I suspect a failing hard disk (the logs being streamed to the FreeBSD system seam to indicate via smartd messages a prefail condition for the boot disk).