>How to install pfsense 2.0 on a Watchguard x750e Core

>I picked up a used Watchguard x750e a while back at a flea market (ham fest) for really cheap. I wasn’t sure what I was going to do with it. I was just excited about finding an fairly good enterprise level firewall at a ham fest.

Fast forward about a year. I came across an article about modding the x750e. After digging deeper I found that people were installing pfsense on them. OMG! I took apart the x750e and sure enough it is essentially just a 1.4 GHz celeron with 512MB ram and and 256MB SanDisk CF card in it. (plus a few other things)

I looked around for a how to but all I could find was a bunch of hard to follow forum posts (and one pretty well documented process, but it seemed to be missing some stuff). Not that they were bad information (I mean I was able to get the thing working based of them), but it was all a bit disconnected.

So here is my walk through. If I’m missing anything or if I plain got something wrong please leave a comment and I’ll fix the post.

OS used: Windows XP (for hyperterminal. However any terminal capable app will do).
Hyper terminal settings: Outside of the baud all settings were default.
Pfsense: current stable release (2.0.1 as of this article)
Replacement CF card: SanDisk 4 GB Ultra (due to pfsense needing at least a 512MB installation target with current requirements.)
Card reader: Dynex all in 1. (just a standard usb multi-card reader)
Files needed: Most of the files needed can be found here: http://documentation.dbernhardt.com/pfsense/article.html (This was by far the most complete how to for installing pfsense on a Watchguard that I found.)
Serial cable: I used one of the billions that accumulated from working with HP Procurve network equipment.

First take the top cover off the x750e.

There are a bunch of un-soldered connectors for things like VGA, USB, etc… which can be connected if desired. However for this I wanted to avoid soldering. :)

Remove watchguard supplied CF card (256mb in my case) and insert into CF card reader on your computer. (I used a Dynex card reader from the local electronics store). Disclaimer: you are about to wipe out the factory software that came with your firewall!

physdiskwrite freedosbios.img

Once it’s done writing the image, copy X750EB6.BIN to bios directory on cf card if it’s not already there. (Note: There are several x750eB#.bin files that can be used. I’ve seen reports that 2 and 4 also worked for people. Only 6 worked for me.)

Place the 256MB CF card back in the firewall.

Connect to the console via serial using 9600 baud and power on the firewall.

Just a FYI to everyone. Yes you are about to flash the bios. This really doesn’t change anything, it just enables the console output for the bios so we can change some hard drive settings to allow the firewall to boot off a larger media. If you don’t want to do this, but still want to run something other that the factory software, you can flash Monowall on the 256MB CF card. Monowall requires much less space that pfsense and will run with no bios modifications.

Once freedos has booted you will see a C:\ prompt:

C:\>cd bios
C:\BIOS>awdflash.exe X750EB6.BIN /py /sn /cc /e

C:\BIOS>

Power off the firewall and remove the 265mb CF card.

Put the 4GB card into the CF reader.

Flash pfsense onto 4 GB card

physdiskwrite -u filename.img

-use -u for cf cards larger than 2gb
-filename.img should be replaced with the name of the pfsense image you downloaded.

Place 4GB CF card in the firewall now that it has pfsense on it and connect to the console via serial at 19200 baud.

Power on the firewall.

You should see the post in a few seconds. Instead of pressing del to enter the bios press tab since you are on a console.

Go into what is normally the hard drive settings and you should see the CF card detected. If it’s not or all values are set to 0, set to auto detection and it should set some values. Now change the following:

Set for:

IDE Channel 0 Master      [Manual]
Access Mode               [CHS]
Head                      [    2]

Save the BIOS and exit. (If you ever need to remove the CF card again you may have to go back in and reset the above values.
Bios will be on 115200 baud from now on after first boot just in case you ever need to get back in there:

Connect to console on 9600 to do initial pfsense configuration. If you see pfsense booting then you should be good to go.

The initial configuration is done from the console. (obviously configuration are going to differ after this point. the rest of this article is just for getting a basic configuration up and running.

Just leave the WAN set to DHCP for now, and give the LAN interface the normal 192.168.1.1 address.
For interfaces I like to use SK0 as the wan interface and SK1 and the lan interface. (SK0 is first from the left of the 8. SK1 is the second from the left.

To test you can connect a computer to the LAN interface and see if you get an IP address. If so point your browser at https://192.168.1.1 and you should get a pfsense login page.
Login: admin
Password: pfsense

The last thing I’d recomend doing is enabling SSH access. You will need to it fix the fan noise.

pfsense will take you through a small wizard to get the rest of the basic configuration up and running.

Post x750e install tasks/fixes:
LOUD NOISES!
The 3 10000 RPM fans in the back of this box present a problem. Noise. There is a wonderful little program that someone wrote called WGXepc. It allows you to clock down the fans in software when pfsense boots. Here is the basic setup:

Quick note about fan speed. (The default speed is FF. We are going to set the speed to 10. That may be too slow though. You should monitor your environment and set accordingly based on need.)

Add the following line to WGXepc.sh
/usr/local/etc/rc.d/WGXepc -f 10

Connect using WINscp or ther secure copy tool as root and your pfsense admin password.

copy WGXepc and WGXepc.sh to using WINSCP:
/tmp

Login using through SSH (I used putty) with root and your pfsense admin password.

Mount FS as read write
/etc/rc.conf_mount_rw

cp WGXepc /usr/local/bin
cp WGXepc.sh /usr/local/etc/rc.d/

chmod 0755 /usr/local/bin/WGXepc
chmod 0755 /usr/local/etc/rc.d/WGXepc.sh

/etc/rc.conf_mount_ro

shutdown -r now
or
run this to set the fan speed now: WGXepc -f 10

All changes done with this command are immediate. Also a system update will remove these settings. You will have to reapply them.

WGXepc will also control the Arm/Disarm light on the front of the x750e. :)

Here are the arguments the command accepts:

-f (fan) will return the current fan speed or if followed by a number in hex, 00-FF, will set it.
-l (led) will set the arm/disarm to the second argument:
red, green, red_flash, green_flash, off

Clock down the CPU
In the web interface goto System->Advanced->Miscellaneous.
Enable PowerD

If you see the following in the system log happening a lot:

kernel: timecounter TSC must not be in use when changing frequencies; change denied

Add the following to Advanced->System Tunables to fix the issue.

Tunable: kern.timecounter.hardware
Value: i8254

Sources:
http://nettechonline.net/index.php?option=com_content&view=article&id=78:x700-led-fix-pfsense-on-watchguard&catid=49:pfsense-watchguard&Itemid=58
http://forum.pfsense.org/index.php?topic=32013.65;wap2
http://documentation.dbernhardt.com/pfsense/article.html
http://doc.pfsense.org/index.php/Remount_embedded_filesystem_as_read-write

There you have it. You should have pfsense working on the x750e hardware. Also if you don’t want to go the full route of flashing a new bios, you can also put Monowall on the 256MB card that comes with the firewall as mentioned before. Monowall requires a lot less space than pfsense. There are some features you won’t get with Monowall that exist in pfsense UPNP and OS fingerprinting for example, but Monowall is a really solid firewall platform. pfsense was actually forked from Monowall.

The most aggravating part of all this work was finding a working computer that still has a COM port. :)

Update 2012-02-20: A couple thing to add. First I didn’t realize this but it appears that there is no way to re-size the tmp volume. This is significant because it’s one of two volumes on pfsense embedded that is created as a ram disk (the other is var). This is significant because tmp is where squidguard and other packages store their updates/blacklists. Essentially if you are going to use pfsense in an embedded configuration (on CF card for example) then you are setting up for access control type functions only. Although I was able to get snort working on the x750e with CF card.

Incidentally if you do a df -h from the console, you will see a total of only about 2 GB total for the entire file system. If you are using a 4 GB card the other 2 GB is used for creating a separate slice that can be booted for testing configs and other stuff. Its kinda handy if you have a working config and want to try something out without breaking your “production” config.

I guess I’m going to start researching mounting a SATA or IDE drive in the x750e.

2012-7-14 Update: The other day I was looking at the dashboard and I noticed that I’ve been running this setup with Snort active for 90 days without reboot. :) So far I’ve been very happy with the setup. Snort has been a little temperamental and there have been some rules I’ve had to create exclusions for, but over all this thing has been rock solid.

2013-04-08 Update: Just did an in place upgrade to 2.0.2. I had to reapply the fan speed and LED tweaks, I read that was going to be the case. I also just came across this great article that explains simply how to get LCDproc working correctly. Never thought about installing both the stable and dev packages… :) http://www.alteredrealms.com/2012/11/07/watchguard-pfsense-tweaks/

24 thoughts on “>How to install pfsense 2.0 on a Watchguard x750e Core”

  1. Used this today to rebuild a retired x750e with pfSense at work. Worked perfectly. Have you made any headway on the IDE drive mounting? I have a 60GB IDE laptop drive hooked up to IDE2, but the BIOS doesn’t even see it.

    1. Cool, i’m glad the walkthough helped. I’ve been running on the the 4 GB CF card shown in the picture. Because the embedded image sizing doesn’t work with some plugins that require lots of space (definition downloads, etc), I’d really like to get a drive to add to the device, I’m not sure about the second IDE port but I would think it should be possible. Unfortunately I don’t have any information on that yet. I’ll write something up if I ever get time to play with the IDE interface. That may be a little hard right now since the watchguard is now the router for my house. :)

      Question for you actually. Have you seen the device work fine for a few days and then all of a sudden the link quality gets so bad that latency increases to something like 300ms? Mine seems to do that every 1-3 days. The only fix I’ve found is to reboot.

      Update: Wow I can’t believe I missed this. The horrible latency issue was caused by my cloud backup software. I guess I need to be more mindful of how much bandwidth I give some apps. :)

      1. I haven’t gotten to use my pfsense firebox for an extended period yet. Got it working and then racked it in my mobile box. I’ve been fairly busy the last couple of months and haven’t even tried looking at this. But we’ve been having issues with our firewall at work, so I’m bringing the pfsense box in until we get it replaced. Might have some extended testing now.

        Glad you found your latency issue. I was worrying about bringing the box to work if it was just going to do that.

        1. So far mine’s still been running pretty well. I had to reboot it the other day. I’m not sure how long it’s been up but it’s been going pretty good. I’m not sure I’d trust this to a production environment without the failover working, however it does seem pretty solid. I’d be curious how removing/disabling snort would affect my runtime between reboots/service restarts. Snort it pretty resource intensive.

    1. To be honest I can’t remember. :) it’s been a while since I had it taken apart. I’m not 100% sure but i think the second set of 4 interfaces is using a PCI slot, but I don’t think it has a mini-PCI slot. Maybe someone else can clarify?

  2. I believe the x750e has a 4x PCI Express slot, as well as two IDE slots. One of the IDE slots takes a somewhat proprietary adapter. Was 4GB of flash memory enough for you to store things like IDS logs, system logs, and squid? I picked up one of these because 8 port Gb directly on my firewall would be nice. However I found that I needed(wanted) the greater amount of space (after looking at my current pfsense config) that my more power hungry DL320 is able to provide. Great tutorial!!! Thanks for sharing the knowledge.

    1. Thanks for the info. Now that you mention it those specs are sounding familiar. :) I to have found the space limitations of the embedded install to be somewhat annoying. There really aren’t many plugins that can be installed when using the CF card version. Right now the only thing I’ve installed is snort. I think for my next build though I’ll use a real drive. :)

  3. Very cool. A few months ago I started at a company that has two of the X750e units. I’d like to drop WatchGuard altogether (it sucks) and go with something different, and this seems like a very practical way to go. I wonder if it is possible to create a fail-over cluster with the two units?

  4. Brent, While I’ve never used one of these in an enterprise situation, pfsense does support hardware failover. It calls the feature CARP. http://doc.pfsense.org/index.php/Configuring_pfSense_Hardware_Redundancy_%28CARP%29

    It should be perfectly doable running pfsense on the watchguard.

    I have to agree about the watchguards in general. The factory software while not the worst enterprise firewall OS I’ve ever used (secure computing – pre-mcafee) it’s certainly not the greatest.

    One thing of warning, the embedded (CF card install) is not large enough to run most plugins. If you want to run more than just a layer 7 firewall, I’d strongly recommend finding a way to put a hard drive in watchguards rather than a CF card. Or better yet, build a couple firewall appliances. I was looking at some of the supermicro chassis on newegg today, some of them might make a decent pfsense firewall with the right hardware. If you don’t care about most of the plugins then the x750e works great for pfsense.

    1. Just a regular FW will do it for me. Question: did you get the LCD, LEDs, and buttons working? I am looking into LCDProc but the instructions are terribly scattered and inconsistent.

      Thanks for the work you put into this guide!

      1. Sadly no. Clear, or even partially clear, instructions have eluded me so far when it comes to LCDProc. I’ve heard people have gotten it to work, but apparently there are a few things that I just don’t understand well enough to make it work right now. If I ever find a solution I’ll update the post.

        Thanks. It’s always good to know that a post was able to help someone. I was a little surprised by the amount of hits this post gets. :)

        1. Mike, feel free to shoot me an e-mail at brent (at) kpg dot com. After sifting through 100+ posts, I finally figured out how to get the LCD and LEDs working. I’ll send you the files and instructions so you can include them in your guide (if you want to).

  5. Mike, I read your article but I am unable to get freedosbios to boot up. I am wondering If I could get some help.

    Thanks
    Hakim

    1. To be honest I don’t really remember to much about that part of the install. the watchguard with pfsense has been very much a fire and forget kind of install process for me. I’ve seen reports that some CF readers have problems, but that would just be a guess.

    2. I just setup pfSense on mine and I ran into an issue with freedosbios. In the end it was my card was two big. I assumed since when i bought the system it was running off of a 512MB card that I could continue to use it for the BIOS steps. That was incorrect. I had to find a smaller card(luckly i had an old unused monowall that had a 32MB card) and then everything worked.

  6. Hi, stephenw10 here, I wrote WGXepc.
    The LCD an buttons are fully supported by lcdproc with the sdeclcd driver. I recently explained it in a forum post: http://forum.pfsense.org/index.php/topic,58426.msg313157.html#msg313157

    The msk interfaces, those furthest from the LCD, suffer from watchdig timeouts under high load. Check your logs. To resolve this add hw.msk.msi_disable=1 to /boot/loader.conf.local as referenced in the above forum post.

    Also modified drivers to correct NIC LED setup. :)

    Steve

  7. stephenw10, Thanks for writing WGXepc! It works great. Thanks for the tips. I haven’t had much time lately, but I’m going to put them into the FW soon.

  8. Where does the WGXepc.sh come from, and what exactly is in it? I tried lurking the forum and thought I found what needed to be in it but on boot the fan speed is not set correctly.

  9. Hi, have you had a chance of putting in a SATA drive in your firebox? Can you use any IDE to SATA adapter or are their specific brands that work.

    Thanks

  10. Pingback: Pfsense help ?

Leave a Reply