1. Monitoring FreeBSD Base System Vulnerabilities with pkg audit

    The FreeBSD base system has been difficult to monitor for published vulnerabilities for a long time. This will improve when we achieve a packaged base system, but that leaves users of currently supported -RELEASE systems without a standardized option.

    The freebsd-version(1) utility has existed since FreeBSD 10.0. This script is capable of correctly identifying the version of the FreeBSD kernel and the FreeBSD base system. It is an important step forward in helping users be confident in identifying the FreeBSD system's patch level.

    I do not like reinventing the wheel, and it occurred to me that for a long time the FreeBSD SA announcements were properly documented in vuxml. This provided an opportunity and scratched an itch I had at work, so here goes nothing:

    I am presenting here a useful albeit unsupported method of monitoring FreeBSD for base system vulnerabilities via pkg(8) utilizing entries in the vuxml database.

    The pkg(8) utility as you probably know can check your system for known vulnerable packages. It does this with the pkg audit command. Additionally you can pass any package name and version string as an argument and it will check the database for results. It is possible to check your system against the vuxml database by converting the freebsd-version(1) output to the correct string and passing it to pkg audit.

    Example of checking the base system (note, this is /bin/sh syntax):

    $ freebsd-version -u
    10.3-RELEASE-p2
    $ pkg audit $(freebsd-version -u | sed 's,^,FreeBSD-,;s,-RELEASE-p,_,;s,-RELEASE$,,')
    FreeBSD-10.3_2 is vulnerable:
    FreeBSD -- Multiple vulnerabilities of ntp
    CVE: CVE-2016-4957
    CVE: CVE-2016-4956
    CVE: CVE-2016-4955
    CVE: CVE-2016-4954
    CVE: CVE-2016-4953
    WWW:
    https://vuxml.FreeBSD.org/freebsd/7cfcea05-600a-11e6-a6c3-14dae9d210b8.html
    
    FreeBSD-10.3_2 is vulnerable:
    libarchive -- multiple vulnerabilities
    CVE: CVE-2015-2304
    CVE: CVE-2013-0211
    WWW:
    https://vuxml.FreeBSD.org/freebsd/7c63775e-be31-11e5-b5fe-002590263bf5.html
    
    FreeBSD-10.3_2 is vulnerable:
    FreeBSD -- Heap vulnerability in bspatch
    CVE: CVE-2014-9862
    WWW:
    https://vuxml.FreeBSD.org/freebsd/7d4f4955-600a-11e6-a6c3-14dae9d210b8.html
    

    Now we have results for the base system! Let's check the kernel:

    $ pkg audit $(freebsd-version -k | sed 's,^,FreeBSD-kernel-,;s,-RELEASE-p,_,;s,-RELEASE$,,')
    FreeBSD-kernel-10.3_2 is vulnerable:
    FreeBSD -- Buffer overflow in keyboard driver
    CVE: CVE-2016-1886
    WWW:
    https://vuxml.FreeBSD.org/freebsd/7bbc0e8c-600a-11e6-a6c3-14dae9d210b8.html
    
    FreeBSD-kernel-10.3_2 is vulnerable:
    FreeBSD -- Kernel stack disclosure in 4.3BSD compatibility layer
    WWW:
    https://vuxml.FreeBSD.org/freebsd/7cad4795-600a-11e6-a6c3-14dae9d210b8.html
    
    FreeBSD-kernel-10.3_2 is vulnerable:
    FreeBSD -- Kernel stack disclosure in Linux compatibility layer
    WWW:
    https://vuxml.FreeBSD.org/freebsd/7c5d64dd-600a-11e6-a6c3-14dae9d210b8.html
    
    FreeBSD-kernel-10.3_2 is vulnerable:
    FreeBSD -- Incorrect argument handling in sendmsg(2)
    CVE: CVE-2016-1887
    WWW:
    https://vuxml.FreeBSD.org/freebsd/7c0bac69-600a-11e6-a6c3-14dae9d210b8.html
    

    The results speak for themselves.

    I have recently finished adding all missing entries to the vuxml database that affect -RELEASE systems since 2013. This covers the tail end of 8.x, much of 9.x, and bleeds into the 10.x RELEASE lifetime. Systems older are End of Life and never supported the FreeBSD pkg(8) utility anyway, so I have not put in the effort to search out those missing entries. This method is useful on FreeBSD systems that do not have the freebsd-version(1) utility, but you will not have a reliable method to get the version of the FreeBSD base system. You can pull the kernel version from uname(1), but you will have to devise your own method of keeping track of the base system version. Beware of the leopard, etc.

    I hope you find this a valuable method for discovering vulnerabilities affecting your servers and help you assess risk and plan patch management. Please remember this is not endorsed by secteam and is liable to be full of errors or out of date. I suggest using this as a compliment to your other monitoring practices. Moving forward I hope to better coordinate with secteam to ensure we have new FreeBSD SA's entered in the vuxml database in a timely manner.

    This post originally appeared on the freebsd-questions mailing list and has been lightly edited.

  2. Java KVM Troubles on OSX

    I was having troubles on OSX getting access to the KVM at work which is a Dell KVM 4322DS. The connection errors from Java about security settings were not resolved by whitelisting the site in the Java console. Turns out newer Java disables some SSL/TLS algorithms and settings that break the connectivity (rightfully so). Unfortunately not even a fully patched KVM fixes this, and people still need to access KVMs, so a workaround is needed.

    After some Googling I came across some information that you need to edit the java.security file and change jdk.tls.algorithms.disabled. On OSX this file is at:

    /Library/Internet Plug-Ins/JavaAppletPlugin.plugin/Contents/Home/lib/security/java.security
    

    The setting originally looks like this:

    jdk.tls.disabledAlgorithms=SSLv3, RC4, MD5withRSA, DH keySize < 768
    

    I had to remove the "DH keySize < 768" to make this KVM work, but YMMV.

  3. Fixing Time Machine / Netatalk (error (null))

    Recently I was setting up a new Time Machine backup on my wife's MacBook so it would use my FreeBSD/ZFS server. My own personal MacBook was already backing up to it successfully and has been for quite some time. When I attempted to start a new backup to the server I received a strange error after a few minutes:

    The backup disk image "/Volumes/backup/Michelle's MacBook.sparsebundle" could not be accessed (error (null)).
    

    After digging through the OSX system logs I also found these entries:

    Feb 12 07:29:54 Michelles-Macbook com.apple.backupd[12798]: Attempting to soft mount network destination URL: afp://shell@skeletor._afpovertcp._tcp.local/backup
    Feb 12 07:29:55 Michelles-Macbook com.apple.backupd[12798]: Mounted network destination at mount point: /Volumes/backup using URL: afp://shell@skeletor._afpovertcp._tc
    p.local/backup
    Feb 12 07:29:57 Michelles-Macbook com.apple.backupd[12798]: Network destination already mounted at: /Volumes/backup
    Feb 12 07:30:25 Michelles-Macbook com.apple.backupd[12798]: Creating disk image /Volumes/backup/Michelle's Macbook.sparsebundle
    Feb 12 07:30:44 Michelles-Macbook com.apple.backupd[12798]: error processing extended attributes: Input/output error
    Feb 12 07:30:44 Michelles-Macbook com.apple.backupd[12798]: Error moving disk image from /var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/com.apple.backupd.syLaqH/Michelle's Macbook.tmp.sparsebundle to /Volume
    s/backup/Michelle's Macbook.sparsebundle - Error Domain=NSCocoaErrorDomain Code=512 "“Michelle's Macbook.tmp” couldn’t be moved to “backup”." UserInfo={NSDestinationFilePath=/Volumes/backup/Michelle's Macbook.
    tmp, NSUserStringVariant=Move, NSFilePath=/var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/com.apple.backupd.syLaqH/Michelle's Macbook.tmp.sparsebundle, NSUnderlyingError=0x7fc1a9529660 {Error Domain=NSCocoaEr
    rorDomain Code=512 "“0” couldn’t be copied to “bands”." UserInfo={NSSourceFilePathErrorKey=/var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/com.apple.backupd.syLaqH/Michelle's Macbook.tmp.sparsebundle/bands/0,
     NSUserStringVariant=(
            ), NSDestinationFilePath=/Volumes/backup/Michelle's Macbook.tmp/bands/0, NSFilePath=/var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/com.apple.backupd.syLaqH/Michelle's Macbook.tmp.sparsebundle/bands/0
    , NSUnderlyingError=0x7fc1a954d2e0 {Error Domain=NSPOSIXErrorDomain Code=5 "Input/output error"}}}}
    Feb 12 07:30:44 Michelles-Macbook com.apple.backupd[12798]: Failed to create disk image /Volumes/backup/Michelle's Macbook.sparsebundle, status: 512
    Feb 12 07:30:44 Michelles-Macbook com.apple.backupd[12798]: Network destination already mounted at: /Volumes/backup
    Feb 12 07:30:44 Michelles-Macbook com.apple.backupd[12798]: Backup failed with error 20: 20
    Feb 12 07:30:45 Michelles-Macbook com.apple.backupd[12798]: statfs call failed, error: 2 No such file or directory
    Feb 12 07:30:45 Michelles-Macbook com.apple.backupd[12798]: Ejected Time Machine network volume.
    

    I managed to recreate the issue by going into /var/folders/zz on the MacBook and finding a file with extended attributes. When attempting to cp it to /Volumes/backup, the location where the Time Machine backups are supposed to go, it produced an error which was easy and quick to reproduce. I investigated the netatalk documentation and found that the ea setting was auto by default. This was not sufficient. The fix was to edit the afp.conf and add the following:

    [default_for_all_vol]
    ea = ad
    

    After restarting the netatalk services I was able to successfully cp the file manually from the MacBook to my server which was the first indication that this problem was solved. The next attempt of a Time Machine backup was successful. Mystery solved.

  4. Mono's DNS is broken

    I recently started playing around with Sonarr by porting it to FreeBSD. It's a pretty well designed app -- I'm not aware of an alternative with such a well thought-out interface and deep integration into the other programs it communicates with. I've never run program via mono before, so I was pleasantly surprised it seemed to be "just working".

    However, I was checking out their forums when I stumbled upon this post indicating a mono bug regarding DNS:

    How does this affect me?
    
    This depends on which platform your running Sonarr on, for Windows users
    this should be pretty seamless, the DNS records will be updated and
    Sonarr will start using the new server automatically, for non-windows
    users Sonarr will still point to the old server until sonarr is
    restarted (this is due to a mono bug that caches dns records for the
    life of the app).
    

    Wow, that seemed weird. Sure enough, after some research I found that mono is stupid enough to have a functional DNS implementation but is not used by the ServicePointManager, a class that manages web requests. Instead, this moronic code caches DNS requests forever. Boy, that sure makes the concept of a cross-platform .NET/mono software that interacts with the internet a waste of time.

    Last I checked when I talked to the Sonarr developers they are about a month in on this IP change and there are still many thousands of Sonarr installations trying to communicate with their old server they're trying to retire. Ridiculous.

    I've actually found there's a patch in the mono codebase that fixes this problem, but it was never merged to the 4.2 branch; instead, it is rotting in master waiting for the next big cut which could be... well, who knows how long? sigh

    If you want to help me badger them into fixing this problem, go leave some comments here.

    How any software can exist this long without having a properly functional DNS implementation for when you actually interact with the internet is quite an achivement. But fear not, Java did this too through JDK 1.5!

    Lets hope that Microsoft's open source .NET implementation is successful. At least that way you'll (hopefully) get an identical experience across platforms.

  5. Routing a FreeBSD Jail through OpenVPN

    I decided I wanted to concoct a solution where I could force all applications in a jail or jails through a VPN connection without affecting the internet connectivity of other daemons on the system. After some headbanging I was able to make this work. The OS version being used in this example is 10.1-RELEASE.

    This post assumes you know how to setup an OpenVPN client & server as well as being familiar with jails.

    Enable multiple routing tables

    Multiple routing tables (fibs) are available out of the box these days but require you define them on boot. I only need two fibs, one for the system and one used by this jail. As many jails/applications as you want can share a fib; you don't need to make a new one for each new setup.

    /boot/loader.conf

    net.fibs=2
    

    Set the default route for the fib

    My OpenVPN setup handles redirecting the default gateway and adding routes to the client's routing table upon connection. In order for this to work successfully it needs to detect a default route. Here's the proper syntax so it starts on boot:

    /etc/rc.conf

    static_routes="vpn"
    route_vpn="default 172.16.1.1 -fib 1"
    

    Let's get the OpenVPN client running. This OpenVPN client exists outside the jail. This is required because you cannot alter routing tables from within a jail.

    When starting the OpenVPN client you will need to execute it via setfib so it is operating under the correct routing table. The openvpn-client doesn't daemonize, so I'll let you solve that whichever way you prefer. I have mine running under daemontools to keep it alive.

    server# setfib -F 1 /usr/local/sbin/openvpn-client /usr/local/etc/openvpn.conf
    

    My OpenVPN client provisions static IPs for each new client, so the IP address is not going to change. I believe this is standard behavior for OpenVPN, but you may want to consult their docs if you run into issues.

    The jail's IP address needs to be our end of the OpenVPN tunnel.

    /etc/jail.conf

    vpnjail {
        host.hostname = "vpnjail";
        ip4.addr = "10.8.0.14";
        exec.fib = 1;
        allow.raw_sockets;
    }
    

    edit: if your IP is dynamically allocated by your VPN provider, use ip4.addr = inherit;

    Now when you start the jail it will use the new fib.

    Please note if you use jexec to enter the jail you will be tricked into executing everything with the wrong fib! This is not well documented behavior. When the jail is launched and things automatically start they will use the correct fib, but if you jexec into the jail and run things from the shell it will not use the correct fib unless you setfib -F1 jexec when entering the jail!

    When in doubt, check your fib with sysctl net.my_fibnum !

    At this point you should have a fully functional FreeBSD jail with all network connectivity being pushed over the VPN.

    You may be asking why I didn't bother using a VNET jail with its own network stack and run the openvpn-client within the jail, too. The reason is that I want the network connectivity to completely fail if the vpn goes down. I do not want any chance of the traffic leaking. You may be able to do a VNET jail and solve this problem with some firewall rules, but that is additional complexity I did not want to introduce to this environment.

    Stay safe :-)

  6. Braindead FreeBSD Backups with Tarsnap and ACTS

    Before I start this simple blog post you should be aware that:

    Now that has been said, let's start with the super basics. Do you know Tarsnap is? No? Ok, let's get you started.

    1. Install ACTS and tarsnap.

      # pkg install acts
      

      or

      # cd /usr/ports/sysutils/acts && make install clean
      
    2. Go sign up for an account and deposit $5.

    3. Generate a key for backup. The key is specific to this machine. Give the machine parameter a name that is recognizable to you.

      # tarsnap-keygen --keyfile /root/tarsnap.key --user you@yours.com --machine your.machine.com
      Enter tarsnap account password:
      
    4. Backup /root/tarsnap.key. I'll wait for you to put a copy somewhere safe. Put it in your Keypass, 1Password, print it off and put it in a safe, or whatever you normally do here. (You do keep a copy of these important things in meatspace, right?)

    5. Configure what you want it to backup. Here's one of mine:

      # vi /usr/local/etc/acts.conf
      
      backuptargets="etc home root usr/local/etc usr/local/www"
      
    6. Setup a cron entry so this happens daily.

      15      23      *       *       *       root    /usr/local/bin/acts
      
    7. Run your first backup.

      # acts
      Creating yearly backup
      Backing up etc...
      Backing up home...
      Backing up root...
      Backing up usr/local/etc...
      Backing up usr/local/www
      acts run took 23 seconds
      

    Ok, whew, that was easy. Now you have a yearly backup. Next run it will generate a monthly, then start doing dailys.

    At this point you're done. Your data is safely backed up and even deduplicated.

    Want to make a key that can only be used to do backups but cannot be used to delete them? How about advanced techniques for data restoration? Passphrase protecting keys? Curious about implementing more advanced backup strategies? You could read the man pages and scrape the internet for tips, or you can just buy the book that has already done all the hard work for you.

    Now go on with your life. Your data is safe from even the NSA.

  7. IPv6 via 6rd on FreeBSD

    My ISP is Charter and they support 6rd for IPv6. Unfortunately 6rd support does not exist in the stf(4) driver in FreeBSD yet. There is a work-in-progress implementation available from hrs in ports, net/stf-6rd-kmod. However, I haven't found very good documentation on exactly how to use it so at the suggestion of nathanw I have impelemented 6rd via a regular gif(4) tunnel.

    The configuration in rc.conf looks like below. Just replace the variables with the real IP addresses for your environment:

      cloned_interfaces="gif0"
      ipv6_activate_all_interfaces="YES"
      ifconfig_gif0="tunnel $MYIPv4 $THEIRIPv4"
      ifconfig_gif0_ipv6="inet6 alias $MYIPv6 $THEIRIPv6 prefixlen 128"
      ipv6_defaultrouter="$THEIRIPv6 -mtu 1280"
    

    There is one main limitation because it's using a gif(4) tunnel and not speaking the complete 6rd RFC: you can't contact other IPv6 addresses that are also using this 6rd service.

    If you don't care about the capability to contact other users on your ISP also using the 6rd tunnel you will not notice any problems.

    Note: I'm setting the default route to an MTU of 1280 because of the behavior of IPv6 tunnels and common issues with PMTU. I suggest you do the same for an optimal experience.

  8. BSD License Audit

    I recently did an audit of the "BSD" licenses in the FreeBSD ports tree. This pertains strictly to those defined as LICENSE=BSD which could be one of several licenses. It was an extremely tedious process manually verifying the license of each port, and except for a dozen which are not identifiable or waiting for email responses from the authors it has been completed successfully.

    Things I've learned:

    • Lots of people don't understand open source licenses and incorrectly label their own license. BSD == MIT, etc.

    • Services like pypi don't get any more granular than "BSD" which made this audit frustrating and perpetuates the idea that there is a single "BSD" license. Go look in PKG-INFO files -- just says License: BSD.

    • Developers have this fantastic idea where they say "This project is under the BSD license" and then never point the enduser to any license text anywhere.

    • Many people are leaving their LICENSE or COPYING files out of their release tarballs -- incredibly daft of them.

    • BSD community members seem to know when you author software you license files not an entire projects, and put the license in the header of every source file. (Thanks!)

    • Some people think they can just edit standard licenses because they're smarter than the lawyers who helped develop these licenses and cause unnecessary work to myself and others. (ZPL2.1 with a clause cut out)

    • There are far too many variants of the MIT license.

    • OpenBSD actually uses the ISCL license, not a classic BSD license. (Don't worry, it's just shorter)

    • Even Debian can make mistakes. (That's not a GPLv3 license.)

    • Tons of copies of the BSD 3-CLAUSE out there that have clauses numbered 1., 2., and 4.. Makes me chuckle every time I see it.

    • An unofficial BSD 1-CLAUSE is floating out there in use by a few projects which indicates the author only cares about its source distribution and not the binary...

    • The Sendmail license had an older variant that implied that you have to fly to California to defend yourself if you violate it.

    • Never trust the license of a package. If you're a vendor you better verify it by hand before selling your product.

    Results:

       1 ART20
       1 BSD1
       1 BSD2 BSD3 ART10
       1 BSD2 MIT
       1 BSD3 TclTk
       1 CC
       1 CPL
       1 GPLv2 BSD3CLAUSE BSD4CLAUSE
       1 GPLv2 ISCL
       1 GPLv3
       1 PHP202
       1 PHP30
       1 Sendmail
       1 ZPL21
       2 BSD2 BSD3
       2 BSD3 MIT
       2 REPOZE -- ZPL21 modified
       4 GPLv2
       4 TclTk
       5 CUSTOM
       8 BSD4
      17 ISCL
      24 MIT
      62 BSD2
     148 BSD3
    

    This isn't 100% accurate either as sometimes there were ports which had multiple licenses defined and I only fixed and noted the "BSD" one. However, those that have multiple licenses listed were instances that I discovered that the project didn't fit strictly under one license.

    What a nightmare.

  9. Outlook-compatible WebDav with Nginx

    Microsoft Outlook has a Publish Online feature for sharing specific calendar information by publishing iCal files to WebDav. I don't use Apache on my personal servers, so here's how to configure it on Nginx.

    You first need to ensure that you have both Nginx WebDav modules installed. They are called http_webdav and webdav_ext. You need the webdav_ext as Outlook attempts some specific functions that are provided by this module. Other calendar clients may not have the same requirements.

    After you have both of these installed you can configure your webdav share like this:

    server {
            listen  80;
            listen  [::]:80;
            server_name cal.yourdomain.com;
            root /usr/local/www/caldav;
    
            location / {
                    dav_methods     PUT DELETE MKCOL COPY MOVE;
                    dav_ext_methods     PROPFIND OPTIONS;
                    create_full_put_path    on;
                    dav_access      user:rw  group:rw  all:r;
            }
    }
    

    You should probably secure this with SSL and possibly password protect your webdav share. Using the limit_except Nginx feature you could be clever and allow read access from everyone but prevent publishing without a password. This will prevent your server from being a target for public file storage. :-)

        limit_except GET {
            auth_basic "Restricted for authorized users!";
            auth_basic_user_file        /usr/local/etc/nginx/htpasswd;
        }
    
  10. Setting up Xymon with Nginx

    Xymon has been a favorite monitoring tool of mine for quite some time now largely due to its simplicity and flexibility. However, I despise running Apache unless absoultely neccessary. Previous attempts at getting Nginx and Xymon to play nice were not successful without some lazy hacks, but I finally sat down and made it work as cleanly as possible. See the below Nginx config. You may have to make minor adjustments if you're not on FreeBSD.

    server {
            listen 80;
            listen [::]:80;
            server_name xymon.feld.me;
            index   index.html;
            root /usr/local/www/xymon/server/www;
    
            location /xymon/ {
                    alias /usr/local/www/xymon/server/www/;
            }
    
            location /cgi-bin/ {
                    alias  /usr/local/www/xymon/cgi-bin/;
            }
    
            location /cgi-secure/ {
                    alias  /usr/local/www/xymon/cgi-secure/;
            }
    
           location ~ ^/.*\.sh$ {
                    gzip off;
                    fastcgi_param SCRIPT_NAME $fastcgi_script_name;
                    fastcgi_param DOCUMENT_ROOT /usr/local/www/xymon/;
            fastcgi_param REMOTE_USER $remote_user;
                    include fastcgi_params;
                    fastcgi_pass unix:/var/run/fcgiwrap/fcgiwrap.sock;
           }
    }
    

    Upstream Xymon supplies an Apache configuration that aliases /xymon-cgi/ to /cgi-bin/ and /xymon-seccgi/ to /cgi-secure/. Due to the way that Nginx handles aliases and fastcgi SCRIPT_NAME we don't have much choice here. We're just going to use them exactly as they're named on the filesystem and adjust Xymon to play along.

    Change the following in your xymonserver.cfg:

    XYMONSERVERCGIURL="/cgi-bin"
    XYMONSERVERSECURECGIURL="/cgi-secure"
    

    Now you should have a fully functional Xymon interface served by Nginx. I strongly suggest you protect the Xymon interface or at least the /cgi-secure/ with a password, though. I'll leave that up to the reader.

Page 2 / 3