# High Performance Samba Server on Freebsd

## Problem

A midsize educational organisation has a requirement to upgrade the operating system on several hundred desktop computer from Windows XP to Windows 7. The timescale to complete the reinstallation is the summer recess. A master disk image has been prepared and tested. The installation process involves PXE booting the the machines into a Windows PE environment, then mounting a cifs file share where a Windows Imaging Format disk image is accessed. The WIM images and associated files are greater then 20 gigabytes in size, and will be accessed simultaneously by all client's being reimaged.

## Solution

Legacy hardware was reinstalled with FreeBSD, using the ZFS filesystem, and Samba to provide a file sharing solution where constraining factors became the edge network and client PC's rather than the disk access on the image file server.

## Overview

When similar reinstallation tasks had been undertaken on earlier occasions it had been found that simultaneous random disk read access had been a limiting factor. A common way to resolve issues of simultaneous access on file servers is to use multiple disks, and "stripe" the file system across them. In this way the disk read access can be increased approximately linearly with the number of disks. Storage Array Networks commonly have large numbers of disks and high capacity network interfaces to make sure that many random disk accesses can be supported. These enterprise products will come with an enterprise price tag though.

Another way to provide fast disk access is to make use of RAM disks. System RAM can be configured to appear as a disk device, providing fast access, but without any persistence after reboot.

The organisation had an old HP Proliant DL380 G5 with 48G RAM, 2x72G SATA disk drives in a hardware mirror configuration ( RAID 1 ) , and 4x 1Gbit network interfaces. This had previously been used as a VMWare host, but had reached end of life. ( It was 6 years old and out of hardware warranty ).

The relatively large RAM to available disk space lead us to choose to install FreeBSD 9, and use the ZFS file system, and export the file system using Samba for Windows clients. This gives us the ability to cache the current working set of files in RAM dynamically, giving high performance. Network bonding allows us to use all the available network interfaces.

### ZFS

ZFS is well supported on FreeBSD, and has some amazing features to provide robust and performant disk access, including some smart caching mechanisms. By default ZFS will use up to 80% of the available system RAM for its ARC cache.

In the use case we are looking at the first client access will cause all the required image files to be cached. This will mean only minimal disk access is required, and disk access should not be a bottleneck.

### Installing FreeBSD 9 with a ZFS root partition

Because of a limitation of the HP Proliant hardware raid, we cannot access the disks directly, but only as a single abstracted RAID 1 Mirror. This means we rely on the HP hardware to inform us of disk errors.

Installing to a ZFS root partition is not directly supported from the FreeBSD 9 installer ( I believe it is in FreeBSD 10 ). To circumvent that we use a modified installation method described here: Installing FreeBSD 9.0 (or later) Root on ZFS using GPT

### Samba

Samba gives us compatibility with Windows. We can join the Active Directory organisation and control access to files and share through Windows groups. Using Samba to join a Windows domain, as a member server, is well documented at samba.org.

socket options=SO_RCVBUF=131072 SO_SNDBUF=131072 TCP_NODELAY
use sendfile=true
aio write size = 16384


and here is the complete smb.conf

[global]
netbios name = wye
wins server = 172.25.1.65
wins server = 172.25.1.67
security = DOMAIN
printcap name = cups
disable spoolss = Yes
show add printer wizard = No
idmap uid = 15000-20000
idmap gid = 15000-20000
winbind use default domain = Yes
printing = cups
server signing = mandatory
socket options=SO_RCVBUF=131072 SO_SNDBUF=131072 TCP_NODELAY
use sendfile=true
aio write size = 16384

[dist2013]
path=/export/dist2013
comment = College Win7 installer
write list = @sgimagebuilders
valid users = @sgimageinstallers @sgimagebuilders


### NIC bonding

FreeBSD lets us bond interfaces using LACP to get loadbalancing and increased throughput. This is configured in the /etc/rc.conf

ifconfig_bce0="up"
ifconfig_bce1="up"
ifconfig_bce2="up"
ifconfig_bce3="up"
cloned_interfaces="lagg0"
ifconfig_lagg0="laggproto lacp laggport bce0 laggport bce1 laggport bce2 laggport bce3 172.25.1.197/16"


This balances the outgoing traffic across the active ports based on hashed protocol header information and accepts incoming traffic from any active port. The hash includes the Ethernet source and destination address and, if available, the VLAN tag, and the IPv4 or IPv6 source and destination address.

The switch ports also need configuring as a LACP group.

## Results

Here are some munin based graphs that show performance over the last year. Peak usage was in July. The munin rrd statistics are averaged down over time, so it's likely that peak will have been higher.

### Network throughput

We can see total network throughput here

And how this is distributed across the bonded interfaces.

### Simultaneous users

The numbers of connected users

### ZFS ARC Cache effiency

Cache hits approach 100%