September 23, 2007

How to administer OpenVPN

This document describes how to administrate OpenVPN on a Debian GNU/Linux server. It does not cover installing a new OpenVPN service from scratch, since that is already covered in the official OpenVPN 2.0 HOWTO. In particular, this document covers:

  1. Logging in via OpenSSH to administrate the system.
  2. Creating X.509 certificates for new OpenVPN users.
  3. Installing the OpenVPN client on a user’s machine.
  4. Re-configuring OpenVPN and restarting the daemon.
  5. Re-installing OpenVPN on a new Debian GNU/Linux server, in case the old server dies or is compromised.

If you haven’t installed OpenVPN on your server yet, please visit the official HOWTO and complete the steps there. Then you can return to this document. I originally wrote this to show co-administrators how to work with an already-running OpenVPN installation.

Logging in with OpenSSH

To administer OpenVPN you must have a user account on the server (obviously). Typically, this involves logging using OpenSSH. If interactive – i.e., password-based – SSH logins are disabled, the server must already have your SSH public key installed for you to get in.

Let’s say you already have a login and know how to access the machine through SSH, and you want to create a new account for a new administrator. These steps can also be forwarded to someone who does have root access, so they can create a new account for you.

This is what to do: first, login to the machine using your SSH client. Once you come to a prompt, run:

$ sudo useradd joeuser

(These examples assume the new user’s name is joeuser). Next, create the home directory for this user and the user directory needed for SSH:

$ sudo mkdir /home/joeuser
$ sudo chown joeuser:joeuser /home/joeuser
$ cd /home/joeuser
$ sudo mkdir .ssh
$ sudo chown joeuser:joeuser .ssh
$ sudo chmod 700 .ssh

The sudo command allows you to more safely run commands as root. The chmod command ensures only joeuser can access his own .ssh directory.

Next you need joeuser’s SSH public key. This will be a text file, generated by PuttyGEN or ssh-keygen, beginning with the text ssh-dss. The default location for this file on Linux is either ~/.ssh/id_dsa.pub or ~/.ssh/id_rsa.pub.

If you are creating a public/private key pair for a Windows user, I recommend the utility PuttyGEN.EXE. Select an RSA or DSA key – using the radio buttons at the bottom of the dialog – and a key length of 2048-bits or higher. Then press the “Generate” button and save the private key to disk as a file named joeuser.ppk.

In a text box at the top of the resulting window will be the public key needed for logging in. Select the entire key (which begins with ssh-dss) and copy/paste it to a file named joeuser.pub. It is not good enough just to press “Save Public Key”, as this does not save the public key in the correct format for the OpenSSH daemon. After copying this data to a file, move the file joeuser.pub to the VPN server and execute the following commands:

$ sudo cp joeuser.pub ~joeuser/.ssh/authorized_keys
$ sudo chown joeuser:joeuser ~joeuser/.ssh/authorized_keys
$ sudo chmod 600 ~joeuser/.ssh/authorized_keys

Now joeuser will be able to login to the OpenVPN machine using his SSH private key. Once he is able to login, he needs to be added as a sudo user, which can be done with this command:

$ sudo visudo

Edit the end of the displayed file so it reads:

joeuser ALL=(ALL) ALL

This should follow the pattern of other lines near the end of this file. By adding joeuser here, he is now be able to use the sudo command to run other commands as root.

NOTE: In order to disable password authentication in OpenSSH – thus requiring that all users login using their public key – edit the file /etc/sshd/sshd_config and make sure it contains these lines:

PasswordAuthentication no
PermitRootLogin no
PermitEmptyPasswords no

This will go great lengths toward securing your OpenSSH service from brute-force password attacks.

Creating new X.509 certificates

How it works

OpenVPN authenticates users using an X.509 certificate exchange. This is the same way SSL works when you connect to websites whose URL begins with “https://”. It’s one of the most widely used and secure ways of establishing an encrypted connection on the Internet.

For X.509 certificate exchange to work, several elements are involved:

  1. The user’s own private and public certificates. His private certificate lives only on his own machine and is used for decrypting data received from the VPN server. His public key certificate lives both on his machine and on the server, and is used by the server to encrypt data intended for his machine. If you’re not familiar with public/private key encryption, the theory is that anything encrypted with a public key can only be decrypted by the corresponding private key. Therefore, anyone who has your public key can freely send you data that only you can read, without involving the exchange of sensitive passwords. For our sample user, these two files are named joeuser.crt and joeuser.key. (There is also a third file which records to the original certificate signature request, named joeuser.csr).

  2. The server’s private and public certificates. After a connection to the OpenVPN server is established, the server will transmit a copy of its public certificate, which is used to encrypt data only the server can read. Once each side has the other’s public key, two-way secure communication becomes possible.

  3. The server’s TLS authentication key. A copy of this key resides on both machines and is used to help avoid “man-in-the-middle” attacks during the initial negotiation process.

  4. The Certificate Signing Authority’s public certificate. This is used to authenticate the “digital signatures” applied to all of the above keys, so both sides can verify the other’s validity. There exists one private/public Certificate Authority pair for the server, named ca.crt and ca.key.

To summarize, the user will end up with five files on his machine:

  • joeuser.crt – The user’s public certificate.

  • joeuser.key – The user’s private certificate.

  • joeuser.csr – The user’s original “certificate request”.

  • ca.crt – The public certificate for the Certificate Signing Authority, which is a single certificate that represents the entire CEG organization. All certificates created by CEG get signed by this certificate, which means that any certificate claiming to come from CEG can be verified as such.

  • ta.key – The TLS authentication key, used to strengthen the security of initial handshaking.

NOTE: The security of the entire system rests in the physical security of each user’s private certificate. This means they should not be kept on removal media that can easily get lost! If anyone should get a hold your private certificate, they could listen in on OpenVPN connections coming in from the Internet and easily compromise the entire system. Keep it secret, keep it safe!

Creating a certificate pair

When joeuser is first created, he has no certificates. Adding a new certificate is easy: just login via SSH and run the following commands:

$ sudo su -
# cd /etc/openvpn
# . ./vars
# ./build-key joeuser

You will now be asked a series of questions relating to joeuser. Answer them to the best of your knowledge, and select Yes when it asks if you want to sign and commit the new key. The user’s keys now exist in the “keys” subdirectory, and should be copied through SSH (or using scp) to your own machine so you can give them to joeuser. Here’s how I do it:

# cp keys/ca.crt ~joeuser
# cp keys/ta.key ~joeuser
# cp keys/joeuser.* ~joeuser
# chown joeuser:joeuser ~joeuser/*
# chmod 400 ~joeuser/*

Now ask joeuser to copy these five files from his home directory using his SSH client. Once the transfer is complete, wipe them from his home directory on the server:

# wipe -f ~joeuser/*

joeuser now has all the certificates he needs to connect to the VPN.

Revoking a certificate

If a certificate pair is ever compromised—or you think it might have been—it should be revoked on the server and a new certificate pair issued. This is also quite easy to do:

$ sudo su -
# cd /etc/openvpn
# . ./vars
# ./revoke-full joeuser

Now if joeuser tries to connect again using his old key the connection will be immediately dropped. Note that for this to work, the CRL list must be checked on the server. This is indicated by the crl-verify option in the server’s configuration file. If this has not been setup yet, see the official HOWTO for instructions.

Installing the OpenVPN client

Once joeuser has his certificates, he needs the OpenVPN client to connect. This can be installed on a Windows or Mac machine using installer packages that are kept on the server. Clients for either OS can be downloaded for Windows or Mac OS/X.

After installing OpenVPN on Windows, for example, it will install a set of programs under the user’s Start menu. One of these is named “Open the OpenVPN configuration directory”. Select this to open an explorer window pointed at the user’s config directory. There should be a single file already in this directory named README.txt.

Into this directory copy all of the user’s certificate files, as well as the standard client configuration file for you server. This client configuration file must be edited once you copy it to reflect the user’s key-file name. I usually create a client configuration template and keep it here:

/etc/openvpn/client.conf

Once these six files are installed in user’s configuration directory, right-click on the OpenVPN icon in the server tray at the lower right of the screen. There will be a “Connect” option now, which will connect you to the VPN. Once the connection succeeds, the user’s assigned IP address is displayed. Their machine can now be seen by other machines and clients at that IP address.

Congratulations! You are now connected to the VPN.

Configuring the server

At some point in time, you may need to reconfigure the OpenVPN server. For example, let’s assume the DNS server has been changed to something else. Here’s how you’d make that change:

Login to the VPN server using SSH, become root with sudo su -, and then edit this file (assuming you named it server.conf):

/etc/openvpn/server.conf

Here is what this file looked like at the time of writing, with comments interspersed. When I get to the section describing the DNS server, I’ll show how to change it.

The configuration file

This section describes the current server configuration, found in server.conf. This is the only file that needs changing to alter the behavior of OpenVPN.

local 63.251.4.10 port 1194
proto udp

These lines define the external (Internet-accessible) address of the VPN server, and the port number it can be accessed by via UDP. If you change the port number you must also change your firewall script to allow connections to the new port.

cd /etc/openvpn

This directive changes the working directory of the OpenVPN daemon so that we can specify relative pathnames in the rest of the file.

ca keys/ca.crt 
cert keys/cronus.crt 
key keys/cronus.key   
dh keys/dh2048.pem 
tls-auth keys/ta.key 0
cipher AES-256-CBC

These are the “security” parameters for the VPN server, specifying the location of the Certificate Authority’s public certificate, the VPN server’s private and public certificate, the Diffie-Helman key used for encryption, the TLS authentication key, and the cipher algorithm used to encrypted the flow of data. AES-256-CBC means that we are using 256-bit AES (American Encryption Standard) in Cipher-Block-Chaining mode, where each block of data sent alters the encryption used on the following block.

dev tun0 
server 10.8.0.0 255.255.255.0

push "redirect-gateway"
push "dhcp-option DNS 1.2.3.4"
push "dhcp-option DNS 1.2.3.5"

These options specify that we are using IP Tunneling mode, where all IP traffic flows across the VPN. The other possible mode is Bridging, where all Ethernet traffic flows across the VPN. The advantage to Ethernet Bridging is that although it is noisier (it lets NetBIOS traffic through, for example), it allows clients to map network drives in Windows.

keepalive 10 120

This statement causes the server to “ping” all clients every ten seconds; if a client does not respond to a ping within 120 seconds, it is considered down and the link is terminated. A similar statement is used by clients to make sure that their connection to the remote side has not been terminated.

;comp-lzo 
;fragment 1400

These two statements are disabled for now. If enabled, they would cause all traffic over the VPN link to be compressed, and all packets over 1400 bytes in size to be broken up into smaller packets. These options may become useful in the future, which is why I haven’t deleted them.

user nobody 
group nogroup

For security purposes, the OpenVPN daemon sets its effective user id after initialization to the user nobody and its group to nogroup. This ensures that the daemon has effectively zero privileges on the system while it’s running.

persist-key 
persist-tun 
ifconfig-pool-persist ipp.txt

These options cause all information relating to client connections to be persisted to data files, so that if the server gets restarted (within 120 seconds) existing connections will not need to be terminated. That is, if OpenVPN is restarted, no one currently connected will notice, except for a temporary pause in service.

status /var/log/openvpn-status.log 
verb 3

These statements influence the amount of logging performed by OpenVPN. The status directive causes a list of all current connection to be written to /var/log/openvpn-status.log. This file gets updated by the server every minute.

chroot /var/run/openvpn

When the server is running—even though its effective user and group ids have no privileges whatsoever—we don’t want it to have any more access to the system than is necessary. To this end, the above directives stick the OpenVPN server process into a “chroot jail”, in the directory /var/run/openvpn, which means that the running daemon cannot access any files outside of this directory. So even if an attacker somehow compromises the daemon and forces it to run a command on the system, there is no system file the command will be able to read or change.

mlock

This security-related option forces all memory related to OpenVPN to remain in system memory. It is never paged or written into swap files, where it might be possible for an attacker on the system to sniff out temporary keys or passwords.

Restarting OpenVPN

Once the configuration file has been changed, OpenVPN needs to be restarted. This can be done with the following command:

$ sudo /etc/init.d/openvpn restart

At the end of this command it should say:

OpenVPN: client(FAILED) server(OK).

It’s OK for the client to fail because we are not running in client mode on the server. You can see the most recent informational messages from the server using this command:

$ sudo tail -30 /var/log/daemon.log

You can also use “tail -f” instead of “tail -30” if you want to “watch” new messages output by the server while people are trying to connect. Each new connection generates several messages during the process of certificate negotiation.

Recreating the server

NOTE: These instructions are for Debian GNU/Linux 4.0, but should be fairly easy to translate to the other Linux variants.

It may happen at some point in time that your current OpenVPN installation crashes or becomes unstable. In that case, it may be necessary to recreate a new one from scratch. The following steps will guide you through the process of creating a new OpenVPN server, whether or not you still have access to the security files on the old server.

First, a new machine is needed with two network cards: one with access to the internal network, and one with access to the Internet.

Step 1: Install Debian GNU/Linux 4.0. This should be done from the netinst CD-ROM image.

Step 2: Once Debian is installed, login as root and type:

# apt-get update
# apt-get dist-upgrade

This will upgrade your Debian installation to the latest stable version.

Step 3: Install the necessary security packages:

# apt-get install openvpn openssl bridge-utils
# apt-get install ssh iptables iproute rsync

Step 4: Uninstall certain default packages which are not needed and pose potential security risks, such as the Apache HTTP server:

# apt-get remove apache2

Step 5: Copy over the configuration files and scripts from the old server:

# rsync -e ssh -av :/etc/openvpn/ /etc/openvpn/
# scp :/etc/init.d/openvpn /etc/init.d
# scp :/etc/ssh/sshd_config /etc/init.d
# mkdir /var/run/openvpn

If you do not have an old server, or if you believe the old server has been compromised, you should create a new OpenVPN environment. This will mean resetting the authentication certificates for all clients and is not a trivial operation for a sizable organization. To do this, follow the steps in the official OpenVPN 2.0 HOWTO.

Step 6: Configure the network interfaces for the new machine. Edit the file /etc/network/interfaces and assign the IP address, gateway and netmask details to your two network interfaces.

Step 7: Reboot the server to incorporate all the above changes. If everything was copied correctly, it should mention in the boot log that the OpenVPN server was started OK.

Security details

The OpenVPN server uses SSH with public key authentication only and OpenVPN via X.509 certificate exchange. It runs OpenVPN in a “chroot jail” (in /var/run/openvpn), meaning that after initialization the server daemon cannot see anything on the system except what is in the /var/run/openvpn directory. Lastly, the OpenVPN server is configured with the mlock option, which prevents it from ever data to the swap volume so that temporary keys cannot be sniffed out, even if an attacker comprises the system and is able to login.

For maximum security, the file /etc/openvpn/keys/ca.key should not be kept on the server, but moved to a physically secure device that only the system administrator has access to. This is the Certificate Authority private certificate, and is required for creating new certificates. Without it, only pre-existing certificates would ever be allowed by the system. If it is copied to a new location, say /mnt/private/ca.key, the configuration file /etc/openvpn/server.conf will have to be changed to refer to this new location. Another possibility would be to copy the contents of /etc/openvpn from the OpenVPN server to another machine which is not accessible to the network, and then to use this machine only for creating new keys.

The OpenVPN server also runs a firewall using the iptables facility in the Linux kernel. You can see the current state of the firewall by using this command:

$ sudo iptables -nL

The firewall should leave at least two ports open to the Internet: TCP port 22 for SSH; and UDP port 1194 for OpenVPN. It must also accept incoming traffic from authenticated OpenVPN clients to the internal network, and from the internal network to all connected clients. Here is a set of commands for iptables that reflect this policy:

INET=eth0
iptables -A INPUT -p udp --dport 1194 -j ACCEPT

iptables -A INPUT -i tun+ -j ACCEPT
iptables -A FORWARD -i tun+ -j ACCEPT
iptables -A FORWARD -o tun+ -j ACCEPT

iptables -A INPUT -i tap+ -j ACCEPT
iptables -A FORWARD -i tap+ -j ACCEPT
iptables -A FORWARD -o tap+ -j ACCEPT

iptables -t nat -A POSTROUTING -o $INET -j MASQUERADE

It is also advisable for clients of the VPN to always have a firewall running and active. The built-in firewall that comes with Windows XP is just fine for this purpose.