Category Archives: System Administration

Secure Email – Exim, Dovecot, Perdition.

Is your email secure?

The default behavior of POP3 and IMAP, is to pass your username and password (and all your e-mail for that matter) over a socket with no encryption. This means that all that information is susceptible to a “man in the middle” type attack, where the plain text packets can be intercepted and read.

Is that really going to happen? Probably not, but you can avoid it, by setting up your e-mail servers to provide SSL encryption for all incoming and outgoing email.

This post will talk only about the mail server software I use: Exim (SMTP), Dovecot (POP3 & IMAP) and Perdition (for POP3/IMAP proxying / load balancing).

SSL vs. TLS vs. STARTTLS

First off, there’s a lot of confusion around the naming conventions.

TLS (Transport Layer Security) is SSL- technically, TLS version 1 is SSL version 3. The use of SSL v1 and v2 should be avoided if not completed disabled, so for all intents and purposes, when we say “SSL”, we mean SSL v3.

SSL uses separate dedicated ports for mail: 993 for secure IMAP, 995 for secure POP3 and 465 for secure SMTP. The encrypted connection is negotiated immediately after the socket is opened, and all communication is encrypted. This is more attune to how HTTPS works. The downside with this implementation, is that the service is provided on different ports, which may mean changes to your firewall rules, ACL’s and mail clients.

STARTTLS is a command implementation that works on the existing IMAP, POP3 and SMTP ports. The email client connects to the normal unencrypted ports, and uses a plain text command (STARTTLS in SMTP and IMAP, and STLS in POP3) to initiate a TLS (SSLv3) connection. From that point on, all communication is encrypted. The plus with this, is that the service works over the existing ports, so no firewall or ACL changes are needed.

A big part of the confusion comes from the mail clients, as they all seem to refer to the implementations with different names:

http://en.wikipedia.org/wiki/Comparison_of_e-mail_clients#SSL_and_TLS_support

In a lot of cases, the options are between “SSL” and “TLS”; SSL being an SSLv3 (TLS) connection on the different ports, and TLS meaning a STARTTLS connection. So for this post, we’ll use the same language.

We’re going to setup both (SSL and STARTTLS), which gives us the most flexibility with our clients, though technically, having dedicated ports for SSL services was deprecated in favor of STARTTLS, as it was thought that using two different ports for plaintext and SSL connections seemed wasteful.

Certificates

For our purposes, we’re just going to use a self-signed certificate for e-mail. This provides the same encryption as a certificate bought from a trusted authority- the only difference is the trust part. Obviously, if you’re going to set this up for customers or for the general public, a trusted authority is better, as you won’t receive any of those annoying trust errors when connecting.

There’s a million tutorials out there that show you how to create a self-signed certificate, so I won’t go on too much about this. There’s even a handy site that will generate it for you online.

http://www.selfsignedcertificate.com/

Create  the key:

openssl genrsa -out mail.example.com.key 2048

Create the certificate:

openssl req -new -x509 -key mail.example.com.key -out mail.example.com.crt -days 3650 -subj /CN=mail.example.com

Exim

The SSL config for Exim is pretty straight forward:

tls_advertise_hosts     = *
tls_on_connect_ports    = 465
tls_certificate         = /path/to/ssl/mail.example.com.crt
tls_privatekey          = /path/to/ssl/mail.example.com.key

This basically says to advertise STARTTLS to everybody (*), to assume connections on port 456 are SSL connections, and use the certificate and key files referenced.

To make SSL work on port 465, you’ll need to tell Exim to also listen on that port. This would likely require adding another entry to your “local_interfaces” option- something like:

local_interfaces = 192.168.0.1.25:192.168.0.1.587:192.168.0.1.465

Restart Exim, and SSL should work. More information can be found here.

Dovecot

Dovecot provides POP3 and IMAP services, and supports both SSL and STARTTLS. The setup, again, is pretty straight forward:

protocols = imap pop3 imaps pop3s
ssl = yes
ssl_cert = </path/to/ssl/mail.example.com.crt
ssl_key = </path/to/ssl/mail.example.com.key

Restart Dovecot and test. More information can be found here.

Perdition

Perdition is a mail retrieval proxy. It handles load balancing and load distribution of POP3/IMAP connections, by proxying based on database or regex lookups. One other nice feature of Perdition, is that it can offload the encryption handling of mail connections. It can handle the SSL/STARTTLS negotiation, and then proxy the connections to local servers unencrytped, which reduces the overhead on the actual mail servers.

In my system, I use Perdition to load balance mail between multiple mail servers, as well as handle all the encryption/decryption.

Copy your imap4 and pop3 config files to new imap4s and pop3s config files, and change each, respectively, to:

protocol IMAP4S

and

protocol POP3S

and then add to each:

ssl_mode ssl_listen
ssl_cert_file /path/to/ssl/mail.example.com.crt
ssl_key_file /path/to/ssl/mail.example.com.key

Now, like I said before, in my case I only listen for encrypted connections with Perdition, then I relay mail internally over a non-encrypted link. Perdition has all sorts of ssl_mod options for handling different setups.

In my existing imap4 and pop3 config files I also added:

ssl_mode tls_listen
ssl_cert_file /path/to/ssl/mail.example.com.crt
ssl_key_file /path/to/ssl/mail.example.com.key

Telling it to listen for STARTTLS requests on the non-secure IMAP and POP3 ports.

Restart your existing IMAP4 and POP3 servers, and then start two new perdition instances, using the new IMAPs and POP3s config files.

Testing

The easiest way to test is to use the OpenSSL command line “s_client”, which lets you connect to encrypted services as a client, and validate that the SSL config is working.

IMAPs (SSL)

openssl s_client -connect mail.example.com:993

IMAP + STARTTLS

openssl s_client -connect mail.example.com:143 -crlf -starttls imap

POP3s (SSL)

openssl s_client -connect mail.example.com:995

POP3 + STARTTLS

openssl s_client -connect mail.example.com:110 -crlf -starttls pop3

SMTPs (SSL)

openssl s_client -connect mail.example.com:465

SMTP + STARTTLS

openssl s_client -connect mail.example.com:25 -crlf -starttls smtp

Adding encryption to your mail system is easy, and pretty much every mail client supports either SSL or STARTTLS, if not both.

No “Fail Whale” for Fonolo After Slashdot

Last Thursday (June 23rd), my company Fonolo was featured on Slashdot, which then it made it’s way to Boing Boing, and a handful of other sites.

Now, this was great news for us, as it generated a TON of traffic into our sites, but I was also understandably concerned about what this extra traffic coming into our system was going to do. I’ve heard (and seen) dozens of horror stories about sites featured on Slashdot, who’s servers simply ended up melting from the extra traffic. Having a full out “fail whale” situation, would simply be an embarrassment to us; and I would consider it a personal failure, given that I have been the chief architect of the Fonolo system.

But despite traffic peaking at ~44MB sustained, our system handled it with tons of room to spare- no interruptions, no failures, no downtime.

and lighttpd handled all the web traffic like a champ.

Simple FreeBSD Tunnels

It’s pretty easy in FreeBSD to create simple tunnels between two FreeBSD machines, between a FreeBSD machine and a Cisco router, or to any other host that supports the GRE interface.

The GRE (Generic Routing Encapsulation) interface (defined in RFC 1701 and 1702) has been included for a while, and provides a “standard” encapsulation protocol for tunneling.

So lets assume first a host-to-host tunnel:

Host A:

External IP: 192.168.10.1
Tunnel IP: 10.255.255.1/30
Internal Block: 10.10.0.1/24

Host B:

External IP: 192.168.20.1
Tunnel IP: 10.255.255.2/30
Internal Block: 10.20.0.1/24

Obviously, in a real-world situation, the “External IP” value in your setup, would likely be a real, valid, routable IP address. Also, I like to use a “stub” host block for the tunnel IP addresses- this is basically a block of four IP’s (/30, 2 usable), that are simply used to route either end of the tunnel.

Yes, you can use an IP in the internal block as the tunnel interface, which means you wouldn’t need the extra routes to route the local blocks, but this can break OSPF, so it’s just good (maybe?) practice to keep the two blocks separate.

First off, you need to make sure the gre interface type is available; your kernel should be compiled with:

device gre

or, if you’ve built all the modules as loadable, you can add:

if_gre_load="YES"

to your /boot/loader.conf; you can then either reboot, or run:

host> /sbin/kldload /boot/kernel/if_gre.ko

Which should load the module.

Next, you need to configure the interfaces on both machines; this assumes you don’t already have a greN interface setup- if you do, then just use the next logical number instead of 0.

On Host A:

ifconfig gre0 create
ifconfig gre0 10.255.255.1 10.255.255.2 netmask 255.255.255.252
ifconfig gre0 tunnel 192.168.10.1 192.168.20.1

On Host B:

ifconfig gre0 create
ifconfig gre0 10.255.255.2 10.255.255.1 netmask 255.255.255.252
ifconfig gre0 tunnel 192.168.20.1 192.168.10.1

You can run ifconfig gre0 on each host, and the interface should show as UP. You can also confirm it’s up by pinging the other end of the tunnel from each host; so from Host A, ping the 10.255.255.2 IP address (which is the remote end of the tunnel):

hosta> ping -c 3 10.255.255.2
64 bytes from 10.255.255.2: icmp_seq=0 ttl=64 time=43.192 ms
64 bytes from 10.255.255.2: icmp_seq=1 ttl=64 time=51.099 ms
64 bytes from 10.255.255.2: icmp_seq=2 ttl=64 time=51.459 ms

--- 10.255.255.2 ping statistics ---
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 43.192/48.583/51.459/3.815 ms

Now you need to add routes for your internal blocks, so each end know to route over the tunnel:

On Host A:

route add 10.20.0.1/24 10.255.255.2

On Host B:

route add 10.10.0.1/24 10.255.255.1

Now don’t forget to add this config to your rc.conf files, so the interfaces get setup when the machines reboot:

On Host A:

cloned_interfaces="gre0"
ifconfig_gre0="inet 10.255.255.1 10.255.255.2 netmask 255.255.255.252 
tunnel 192.168.10.1 192.168.20.1"
static_routes="tunnel"
route_tunnel="10.20.0.1/24 10.255.255.2"

On Host B:

cloned_interfaces="gre0"
ifconfig_gre0="inet 10.255.255.2 10.255.255.1 netmask 255.255.255.252 
tunnel 192.168.20.1 192.168.10.1"
static_routes="tunnel"
route_tunnel="10.10.0.1/24 10.255.255.1"

(the ifconfig line was broken for display purposes; it should be one long line in you rc.conf file)

Now if your Host B in this scenario is not a FreeBSD box, but a Cisco device, the Cisco setup goes like this:

interface Tunnel0
  ip address 10.255.255.2 255.255.252
  tunnel mode gre
  tunnel source 192.168.20.1
  tunnel destination 192.168.10.1
!

ip route 10.10.0.1/24 255.255.255.0 10.255.255.2

Remember, this is just a simple encapsulation protocol, and not encryption- so data is not secure. Later, I’ll do an update which includes IPSEC on top of the GRE interface.

Using DKIM in Exim

Since Exim 4.70, DKIM (DomainKeys Indentified Mail – RFC4871) has been supported by default.

The current implementation supports signing outgoing mail, as well as verifying signatures in incoming messages, using the acl_smtp_dkim ACL. By default, DKIM signatures are verified as new messages come in, though no action is taken unless you’ve implicitly configured rules in the DKIM ACL.

After installing Exim (>= 4.70), you should see debug logs for incoming mail from servers that have DKIM signatures setup- they look like:

DKIM: d=gmail.com s=gamma c=relaxed/relaxed a=rsa-sha256 [verification succeeded]
Verifying Incoming Mail

By default, Exim does not filter any mail based on the validity of the DKIM signature- it’s up to you to add ACL rules to control what happens when you receive messages with “bad” signatures.

First add an ACL section for the DKIM processing; this should be included with your other ACL statements:

acl_smtp_dkim = acl_check_dkim

Next, after the “begin acl”, section, add your DKIM ACL section, and by default, accept all messages in this ACL:

acl_check_dkim:

	accept

Now you need to decide what kind of rules you want to setup- you probably don’t want to put a rule that applies to all domains- though, if the company went to the trouble of adding DKIM signatures to their e-mail, you’d hope they’d get it right, and not publish invalid public keys.

For now, let’s add a simple rule for gmail; google knows what they’re doing, so their systems should be setup correctly:

acl_check_dkim:

	#
	# check the DKIM signature for gmail
	#
	deny 	message 	= Common guys, what's going on?
		sender_domains 	= gmail.com
		dkim_signers 	= gmail.com
		dkim_status 	= none:invalid:fail

	accept

You can add as many rules, for whatever domains you want in this ACL.

Signing Outgoing Mail

Now that you’re checking incoming mail, you probably want to sign mail coming out of your system. This is a relatively easy process, that I’ve broken down into three steps:

Step1– Generate a private and public key to sign your messages; you can do this easily with openssl:

#openssl genrsa -out dkim.private.key 1024

Then extract the public key from the private key:

#openssl rsa -in dkim.private.key -out dkim.public.key -pubout -outform PEM

Step2– Configure the Exim remote-smtp transport to sign outgoing messages, using your new private key. You’ll need to pick a domain and a selector for this process.

When remote SMTP servers validate your DKIM signatures, they simply do a DNS look up, based on the selector and your domain- the domain needs to (obviously) be a valid domain you own, that you can add DNS entries to, and the selector can be any string you want. So, for example, using the domain “example.com”, and the selector “x”, you would add to the remote_smtp transport in Exim:

remote_smtp:
        driver = smtp
        dkim_domain = example.com
        dkim_selector = x
        dkim_private_key = dkim.private.key
        dkim_canon = relaxed

This tells Exim to sign any outbound e-mail, using the domain example.com, the selector “x”, and the private key we just generated. The dkim_canon = relaxed, sets the canonicalization method to use when signing messages. DKIM supports “simple” and “relaxed” algorithms- to understand the difference, see section 3.4 of the DKIM RFC.

Step3– add your DKIM public key to your DNS.

The DKIM public key generated above is advertised to other SMTP servers, using a DNS TXT record. In DNS for the domain example.com, add a new TXT record:

x._domainkey.example.com.   TXT v=DKIM1; t=y; k=rsa; p=<public key>

Where “x” is the selector you used above, and <public key> is the public key data (minus the key header/footer text).

When setup correctly, your DKIM text record should look something like this:

# host -t txt x._domainkey.example.com

x._domainkey.example.com descriptive text "v=DKIM1\; t=y\; k=rsa\; p=MIGfMA0GCS
qGSIb3DQEBAQUAA4GNADCBiQKBgQC5k8yUyuyu9UAVHHU7Al4ppTDtxFWsZ6Pqd9NWZnomtewBdz8I
2LJkqmA/3Cyb5Eiaqk4NulPFfDbfA0Lkw7SNyOS9BRN02KGtKIWjFqDwjB99haaWYw9H4IZcuJp0Y
q0kySCdBp/sPP+iTotdBiE85Jakw3tzgYkdvaS05ZUdBwIDAQAB"

(lines breaks were added for readability- your entry should be one continuous line)

This DNS record is referred to as the “selector” record; you need to also setup a “policy” record. The policy record is your domains policy for domain keys- you should start with something like:

_domainkey.example.com. t=y; o=~;

The t=y specifies that you are in test mode and this should be removed when you are certain that your domain key setup is functioning properly. The “~” in the o=~ specifies that some of the mail from your domain is signed, but not all. You could also specify o=- if all of the mail coming from your domain will be signed.

Once you have all of that in-place,  restart Exim, and send out a message using the remote-smtp transport. You should now see a DKIM-Signature: header listed in the message headers, which lists your domain (as d=), and selector (as s=), as well as a signature for this e-mail, which can be validated against your public DKIM key, that you’ve published in DNS.

For more information, see the Exim DKIM page, or the DKIM RFC.

UPDATE:

Once you’ve set everything up, you can test your DKIM (and SPF and SenderID, etc) install, by using the port25.com validation service.

Just send an e-mail to check-auth@verifier.port25.com, and it will auto-respond with a validation report

UPDATE 2:

I’ve updated this to use a key length of at least 1024 bits, otherwise it’s possible to crack the DKIM key, and fake it to show that your email is valid. This came to light largely because of an article about how Google was using a 512 bit key and a “hacker” factored the key, and spoofed emails to the Google founders:

http://www.wired.com/threatlevel/2012/10/dkim-vulnerability-widespread/

I previously listed a key length of 768, which is significantly harder to break than a 512 bit key, but just to be safe, use 1024 or better yet, 2048.