BYOES: Build Your Own Email Server on OpenBSD

So you want to own your own data and control your own destiny. You are tired of giving away all your personal information to "big tech" and being a slave to their ever expansive terms of service.  

One of the big lie's of the Internet is that "email is hard and you shouldn't do it yourself".  

I will say, that if you are not comfortable with administering servers, updating DNS records, troubleshooting configurations, then creating and running an email server should not be your first project.  But email is a very simple and forgiving protocol. Perhaps this simplicity, and the abuse that has risen from it, is why people paint such a dire picture of people running their own email server.

This article will not make you an expert on running an email server.  However, I hope it will provide an introduction in to the different aspects of running an email server and can help get you started running your own server is a safe and responsible manner. So let's get started building your own email server using OpenBSD and OpenSMTPD.

Server

The first thing that will be needed is some place to host the email server.  If you already have a server then skip to the next step. Today there is really no shortage of virtual hosting providers, making this step very easy.  If you do not have a provider you already work with, allow me suggest using Vultr.  What makes them stand out is that they have OpenBSD images ready to go. While other providers may require you to create your own OpenBSD image.  Creation of the image is left as an exercise to the reader.  If you choose to use Vultr, I have an affiliate link you can use... or not, your choice.  Whatever hosting provider you use, you'll want to make sure that they allow SMTP traffic.  Most providers block it by default and require you to specifically ask to allow the traffic.  For Vultr the page explaining how to request that information can be found here.

Domain

The very next thing you'll need is a domain.  Once again, getting a domain is very easy.  If you don't already have a registrar, I'll recommend Gandi.  They are no nonsense.  The don't spam you with "up sell" and they protect your registry details at no extra cost.  In this example I'll use 'byoes.example.org' as the server name and domain.

Certificates

If we are to be exchanging email, we would like that exchange to be secure.  We will do this with certificates.  OpenBSD provides a nice client acme-client(8) which can be used to get a certificate from any ACME certificate provider like Lets Encrypt.  Or you can provide your own certificate.  The use of acme-client(8) is out of scope for this tutorial.  That said, the man pages on OpenBSD walk you thru setting it up and obtaining a certificate.

SMTP Server

Now that we have a server and a domain, we can enable the SMTP mail service. OpenBSD comes with OpenSMTPD email service.  By default it is started in "local" mode.  This will allow local mail delivery from local users and allow you to send external mail but not accept external mail.  As a default configuration this is a good setup, but we want to be able to accept external mail for our domain.  So lets make a few changes.

# example.org config
# certificate
pki mail_cert cert "/etc/ssl/byoes.example.org.fullchain.pem"
pki mail_cert key  "/etc/ssl/private/byoes.example.org.key"

table aliases file:/etc/mail/aliases

listen on socket
listen on all tls pki mail_cert

action "local_mail" mbox alias <aliases>
action "outbound" relay

match from any for domain "example.org" action "local_mail"
match from local for local action "local_mail"
match from local for any action "outbound"

We'll be making some more changes throughout this article, but lets run through the config as it stands...

First two lines specific the certificate and key to use.  

Next line creates a table to lookup of standard aliases.  Additional aliases can be added to this file.

Next we listen on a socket for command from smtpctl(8) and we listen on all interfaces for incoming SMTP connection and we optimistically allow for TLS connections (but we don't require it).

Next 2 lines is how we will process email.  If the email is marked as "local_mail" it will be delivered to mbox according to the aliases table.  If the mail is marked as "outbound" is will be relayed to another mail server.

The last 3 lines match incoming email and determine what needs to be done with it.  Please notes if an email does not match any match rule, it will be rejected by default.  This is a good thing because it will prevent any "open relays" which is the number 1 way to get your IP placed on a black list.

Now that we have a config we need to restart the smtpd(8) server to have it re-read the config.  First we'll check to make sure there are no typos.  Assuming everything is ok, we'll restart the smtpd(8) service using the rcctl(8) command.

# smtpd -nv
configuration OK
# rcctl restart smtpd
smtpd(ok)

The astute among reader will have already been asking 'Where do I specify what users are on my email server?'  By default smtpd(8) will utilize the /etc/passwd file as the user list.  So to add a new user we can use the adduser(8) or useradd(8) commands.  If we do not want these users actually logging in to the server we can set their shell to '/sbin/nologin'.  Setting the users shell to /sbin/nologin will cause an issue in this example because they will have no way to get at their email.  (In a future article I will cover creating virtual users and enabling IMAP for remote email access.  But for now I will keep this article focused on SMTP)

Finally tell the world where to find our mail server.  This is done by publishing MX DNS records.  Below is an example of a record that would be found in a DNS zone file.

example.org. 600 IN MX 10 byoes.example.org.

Since we only have one server we will only have one record.  We can add additional records with additional severs.

Internet... Not a nice place...

Now this configuration is sufficient for sending and receiving email.  However it is not complete. To complete it we need to address some realities of todays Internet. Specifically, it's not a nice place.  There are nefarious actors out there that will try impersonate your email server, sending spam as your domain. They will also try to send spam and exploits to you and your users. Let's see how we address each of those issues.

Impostors everywhere

First thing to do to secure your email server is get and keep a good reputation.  This is done is 3 steps for your server.   The first is to publish Sender Policy Framework (SPF) records.  These are TXT records that tell the world what MX servers your domain will originate traffic from. Check out open-spf.org for format of the SPF record.  The record below instructs visitors that any mail from our domain will originate for our MX servers and no place else.

example.org.  600  IN  TXT  "v=spf1 mx -all"

Second thing  to do is to sign all outbound email.  This done with Domain Keys Identified Mail (DKIM) records.  

To sign outbound email we need to install a filter package for smtpd(8) and then update the smtpd configuration.

pkg_add opensmtpd-filter-dkimsign

filter dkimsign_rsa proc-exec "filter-dkimsign -d example.org -s selector1 \
        -k /etc/mail/dkim/example.org.key" user _dkimsign group _dkimsign

listen on socket filter dkimsign_rsa
listen on all tls pki mail_cert

The first line creates the filter and then we assign the filter to the local socket interface.  This is the interface we will use for sending mail (in this example).

The next step we will create a public and private key signing key. The public key is published in DNS records.  The private key is then used to sign the headers of outgoing email.  I have created a script  to  help with the creation of the domain key files. Create the /etc/mail/dkim directory (if it doesn't already exist), download the script and place it in the folder, then run the script.

# ./make_dkim_key.sh example.org
Generating RSA private key, 1024 bit long modulus
....
...................
e is 010001 (0x65537)
writing RSA key
add the example.org.dns to the zone file

This script creates 3 files.  The example.org.pub is the public key and the example.org.key is the private key.  The final example.org.dns is the DNS record of the public key to publish in DNS.

selector1._domainkey.example.org. 3600 IN TXT "v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDD7xgv90I27tyA0eFXKdz/pUpZ2RQm/NwJzEIiRwnvIjA8tQbSdLcvWtVR07HkNXq5NyHy8UqHrmwsmUJMgf888NLBD6XJLSS0bORNfW3rPmoysoLi8iRZzItb/LUjjI9+A12tY/1W9ZKjfuTX5d5Dml+a8RYnnWuCLCpITlfugQIDAQAB"

Finally the last step is to publish a Domain-based Message Authentication, Reporting and Conformance (DMARC) record. The DMARC record is a way that we can specify to the reset of the world what should happen to email that is not signed and sent properly.  Now this is only a suggestion.  Email servers that receive email purporting to be from use can choose what they want to do if the email is not signed or fails signature.  mxtoolbox.com has a good page showing all the DMARC tags.

_dmarc.example.org. 600 IN TXT "v=DMARC1;p=quarantine;pct=100;rua=mailto:[email protected]."

It is also good practice to create and alias of dmacreports in the /etc/mail/aliases file to receive these reports.  Or you can change this address to whatever address you would like.

Go away spammers

There is a ton of spam out there and running our own email server means we will be receiving a ton of it.  Fortunately there are several effective tools that we can use to "pre-filter" the spam.

Are you a real mail system?

Many spammers utilize compromised systems, poorly maintained remote access locations etc.  But one thing they all have in common is the programs that send out the spam are not real mail systems.  They do not operate like real spam systems.  We can utilize this fact and another program that come installed on OpenBSD by default called spamd(8).  This program with some clever PF rules will intercept SMTP requests from unknown servers.  It will pretend to be an SMTP server up until the unknown server tries to send data, then spamd will issue a temporary failure.  If the SMTP server is a legitimate SMTP server, it will queue the message and try back after a set length of time.  Once the SMTP server tries back, spamd will place the server on an allow list and will not longer intercept communications between the SMTP server and smtpd(8).  This simple program will eliminate spam from the vast majority of "hit and run" spam operations.

Set up the PF rules using the example pf.conf(5).  This example was taken almost verbatim from the spamd(8) man page.

table <spamd-white> persist
table <nospamd> persist file "/etc/mail/nospamd"
pass in on egress proto tcp to any port smtp divert-to 127.0.0.1 port spamd
pass in on egress proto tcp from <nospamd> to any port smtp
pass in log on egress proto tcp from <spamd-white> to any port smtp
pass out log on egress proto tcp to any port smtp

Quick explanation is in order.  Two tables are used.  The first spamd-white is used by spamd(8) to add IP's of compliant email servers.  The second table is used to "pre-authorize" email servers.  The next rules are processed with last match winning.  First any email traffic is diverted to spamd. Unless that address is in the nospamd table, or unless the address is in the spamd-white table.  The last line logs outbound connections.  spamlogd(8) listens to the pflog(4) interface.  Any outbound mail will add the receiving email sever to the spamd-white table as well.  This way we can allow emails server that we send email to.

However there are legitimate email servers and services out there that may still have problems with spamd(8).  For these systems we can place their published servers on an allow list by using the SPF records that those services publish.  We may also want to place large email providers like gmail, yahoo, etc in the allow list as well.

The best way to do this is to create a file in /etc/mail/nospamd_domains.txt and add any/all domains of email provider locations that you want to per-authorize.  The run the command

smtpctl spf walk < /etc/mail/nospamd_domains.txt >/etc/mail/nospamd

to create the nospamd file.  This command can be setup in cron(8) to periodically lookup the SPF records of the domains and update the nospamd file. Next to reload the pf nospamd table run

pfctl -T replace -t nospamd -f /etc/mail/nospamd

The spamd(8) service needs to be started and crontab needs to be edited to enable the spamd-setup(8) command to be run periodically.  To enable spamd-setup in crontab use

crontab -e

and remove the # in front of the line that contains /usr/libexec/spamd-setup.

Now enable and start the spamd(8) service.

rcctl enable spamd spamlogd
rcctl start spamd spamlogd

While it may take several hours to days to have any entries in the spamdb database, to check the list of address in the spamdb run the spamdb(8) command.  See the spamdb man page for the format of the output.

Belts and Braces...

While spamd(8) will get rid of most of the spammers, it will not stop all.  For that we have some additional filters in smtpd(8) that we can use to make sure that the mail sever is properly registered and is trying to do all the right things.  For this we head back to the smtpd.conf(5) filters.

rDNS

Check for reverse DNS

filter check_rdns phase connect match !rdns \
        disconnect "550 no rDNS"

FCrDNS

Check for forward confirmed reverse DNS.

filter check_fcrdns phase connect match !fcrdns \
        disconnect "550 no FCrDNS"

Residential Connections

Check the names of the connecting server to see if they "match" possible residential or dynamic connections.

# check for dynamic dns sources
filter check_dyndns phase connect match rdns regex { '.*\.dyn\..*', '.*\.dsl\..*', '.*dynamic.*', '*\.vodafonedsl\.*', '*\.vodafone\.*' } \
        disconnect "550 no residential connections"

filter check_ip_names phase connect match rdns regex { '[0-9]{1,3}-[0-9]{1,3}-[0-9]{1,3}-[0-9]{1,3}\..*' } \
        disconnect "550 no residential connections"

Now add all the filters to a filter chain and update the listen statement.

filter initial_checks chain { check_rdns, check_ip_names, check_fcrdns, check_dyndns, dkimsign_rsa }

listen on all tls pki mail_cert filter initial_checks

Putting a bow on it...

With all that complete we should be able to send and receive email.

Hello world...

To test email sending we can use the mail(1) command to send an email to ourselves  We can also use that same command to read the email.

byoes# date | mail -s "test" [email protected]
byoes# mail
Mail version 8.1.2 01/15/2001.  Type ? for help.
"/var/mail/root": 1 message 1 new
>N  1 [email protected]  Wed Aug 31 13:04   20/835   test
& 1
Message 1:
From [email protected] Wed Aug 31 13:04:07 2022
Delivered-To: [email protected]
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; s=selector1; bh=ymFhCsqNTm
        Eo0XOLAQu3Dkov+eM7C4ikBg+j71KhuQI=; h=subject:to:date:from;
        d=example.org; b=im+00x2t0cqflEGGt1zdmWPVUObepydJCqZulKK5hFLYTTVetHRLH
        IEtFg6Zk/Vk7ry0WwFJ4bjsjYWcSwiCPM0i0Ls4oTUuBjsHE7sW2o6C4m9tOE/0d5xTIa2
        jSkJzWtB7G5+61ylp5cT0dqX7nILwpFYLN5Yfza3PljSBNC8=
From: Charlie Root <[email protected]>
Date: Wed, 31 Aug 2022 13:04:07 +0000 (UTC)
To: [email protected]
Subject: test

Wed Aug 31 13:04:07 UTC 2022

& q
Saved 1 message in mbox

The first line sends the current date in the body to the [email protected] address.  The second line start the email reading and displaying the first message we can see the DKIM signature that was added when the email was sent.

What is something goes wrong...

To test receiving email, from the other email system send a message to your new email account.  Did you receive it?  You can watch the receipt in the mail log by tailing the maillog

tail -f /var/log/maillog

If something still isn't correct, try adding the IP address of the other mail server into the nospamd file.  This will avoid the mail getting stopped by spamd(8).

Lastly, did you check with your ISP/hosting provider to make sure they are not blocking SMTP?

Conclusion

This was just a  quick example on how to setup your own simple email server.  There are many more things that can be done, from adding redundancy with multiple mail servers, to doing DNSBL and bayesian analysis and filtering on incoming email to detect SPAM.

Please let me know if I missed anything important.

Up next, adding an IMAP servrer.