Software Used:

OpenBSD 7.7 (as VM)
OpenSMTPD (part of OS)
Dovecot 2.3.21.1 (from ports)
RoundcubeMail 1.6.11 (from ports)
twofactor_webauthn 1.3.0

Overview

Email is a surprisingly vast and charged topic. It is one of the first services that someone/anyone can deploy on the Internet and begin communicating. However, email has gotten this reputation that it is "hard". I disagree. Just about anything can be messed up if you don't understand what you are doing. Email is no different.

I hope to dispel the myth of email being "hard" and provide a guide that will get you started on the right foot. I will touch on some topics only lightly and try to provide some links where you can get more information.

A little background

Before we begin creating services and sending email it is always good to understand the protocols and interactions that are involved.

To begin, email is transmitted over the internet using SMTP. Email access from an client can occur over several protocols. The protocol we will use to access email is called IMAP. When sending email to others, and not be mistaken for SPAM, the email needs to be seen as legitimate. In order to prove the legitimacy of email being sent a couple of protocols a are used: SPF and DKIM. SPF allows a receiving email server to verify that the sending server is allowed to send email for that domain. DKIM provides a cryptographic way that the email being received actually originated under to control of that domain. DKIM and SPF are wrapped up into an additional protocol called DMARC. DMARC is a way for a domain owner to publish how receiving servers should treat email that does not pass the SPF and DKIM tests. There is no way to force the receiving servers to act per the DMARC policy but DMARC is a way to signal to the Internet that servers falsely claiming to operate under a domain should be suspect or better yet ignored.

SPF, DKIM and DMARC are all controlled using DNS records. In addition to the above protocols simply publishing proper A and PTR records for the email servers will also help in proving legitimacy. In fact in the next episode when we deploy an external email forwarder, we will use the fact that improper A and PTR records are indicators of illegitimate mail servers.

With the background out of the way and to keep the things simple, I have broken down this guide into 4 parts. There is 1) creating the VM, 2) the mail storage, 3) the mail exchanger, and 4) mail access sections.

Create the mail VM...

First this we need to do is create the VM. Similar to building the firewall.

vm create -t openbsd77 -s 50G -m 4G -c 2 mail

Next add the DMZ network to the VM.

vm add -t network -s net-dmz mail

The image is was already downloaded so we can skip this step. Now install the OS.

vm install mail install77.iso

and connect to the console

vm console mail

Finish installing the OS and shut it down. When the installation is complete break out of the console using ~. escape. This will take you back to a the hypervisor. From the hypervisor make sure the VM is halted using the 'vm list' command. The state should be "Stopped"

root@hyperv01:~ # vm list
NAME      DATASTORE  LOADER  CPU  MEMORY  VNC  AUTO  STATE
mail      default    uefi    2    4G      -    No    Stopped

Start the VM using the 'vm start mail' command. Next we can install the software.

Mail Storage

There needs to be a place to store the mail that we receive. This is the mail servers job. The mail server will need to do several things.

Authenticate against the LDAP directory.
To be able to "sort" the mail as it comes into the server. This capability is referred to as Sieve.
It will need to store the mail (obviously). But we have options in now we store the mail. We can use single mailboxes (called the mbox) format. Which is about as old a email itself. However there is a better options. If the mail is stored in in Maildir format then each email is kept separate and we have options for folders. This is the options that will be show here.

Install mail server

For the mail server we will use an application called Dovecot.

First install the packages

pkg_add dovecot-- dovecot-ldap-- dovecot-pigeonhole--

Next configure the packages. Dovecot installs a lot of files under the /etc/dovecot directory. We will only need to update some of those files.

First we will enable LDAP access.

# LDAP connectivity setting
uris = ldaps://ldap.example.org:636
uris = ldap://ldap.example.org
dn = cn=ldapadmin,dc=example,dc=org
dnpass = letmein
tls = no
tls_ca_cert_file = /etc/ssl/example.org_root_ca.pem
tls_cert_file = /etc/ssl/mail.fullchain.pem
tls_key_file = /etc/ssl/private/mail.key

debug_level = 0
auth_bind = no

base = dc=example,dc=org
user_attrs = homeDirectory=home,uidNumber=uid,gidNumber=gid
user_filter = (&(objectClass=posixAccount)(|(mail=%u)(uid=%u)))
pass_attrs = uid=user,userPassword=password
pass_filter = (&(objectClass=posixAccount)(mail=%u))
iterate_attrs = mail=user
iterate_filter = (objectClass=posixAccount)
default_pass_scheme = SSHA

/etc/dovecot/dovecot-ldap.conf.ext

#
#
passdb {
        driver = ldap
        args = /etc/dovecot/dovecot-ldap.conf.ext
}

# override all ldap settings
userdb {
        driver = ldap
        args = /etc/dovecot/dovecot-ldap.conf.ext
        # override ldap values
        override_fields = uid=vmail gid=vmail home=/var/mail/vmail/%n
}

/etc/dovecot/conf.d/auth-ldap.conf.ext

#
# for mutual auth
#auth_ssl_require_client_cert = yes
#auth_ssl_username_from_cert = yes

#
auth_mechanisms = plain

# use ldap to auth users
!include auth-ldap.conf.ext

/etc/dovecot/conf.d/10-auth.conf

Enable the server to support TLS connections.

#
ssl = yes
ssl_cert = </etc/ssl/mail.fullchain.pem
ssl_key  = </etc/ssl/private/mail.key
ssl_ca = </etc/ssl/example.org_root_ca.pem
#ssl_key_password =

# only validate certs the we generated
#ssl_client_ca_dir =
ssl_client_ca_file = </etc/ssl/example.org_root_ca.pem
ssl_client_cert = </etc/ssl/mail.fullchain.pem
ssl_client_key = </etc/ssl/private/mail.key

#ssl_cert_username_field = commonName
ssl_client_require_valid_cert = no
ssl_require_crl = no
ssl_verify_client_cert = no

/etc/dovecot/conf.d/10-ssl.conf

Now set the remaining variables into the local.conf file

# local changes to dovecot setting
# This file is processed last and overrides the conf.d/*.conf 
# configs.

# only support imap, lmtp and sieve protocols.  disable pop3
protocols = imap lmtp sieve

# set the mail location
mail_location = maildir:~/Maildir

# set the perms on the auth listener (used by smtpd)
service auth {
  unix_listener auth-userdb {
    mode = 0660
    user = vmail
    group = vmail
  }
}

# change the authworker to run as the same user (not root)
service auth-worker {
  user = $default_internal_user
}

# set the metrics
metric auth_success {
  filter = event=auth_request_finished AND success=yes
}
metric auth_failures {
  filter = event=auth_request_finished AND NOT success=yes
}
metric imap_command {
  filter = event=imap_command_finished
  group_by = cmd_name tagged_reply_state
}
metric smtp_command {
  filter = event=smtp_server_command_finished
  group_by = cmd_name status_code duration:exponential:1:5:10
}
metric mail_delivery {
  filter = event=mail_delivery_finished
  group_by = duration:exponential:1:5:10
}

### conf.d/15-lda.conf
# temp fail if user over quote... do not just bounce email
quota_full_tempfail = yes

# enable SIEVE for pre-sorting email
protocol lda {
  mail_plugins = $mail_plugins sieve
}

### conf.d/15-mailboxes.conf
# add other common mailboxes that are used by different clients and point them to consisent destinations
namespace inbox {
  mailbox Spam {
    special_use = \Junk
  }
  mailbox "Deleted Items" {
    special_use = \Trash
  }
}

### conf.d/20-imap.conf
# add sieve to imap
protocol imap {
  mail_plugins = $mail_plugins imap_sieve
}
### conf.d/20-lmtp.conf
# disable verify quota before replying rcpt to
lmtp_rcpt_check_quota = no

# add sieve to lmtp
protocol lmtp {
  mail_plugins = $mail_plugins sieve
}

# mail process statistics tracking
service stats {
  unix_listener stats-writer {
    user = 
    group = $default_internal_group
    mode = 0660
  }
}

### conf.d/90-plugin.conf
# automate the reporting of SPAM and HAM 
plugin {
  sieve_plugins = sieve_imapsieve sieve_extprograms
  sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.environment

# uncomment if spam/ham processing is configured
#  imapsieve_mailbox1_name = Junk
#  imapsieve_mailbox1_causes = COPY
#  imapsieve_mailbox1_before = file:/etc/dovecot/sieve/report-spam.sieve
#  imapsieve_mailbox2_name = *
#  imapsieve_mailbox2_from = Junk
#  imapsieve_mailbox2_causes = COPY
#  imapsieve_mailbox2_before = file:/etc/dovecot/sieve/report-ham.sieve

  sieve_before = /etc/dovecot/sieve/default.sieve
}

### conf.d/90-sieve-extprograms.conf
plugin {
  sieve_pipe_bin_dir = /etc/dovecot/sieve
}

### conf.d/90-sieve.conf
plugin {
  sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.environment
  sieve_plugins = sieve_imapsieve sieve_extprograms
}

# enable extended logging to troubleshoot
auth_debug = yes
auth_verbose = yes
verbose_ssl = yes

/etc/dovecot/local.conf

We will add the default.sieve file. This file will be run on all incoming emails.

require ["fileinto"];
# rule:[SPAM]
if header :contains "X-Spam" "yes" {
  fileinto "Junk";
}

if header :contains "X-Spam-Flag" "YES" {
  fileinto "Junk";
}

/etc/dovecot/sieve/default.sieve

We can test the configuration for errors using the doveconf(1) command.

doveconf -n

The output will show non-default config options. Use the '-a' to show all the config options. If there are errors in the config they will be indicated. Fix the errors before continuing.

Finally enable and start the mail storage server

rcctl enable dovecot
rcctl start dovecot

To verify that the server is running and LDAP connection is functioning we can lookup the users by using the doveadm(1) command.

doveadm user '*'

This command will return a list of all valid users in LDAP.

Mail Exchanger

The mail exchanger is the server that is responsible for communicating with other mail exchangers either locally or in the Internet. This device uses the SMTP protocol to exchange emails. We will be using OpenSMTPd as our mail exchanger software. OpenSMTPd is built into the OpenBSD operating system. We will install a few extensions to tie the service into the LDAP server for user accounts.

pkg_add opensmtpd-table-ldap opensmtpd-filter-dkimsign

First configure the LDAP access.

# Connection information
url: ldap://ldap.example.org
username: cn=ldapadmin,dc=example,dc=org
password: letmein
basedn: dc=example,dc=org

# filter for aliases
# used in aliases and virtual tables
# attribute is the destination user or another mailaddress
alias_filter: (&(objectclass=posixaccount)(mail=%s))
alias_attributes: uid

# filter for username
# attributes are username and hashed password
# the account need to be allowed to read the hashed password
credentials_filter: (&(objectclass=posixaccount)(mail=%s))
credentials_attributes: uid,description

## domain table
## used for: match for domain <ldap>
#domain_filter (&(objectclass=dnsdomain)(cn=%s))
#domain_attributes: cn

# user info
# extra user info
userinfo_filter: (&(objectclass=posixaccount)(uid=%s))
userinfo_attributes: uidNumber,gidNumber,homeDirectory

## filter for mailaddr info
## list of mailaddresses for match statements
## attribute will be ignored but is needed for ldap
#mailaddr_filter: (&(objectclass=posixaccount)(mail=%s))
#mailaddr_attributes: mail

## filter for mailaddrmap info
## used for: liste on ... sender <senders>
#mailaddrmap_filter: (&(objectclass=posixaccount)(uid=%s))
#mailaddrmap_attributes: mail

## filter for netaddr
## used for: match from src <ldap>
## not real useful, because match is on IP not CIDR
#netaddr_filter: (&(objectclass=host)(ipaddr=%s))
#netaddr_attributes: ipaddr

/etc/mail/ldap.conf

Next configure the SMTPd configuration file. Here we will specify that LDAP should be used for various table lookups as well as a static file for a list of domains the email server supports. We will leave some of the configuration commented out until we have setup the relay server.

#

smtp max-message-size "100M"
ca  local_ca cert "/etc/ssl/example.org_root_ca.pem"
pki mail_cert cert "/etc/ssl/mail.fullchain.pem"
pki mail_cert key  "/etc/ssl/private/mail.key"

table ldap      ldap:/etc/mail/ldap.conf
table domains   file:/etc/mail/domains
#table relayaccts        file:/etc/mail/relayaccts
#table creds             file:/etc/mail/credentials


# create filter for signing emails for supported domains.
filter dkimsign_rsa proc-exec "filter-dkimsign -d example.org -s selector01 -k /etc/mail/dkim/example.org_priv.rsa.key" user _dkimsign group _dkimsign

# create a filter chain to allow more than one filter.
filter dkimsign chain { dkimsign_rsa  }

listen on 0.0.0.0 port submission tls-require pki mail_cert auth <ldap> filter dkimsign
listen on 0.0.0.0 port smtps smtps pki mail_cert auth <ldap> filter dkimsign
listen on 0.0.0.0 tls pki mail_cert
#listen on 0.0.0.0 port 9465 smtps pki mail_cert auth <relayaccts> tag RELAY


# setup the action to handle local email.
# leave external email commented out for now.
action "local" lmtp "/var/dovecot/lmtp" rcpt-to user "vmail" userbase <ldap> virtual <ldap>
#action "relay" relay host smtps://ponyexpress@newserver.example.org:9465 tls auth <creds> helo "mail.example.org"


# Create a match rule for email that has been received.
match for local action "local"
match for domain <domains> action "local"
match from auth for domain <domains> action "local"
#match from auth for any action "relay"

# by default any email that doesnt match a rule is dropped

/etc/mail/smtpd.conf

Create a file for domains that we will support. This will allow us to add additional email domains as things grow/change.

# domains supported by this mail server
example.org

/etc/mail/domains

DKIM keys are like other cryptographic keys. There is a private key and a public key. To create the private key use the following command.

[[ ! -d /etc/mail/dkim ]] && mkdir -p /etc/mail/dkim
openssl genrsa -out /etc/mail/dkim/example.org_priv.rsa.key 2048

Generate RSA DKIM key

Later we will generate the public key record for deploying to DNS.

Confirm that the configuration are correct. Fix any errors.

smtpd -nvf /etc/mail/smtpd.conf

Check validity of smtpd config

If all configuration changes succeed, restart the smtpd service.

rcctl restart smtpd

Restart smtpd service

Mail Access

Now that we can receive email and store email. We need to be able to access the emails. This can be done in two ways. First will be over the web and second will be using a native email client that can talk directly to the email server. Both options will be covered. However in the second option only the basics will be touched on as each email client is a little different.

Web email client

Perhaps the most widely used web based email client is Roundcube. Do the following to install roundcube and its dependencies.

pkg_add roundcube php-pdo_sqlite8.2 php-curl8.2 php-gd8.2 php-ldap8.2

Also download and extract the webauthn two factor plugin.

ftp -o /tmp/twofactor_webauthn.tar.gz "https://github.com/bartnv/twofactor_webauthn/archive/refs/tags/v1.3.0.tar.gz"
tar -C /var/www/roundcubemail/plugins -xzf /tmp/twofactor_webauthn.tar.gz

Initialize the roundcube database.

sqlite3 -init /var/www/roundcubemail/SQL/sqlite-initial.sql /var/www/roundcubemail/db/sqlite.db

Initialize the main configuration.

<?php
$config = [];
// seth the database location
$config['db_dsnw'] = 'sqlite:///db/sqlite.db?mode=0660';
// set the IMAP host and port used for authentication
$config['imap_host'] = 'mail.example.org:143';
// set the SMTP host.  used for sending emaik
$config['smtp_host'] = 'ssl://mail.example.org:465';
$config['smtp_user'] = '%u';
$config['smtp_pass'] = '%p';
$config['smtp_conn_options'] = [
        'ssl' => [
                'verify_peer'   => false,
                'cafile'        => '/etc/ssl/example.org_root_ca.pem',
                'security_level'=> 4,
        ],
];
$config['support_url'] = '';
// set our name
$config['product_name'] = 'example.orgt Webmail';
// change the des_key to something random.
$config['des_key'] = 'CHANGE_ME_PLEASE!!!';
// enable some useful plugins
$config['plugins'] = [
    'archive',
    'zipdownload',
    'managesieve',
    'markasjunk',
    'jqueryui',
    'twofactor_webauthn',
];
$config['skin'] = 'elastic';
// disable the installer
$config['enable_installer'] = false;
// setup logging of some functions
$config['log_logins'] = true;
$config['log_session'] = true;
$config['smtp_debug'] = false;
$config['imap_debug'] = false;

/var/www/roundcubemail/config/config.inc.php

Configure and enable the httpd server

server "*" {
        listen on * port 80
        location "/.well-known/acme-challenge/*" {
                root "/acme"
                request strip 2
        }
        location * {
                block return 302 "https://$HTTP_HOST$REQUEST_URI"
        }
}
server "*" {
        log style combined
        connection max request body 102400000
        listen on * tls port 443
        tls {
                certificate "/etc/ssl/mail.fullchain.pem"
                key "/etc/ssl/private/mail.key"
        }
        # used for autodiscovery
        location "/mail/*" {
                root "/htdocs"
                request strip 1
                directory no index
        }
        root "roundcubemail/public_html"
        directory index index.php
        # block direct access to locations that shouldn't be accessable from HTTPD
        location "/*.git" { block }
        location "/*.tx" { block }
        location "/*SQL*" { block }
        location "/*bin*" { block }
        location "/*config*" { block }
        location "/*logs*" { block }
        location "/*temp*" { block }
        location "/*tests*" { block }
        location "/*install*" { block }
        # all other php request sent to FPM socket
        location "*.php*" {
                fastcgi socket "/run/php-fpm.sock"
        }
}

/etc/httpd.conf

Link the PHP libraries.

for f in `ls /etc/php-8.2.sample`; do ln -s /etc/php-8.2.sample/${f} /etc/php-8.2/${f}; done

Add a custom.ini file for some tweaks

; Custom PHP settings
; override the defaults
file_uploads = On
upload_max_filesize = 100M
max_file_uploads = 20
post_max_size = 100M

/etc/php-8.2/custom.ini

Finally enable web email

rcctl enable php82_fpm httpd
rcctl start php82_fpm httpd

To confirm things are working connect to https://mail.example.org/ and login with the LDAP user mail address and password.

Native email client

Similar to the web email client, dedicate email clients use SMTP and IMAP to send and access email.

Establishing trust in our CA

Recall we are using our own certificate authority to secure our servers. We will need to load the CA certificate into the clients / systems so that our certificate authority is trusted. The trusted certificate can be download from https://certauth.example.org/roots.pem. Once downloaded, follow instructions of the device / client to trust the certificate.

Setting up Autodiscover for easy configuration

There are several things we can do to assist the different mail clients to find our services and auto-configure themselves properly.

Add a DNS alias for autoconfig.example.org

autoconfig IN CNAME mail.example.org.
autodiscover IN CNAME mail.example.org.
_autodiscover._tcp IN SRV 0 0 443 mail.example.org.
_imap.tcp IN SRV 10 1 143 mail.example.org.
_imaps.tcp IN SRV 0 1 993 mail.example.org.
_pop3.tcp IN SRV 0 0 0 .
_pop3s.tcp IN SRV 0 0 0 .
_submission._tcp IN SRV 0 1 587 mail.example.org.
_submissions.tcp IN SRV 0 1 465 mail.example.org.

Create an autodiscovery config file that can be fetched from the email server.

<?xml version="1.0" encoding="UTF-8"?>
<clientConfig version="1.1">
      <emailProvider id="example.org">  
         <domain>example.org</domain>
         <incomingServer type="imap">
            <hostname>mail.example.org</hostname>
            <port>993</port>
            <socketType>SSL</socketType>
            <username>%EMAILADDRESS%</username>
            <authentication>password-cleartext</authentication>
         </incomingServer>
         <outgoingServer type="smtp">
            <hostname>mail.example.org</hostname>
            <port>465</port>
            <socketType>SSL</socketType>
            <username>%EMAILADDRESS%</username>
            <authentication>password-cleartext</authentication>  
         </outgoingServer>  
      </emailProvider>
</clientConfig>

Thunderbird client

When the Thunderbird client first starts it will ask to create an account. Normally this is exactly what we want to happen. However, since Thunderbird does not use the system certificate store, we need to add our CA certificate to the trusted store on Thunderbird.

To do this exit the account creation dialogue.

Open the Setting dialogue

Select Privacy & Security and scroll down to find the Manage Certificates button and click it.

In the Certificate Manager dialog select the Authorities tab then Import button.

Select the certificate PEM file that contains the our CA certificate. Then click Open.

Select the check boxes to trust this certificate for web sites and email users. Then click OK button.

With that complete, click on the gear icon again and select Add Mail Account to start the account setup process again.

Enter your username email address and password into the boxes and click Continue. Thunderbird will try to guess your sites settings.

The settings should come back with something similar. However the username does not have the domain. To correct this click the Configure Manually link.

Retype the the email in the username field. Then click the Done button.

Finally you should receive the Account successfully created window. Click Finish button

You should now be able to send and receive emails... to yourself. In the next episode we will build the proxy server that will allow us to send and receive emails with the outside world.

Conclusion

If you made it to here, congratulations! You now have a working email server free from spying and corporate policies that are aggressively against you.