Read this first
Email is the one service where "just self-host it" comes with real caveats. Sending mail that reliably reaches other inboxes depends on your server's reputation, and large providers distrust new senders by default. Very doable — but go in clear-eyed.
Which OS for a mail server
Mail stacks are built and tested on Linux, and the VPS you should use for mail will run Linux. macOS and Windows can manage the server, but shouldn't host the mail itself — the tabs are honest about that.
This is the path. Use a curated all-in-one stack rather than assembling parts by hand:
- Mailcow — Docker-based, modern admin, actively maintained. A good default.
- Mail-in-a-Box — one script turns a fresh server into a complete mail system. The simplest on-ramp.
- Docker Mailserver — leaner, config-file driven, for more control.
Don't host mail on macOS — reputation, uptime, and residential-IP issues make it a poor fit. Instead, use your Mac as the admin workstation: SSH into the Linux VPS that runs the mail stack, and manage it from there.
# then follow the Linux tab on the server
Same guidance: Windows isn't a mail-server host. Manage the Linux VPS from Windows Terminal or WSL2 over SSH, and run the stack there.
# the mail stack lives on Linux; Windows just drives it
The three records that decide everything
Whether big providers trust your mail comes down to three DNS records proving your server is a legitimate sender. Your stack generates the values; you paste them into your domain's DNS. These are the same no matter what you administer from:
@ TXT "v=spf1 mx ~all"
# DKIM — cryptographically signs every message
mail._domainkey TXT "v=DKIM1; k=rsa; p=MIGf..."
# DMARC — what receivers do on failure
_dmarc TXT "v=DMARC1; p=quarantine; rua=mailto:you@yourdomain.com"
You also need a reverse DNS (PTR) record — set through your VPS provider — matching your server's hostname. Missing rDNS gets mail rejected outright.
Warm the domain slowly
A brand-new domain that suddenly sends a hundred messages looks exactly like a spammer. Start with a trickle to addresses that will mark you "not spam," and build over a couple of weeks.
Keep a safety net
Until you fully trust the setup, route outbound mail through a reputable SMTP relay so your messages inherit an established reputation while yours grows. You still own the inbox and the data; only the last delivery hop is borrowed.
Before you move on
- Mail stack installed on a clean-IP Linux VPS
- SPF, DKIM, DMARC, and rDNS all set and passing
- Domain warmed up gradually
- Old address still live during a generous overlap
- A relay configured as a fallback