Failai
Stalwart coolify

Stalwart on coolify

virgis mirtis
FA

Failer

Published

Intro

Hosting stalwart on coolify was not an easy task. There was very little documentation on what to do other than this post: https://aldertvaandering.com/posts/setting-up-stalwart-on-coolify. I struggled a lot to set it up - failed to get certificates, getting timed out by acme servers, things just not working as it is usual for me, just things not working and not knowing where to find the logs. It was frustrating. I tried to setup stalwart on coolify for a week, kind of succeeded, I could receive emails, but could not send them, then I tried to fix sending, and something broke and i could not fix it, so I had to start over. So I am writing this so that I could reference some mistakes and have a nice setup later if things don't change too much.

Traefik config

The first thing I had to do was configure coolify proxy traefik by editing its configuration. It can be edited through coolify dashboard by going into Servers -> Your server and then the Proxy tab. Here I copied most of the config from other post I mentioned, but I configured mine for Cloudflare.

Here is the my configuration for traefik:

yaml
1name: coolify-proxy
2networks:
3  coolify:
4    external: true
5services:
6  traefik:
7    container_name: coolify-proxy
8    image: 'traefik:v3.6'
9    restart: unless-stopped
10    environment:
11      - CF_DNS_API_TOKEN=CF_TOKEN
12    extra_hosts:
13      - 'host.docker.internal:host-gateway'
14    networks:
15      - coolify
16    ports:
17      - '80:80'
18      - '443:443'
19      - '443:443/udp'
20      - '8080:8080'
21    healthcheck:
22      test: 'wget -qO- http://localhost:80/ping || exit 1'
23      interval: 4s
24      timeout: 2s
25      retries: 5
26    volumes:
27      - '/var/run/docker.sock:/var/run/docker.sock:ro'
28      - '/data/coolify/proxy/:/traefik'
29      - '/etc/localtime:/etc/localtime:ro'
30      - '/etc/traefik:/etc/traefik'
31    command:
32      - '--ping=true'
33      - '--ping.entrypoint=http'
34      - '--api.dashboard=true'
35      - '--entrypoints.http.address=:80'
36      - '--entrypoints.https.address=:443'
37      - '--entrypoints.http.http.encodequerysemicolons=true'
38      - '--entryPoints.http.http2.maxConcurrentStreams=50'
39      - '--entrypoints.https.http.encodequerysemicolons=true'
40      - '--entryPoints.https.http2.maxConcurrentStreams=50'
41      - '--entrypoints.https.http3'
42      # Entrypoints for Mail
43      - "--entrypoints.smtp.address=:25"
44      - "--entrypoints.smtps.address=:465"
45      - "--entrypoints.imaps.address=:993"
46      - "--entrypoints.submission.address=:587"
47      # Providers
48      - '--providers.file.directory=/traefik/dynamic/'
49      - '--providers.file.watch=true'
50      # use dnschallenge instead of httpchallenge
51      # - '--certificatesresolvers.letsencrypt.acme.httpchallenge=true'
52      # - '--certificatesresolvers.letsencrypt.acme.httpchallenge.entrypoint=http'
53      - '--certificatesresolvers.letsencrypt.acme.dnschallenge.provider=cloudflare'
54      - '--certificatesresolvers.letsencrypt.acme.dnschallenge.delaybeforecheck=0'
55      - '--certificatesresolvers.letsencrypt.acme.keytype=EC256'
56	  # Probably not necessary
57      - '--certificatesresolvers.letsencrypt.acme.email=YOUR_EMAIL_ADDRESS'
58      # staging
59      #- '--certificatesresolvers.letsencrypt.acme.caServer=https://acme-staging-v02.api.letsencrypt.org/directory'
60      - '--certificatesresolvers.letsencrypt.acme.storage=/traefik/acme.json'
61      - '--api.insecure=false'
62      - '--providers.docker=true'
63      - '--providers.docker.exposedbydefault=false'
64    labels:
65      - traefik.enable=true
66      - traefik.http.routers.traefik.entrypoints=http
67      - traefik.http.routers.traefik.service=api@internal
68      - traefik.http.services.traefik.loadbalancer.server.port=8080
69      - coolify.managed=true
70      - coolify.proxy=true
71

72  traefik-certs-dumper:
73    image: ghcr.io/kereis/traefik-certs-dumper:latest
74    container_name: traefik-certs-dumper
75    restart: unless-stopped
76    depends_on:
77      - traefik
78    volumes:
79      - /etc/localtime:/etc/localtime:ro
80      - /data/coolify/proxy:/traefik:ro
81      - /data/coolify/certs:/output

There are a few things to note:

  1. When I was testing I reached acme api rate limit and started getting errors, which did not make sence to me. So for testing i recommend using staging endpoint which has higher rate limits. Uncomment certificatesresolvers.letsencrypt.acme.caServer line when testing.
  2. The http challenge did not work for me, or I did not set it up correctly, so I tried using DNS challenge. For that I had to go and grab a Cloudflare API token which can be obtained by going into Cloudflare -> User Icon -> Profile -> API tokens -> Create token. I added IP filtering and set 3 permissions:
    Account - Account settings - Read
    Zone - Zone - Read
    Zone - DNS - Edit
    Selected my account for account resources, for zone resources I selected all zones. I tried being more restrictive, but I was getting errors from acme, I don't know if it was because I reached rate limit or not, but these settings worked.
  3. I remember seeing somewhere in configs that you have to provide certificatesresolvers.letsencrypt.acme.email, but when testing everything worked without it, but I still included it.

I decided to use smtp relay for sending mail. For that I am using 587 port for sending emails, because Hetzner is blocking ports 25 and 465 because of spammers. Thats why I included submission port in the config.

Traefik handles all of the TLS certificates, but they are not exposed and I can't see them, for that I used traefik-certs-dumper service. It exports all TLS certificates for all domains on my coolify server into /data/coolify/certs.

A few times I made mistakes configuring traefik and the dashboard and all of my websites became inaccessible. To fix this I had to ssh into my server, then go into /data/coolify/proxy and edit docker-compose.yml file there to fix my mistakes. After fixing I just started the containers with docker compose up.

Stalwart

Stalwart version in the references post was old and did not work for me. I had to google and go into stalwart docs to figure out that they renamed a few things. First they changed image name from stalwartlabs/mail-server:latest to stalwartlabs/stalwart:latest, then the config directory was changed from /opt/stalwart-mail to /opt/stalwart. There probably was more changes but these are the most memorable that I struggled with.

To deploy stalwart go into and create a new resource and choose Docker Compose Empty. Somewhere there should be a place where you can write docker compose config. Here is mine:

yaml
1services:
2  stalwart-mail:
3    image: 'stalwartlabs/stalwart:latest'
4    container_name: stalwart-mail
5    networks:
6      - coolify
7    ports:
8      - '25:25'
9      - '587:587'
10      - '465:465'
11      - '143:143'
12      - '993:993'
13      - '4190:4190'
14      - '110:110'
15      - '995:995'
16    volumes:
17      - '/var/lib/stalwart2:/opt/stalwart'
18      - '/etc/localtime:/etc/localtime:ro'
19      - '/data/coolify/certs:/data/certs:ro'
20    labels:
21      - traefik.enable=true
22      - 'traefik.http.routers.mailserver.rule=Host(`mail.YOUR_DOMAIN.com`) || Host(`autodiscover.YOUR_DOMAIN.com`) || Host(`autoconfig.YOUR_DOMAIN.com`) || Host(`mta-sts.YOUR_DOMAIN.com`) || Host(`mx.YOUR_DOMAIN.com`) || Host(`smtp.YOUR_DOMAIN.com`) || Host(`pop.YOUR_DOMAIN.com`) || Host(`imap.YOUR_DOMAIN.com`)'
23      - traefik.http.routers.mailserver.entrypoints=http
24      - traefik.http.routers.mailserver.service=mailserver
25      - traefik.http.services.mailserver.loadbalancer.server.port=8080
26      - traefik.http.routers.mailserver.tls.certresolver=letsencrypt
27      - traefik.http.routers.mailserver.tls=true
28      - 'traefik.http.routers.mailserver.tls.domains[0].main=mail.YOUR_DOMAIN.com'
29      - 'traefik.http.routers.mailserver.tls.domains[0].sans=autodiscover.YOUR_DOMAIN.com,autoconfig.YOUR_DOMAIN.com,mta-sts.YOUR_DOMAIN.com,mx.YOUR_DOMAIN.com,smtp.YOUR_DOMAIN.com,pop.YOUR_DOMAIN.com,imap.YOUR_DOMAIN.com'
30      - 'traefik.tcp.routers.smtp.rule=HostSNI(`*`)'
31      - traefik.tcp.routers.smtp.entrypoints=smtp
32      - traefik.tcp.routers.smtp.service=smtp
33      - traefik.tcp.services.smtp.loadbalancer.server.port=25
34      - traefik.tcp.services.smtp.loadbalancer.proxyProtocol.version=2
35      - 'traefik.tcp.routers.jmap.rule=HostSNI(`*`)'
36      - traefik.tcp.routers.jmap.tls.passthrough=true
37      - traefik.tcp.routers.jmap.entrypoints=https
38      - traefik.tcp.routers.jmap.service=jmap
39      - traefik.tcp.services.jmap.loadbalancer.server.port=443
40      - traefik.tcp.services.jmap.loadbalancer.proxyProtocol.version=2
41      - 'traefik.tcp.routers.smtps.rule=HostSNI(`*`)'
42      - traefik.tcp.routers.smtps.tls.passthrough=true
43      - traefik.tcp.routers.smtps.entrypoints=smtps
44      - traefik.tcp.routers.smtps.service=smtps
45      - traefik.tcp.services.smtps.loadbalancer.server.port=465
46      - traefik.tcp.services.smtps.loadbalancer.proxyProtocol.version=2
47      - 'traefik.tcp.routers.submission.rule=HostSNI(`*`)'
48      - traefik.tcp.routers.submission.tls.passthrough=true
49      - traefik.tcp.routers.submission.entrypoints=submission
50      - traefik.tcp.routers.submission.service=submission
51      - traefik.tcp.services.submission.loadbalancer.server.port=587
52      - traefik.tcp.services.submission.loadbalancer.proxyProtocol.version=2
53      - 'traefik.tcp.routers.imaps.rule=HostSNI(`*`)'
54      - traefik.tcp.routers.imaps.tls.passthrough=true
55      - traefik.tcp.routers.imaps.entrypoints=imaps
56      - traefik.tcp.routers.imaps.service=imaps
57      - traefik.tcp.services.imaps.loadbalancer.server.port=993
58      - traefik.tcp.services.imaps.loadbalancer.proxyProtocol.version=2
59    tty: true
60    stdin_open: true
61    restart: always
62volumes:
63  data: null
64networks:
65  coolify:
66    external: true
67

I struggled a lot with the ports. If a new port is defined in stalwart docker compose labels section, then that port must be defined in the traefik config.

Now stalwart docker compose can be deployed. After deployment check the logs in the logs tab. There should be admin password, if the deployment was successful, make sure to grab that. Now go into /terminal and open your server terminal for localhost. Then cd into /var/lib/stalwart/etc and open config.toml using nano config.toml. In this file I had to tell where the certificates for my mail domain was and I setup credentials to relay emails to other domains to resend.com. Before making any changes I recommend making a copy of current config with cp config.toml config.toml.bak just in case something happens.

Make sure to add or configure these config lines for certificates, hostname and other ports that are need:

typescript
1certificate.default.cert = "%{file:/data/certs/mail.YOUR_DOMAIN.com/cert.pem}%"
2certificate.default.default = true
3certificate.default.private-key = "%{file:/data/certs/YOUR_DOMAIN.com/key.pem}%"
4server.hostname = "mail.YOUR_DOMAIN.com"
5server.listener.submission.bind = "[::]:587"
6server.listener.submission.protocol = "smtp"
7server.listener.submissions.bind = "[::]:465"
8server.listener.submissions.protocol = "smtp"
9server.listener.submissions.tls.implicit = true

By default the config looks very nice and structured, but after a few restarts and visits to dashboard, it becomes this de-structured mess. I assume because the dashboard edits this config file as well.

Now also if you're using Hetzner or other VPN provider you can also setup relay using resend.com or other service.

yaml
1[server.listener.smtp]
2bind = "[::]:25"
3

4[session.rcpt]
5catch-all = true
6protocol = "smtp"
7

8[certificate.default]
9cert = "%{file:/data/certs/mail.domain.com/cert.pem}%"
10default = true
11private-key = "%{file:/data/certs/mail.domain.com/key.pem}%"
12

13[queue.route."resend"]
14type = "relay"
15address = "smtp.resend.com"
16port = 587
17protocol = "smtp"
18

19[queue.route."resend".tls]
20implicit = false
21

22[queue.route."resend".auth]
23username = "resend"
24secret = "re_API_KEY"
25

26[queue.strategy]
27route = [
28  { if = "is_local_domain('', rcpt_domain)", then = "'local'" },
29  { else = "'resend'" }
30]
31

32[smtp.outbound.tls]
33dane = false
34mta-sts = false
35

36[lookup.default]
37hostname = "mail.domain.com"
38

39[server.listener.submission]
40bind = "[::]:587"
41protocol = "smtp"
42

43[server.listener.submissions]
44bind = "[::]:465"
45protocol = "smtp"
46tls.implicit = true
47

48[server.listener.imap]
49bind = "[::]:143"
50protocol = "imap"
51

52[server.listener.imaptls]
53bind = "[::]:993"
54protocol = "imap"
55tls.implicit = true
56

57[server.listener.pop3]
58bind = "[::]:110"
59protocol = "pop3"
60

61[server.listener.pop3s]
62bind = "[::]:995"
63protocol = "pop3"
64tls.implicit = true
65

66[server.listener.sieve]
67bind = "[::]:4190"
68protocol = "managesieve"
69

70[server.listener.https]
71protocol = "http"
72bind = "[::]:443"
73tls.implicit = true
74

75[server.listener.http]
76protocol = "http"
77bind = "[::]:8080"
78

79[storage]
80data = "rocksdb"
81fts = "rocksdb"
82blob = "rocksdb"
83lookup = "rocksdb"
84directory = "internal"
85

86[store.rocksdb]
87type = "rocksdb"
88path = "/opt/stalwart/data"
89compression = "lz4"
90

91[directory.internal]
92type = "internal"
93store = "rocksdb"
94

95[tracer.log]
96type = "log"
97level = "info"
98path = "/opt/stalwart/logs"
99prefix = "stalwart.log"
100rotate = "daily"
101ansi = false
102enable = true
103

104[authentication.fallback-admin]
105user = ""
106secret = ""

Hetzner

Now if hetzner is being used then it is recommended to setup a firewall to protect some of the ports. This can be done by going into hetzner dashboard -> choose your server -> firewall. Here a new firewall can be created. I have inbound rules set up for ports 22, 25, 80, 110, 143, 443, 456, 587, 993, 995, 4190 all for tcp and I allow any IPv4 or IPv6

Hetzner firewall