Virtual server on a subdomain which is hosting a mailserver on a subdomain...from another domain

Hi there!
I think I’m over my head with this project but I feel i’m close enough to make complete

I will deploy the config of my current setup
I have a domain, let’s call it domain1.com, on a shared hosting
I have created a subdomain on domain1.com and added the A record so that the subdomain vps.domain1.com points to the ip address x.x.x.x I was given from the VPS hosting
I have installed the following on my VPS with the LEMP option.
** CentOS Linux 7.7.1908
Webmin version 1.941
Usermin version 1.791
no website or mail server is beeing used on this subdomain
later I aquired a second domain domain2.com on another shared hosting for a small website.
i wanted to pass the mail hosting part of this hosting to my VPS. so I created a virtual server on the VPS with the name domain2.com and added the A and MX records for the mail.domain2.com and webmail.domain2.com to point to the VPS,
this is a redacted version of my dns records on domain2.com

domain2.com. NS ns1.nameserver.com.
ns1.domain2.com. A x.x.x.x (dns from hosting
domain2.com. NS ns2.nameserver.com.
ns2.domain2.com. A x.x.x.x (dns from hosting)
domain2.com. A x.x.x.x (dns from hosting)
webmail.domain2.com. A x.x.x.x (vps ip)
domain2.com. MX (10) mail.domain2.com.
mail.domain2.com. A x.x.x.x (vps ip)
ipv4.domain2.com. A x.x.x.x (dns from hosting)
ftp.domain2.com. CNAME domain2.com.
domain2.com. TXT v=spf1 +a +mx include:spf.sharedhosting2.com -all
_dmarc.domain2.com. TXT v=DMARC1; p=none
www.domain2.com. CNAME domain2.com.
autoconfig.domain2.com. A x.x.x.x (vps ip)
autodiscover.domain2.com. A x.x.x.x (vps ip)

and now comes the tricky part…
Virtualmin configured most of the parts alone for the new Virtual server, I configured it to get a lets’encrypt cert for webmail.domain2.com and created the nginx rule to return the https instead of the http site, then I installed roundcube in public_html, so I can access it from the webmail subdomain.
and below are the DNS records on the vps for the virtual server

 $ttl 38400
@	IN	SOA	vps.domain1.com. root.vps.domain1.com. (
			1580416180
			10800
			3600
			604800
			38400 )
domain2.com.	IN	NS	ns1.nameserver.com.
domain2.com.	IN	NS	ns2.nameserver.com.
domain2.com.	IN	A	x.x.x.x (ip from hosting2)
www.domain2.com.	IN	A	x.x.x.x (ip from hosting2)
ftp.domain2.com.	IN	A	x.x.x.x (ip from hosting2)
m.domain2.com.	IN	A	x.x.x.x (ip from hosting2)
localhost.domain2.com.	IN	A	127.0.0.1
webmail.domain2.com.	IN	A	x.x.x.x (vps ip)
mail.domain2.com.	IN	A	x.x.x.x (vps ip)
domain2.com.	IN	MX	5 mail.domain2.com.
mail.domain2.com.	IN	TXT	"v=spf1 a mx a:mail.domain2.com ip4:x.x.x.x (vps ip) ~all"
_dmarc.domain2.com.	IN	TXT	"v=DMARC1; p=none; pct=100; ruf=mailto:postmaster@domain2.com; rua=mailto:postmaster@domain2.com"
autodiscover.domain2.com.	IN	A	x.x.x.x (vps ip)
autoconfig.domain2.com.	IN	A	x.x.x.x (vps ip)
default._domainkey.domain2.com.	IN	TXT	( "v=DKIM1; k=rsa; t=s; p=long_key_here"
	"very_long_key_here_1"
	"very_long_key_here_2"
	"very_long_key_here_3"
	"very_long_key_here_4"
	"very_long_key_here_5" )
domain2.com.	IN	TXT	"v=spf1 a mx a:domain2.com ip4:x.x.x.x (vps ip) ?all"

what I have accomplished so far?
I can send and receive e-mails from within roundcube…I can have thunderbird use autoconfigure to get most of the settings for the mail service.
The spf1 and dmark record from the domain2.com dns helps so that my e-mails are not considered spam on google.
problems…
when thunderbird is asking for a certificate I’m getting the imap.example.com certificate!
if I use the mxtoolbox service to check the mail server on domain2.com (not mail.domain2.com) it is getting a response from the actual domain2.com mail server and not the one defined on the MX record.
I selected CRAM-MD5 and DIGEST-MD5 on dovecot and upon hitting the apply button it crashes and I have to restart the VPS for dovecot to recover. Althouth I have removed these options, I’m experiencing a strange behavior that alghough Dovecot is running the server dashboard and Dovecot configuration Page say that it is NOT running.

[root@vpsdomain1 user]# systemctl status dovecot.service
● dovecot.service - Dovecot IMAP/POP3 email server
   Loaded: loaded (/usr/lib/systemd/system/dovecot.service; enabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since Mon 2020-02-03 21:42:19 EET; 28min ago
     Docs: man:dovecot(1)
           http://wiki2.dovecot.org/
  Process: 2577 ExecStop=/usr/bin/doveadm stop (code=exited, status=75)
  Process: 4081 ExecStart=/usr/sbin/dovecot (code=exited, status=89)
  Process: 4075 ExecStartPre=/usr/libexec/dovecot/prestartscript (code=exited, status=0/SUCCESS)
 Main PID: 1479 (code=exited, status=0/SUCCESS)

Feb 03 21:42:19 vps.domain1.com systemd[1]: Starting Dovecot IMAP/POP3 email server...
Feb 03 21:42:19 vps.domain1.com dovecot[4081]: master: Fatal: Dovecot is already running with PID 3674 (read from /var/run/dovecot/master.pid)
Feb 03 21:42:19 vps.domain1.com dovecot[4081]: Fatal: Dovecot is already running with PID 3674 (read from /var/run/dovecot/master.pid)
Feb 03 21:42:19 vps.domain1.com systemd[1]: dovecot.service: control process exited, code=exited status=89
Feb 03 21:42:19 vps.domain1.com systemd[1]: Failed to start Dovecot IMAP/POP3 email server.
Feb 03 21:42:19 vps.domain1.com systemd[1]: Unit dovecot.service entered failed state.
Feb 03 21:42:19 vps.domain1.com systemd[1]: dovecot.service failed.

thank you in advance if you can help me with anything!

1 Like

just an update to my situation…
searching through the forums I managed to fix the imap.example.com certificate on thunderbird issue by fixing the entries in the dovecot.conf file
the system had made entries with local_name domain2.com and www.domain2.com but not one for the mail.domain2.com since it didn’t know about it…(my fault because of the name of the virtualserver!)
so now I have
local_name domain2.com {
ssl_cert = </home/domain2/ssl.cert
ssl_key = </home/domain2/ssl.key
ssl_ca = </home/domain2/ssl.ca
}
local_name mail.domain2.com {
ssl_cert = </home/domain2/ssl.cert
ssl_key = </home/domain2/ssl.key
ssl_ca = </home/domain2/ssl.ca
}

Also Dovecot now seems to work OK for no apparent reason!
so two problems down…
I would like some help on my DNS configuration for someone to tell me if i can make it any better
Any help is appreciated, since it’s my first built and still learning…
regards