In an effort to prove to myself that I am actually trying to do work this month, I’m making a note of all the bugs in 3rd party software I find.
Today is a bug reported to Memset Ltd (part of iomart Group plc) on the 20th June 2022 under their ticket id 871251564 which caused me to waste a day diagnosing the problem – as I was blaming my server configuration after changing a setting and then verifying it.
The conclusion was that today, 28th June 2022, the server hosting my blog was down for around an hour whilst it was migrated to a new datacenter. I then had to spend the rest of the day confirming settings, resetting the firewall (iomart’s firewall configuration suite is no where as good as Memset’s), checking settings, resetting the IPv6 address on all sites (turns out cPanel hasn’t got a nice way of doing this) and then updating all the SPF mail server records. What fun!
Initial Bug Report
I’m having a problem with SMTP over TLS using port 587 on this server with cPanel Exim. Whilst I’ve configured the CSF firewall software to allow port 587 inbound, verified locally with nmap it is open on both ipv4 and ipv6 (along with checking listening ports) – it seems connections to TCP port 587 are being blocked by Memset – but only on IPv6 addresses (IPv4 connections work and pass).
I’ve even separated out the Memset self-managed firewall to confirm that port 587 should be allowed on both IPv4 and IPv6, but it hasn’t made a different (yes, I waited for the firewall to reload).
Is there some sort of block/firewall prevention stopping IPv6 addresses reaching TCP 587?
I tested with the following settings from my dual-stack home connection:
SMTP 25: telnet ipv4.example.com 25
SMTP TLS 25: openssl s_client -4 --starttls smtp -showcerts -connect ipv4.example.com:25 -servername mail.example.co.uk
POP3 110: telnet ipv4.example.com 110
IMAP 143: telnet ipv4.example.com 143
SMTP SSL 465: openssl s_client -4 -showcerts -connect ipv4.example.com:465 -servername mail.example.co.uk
SMTP TLS 587: telnet ipv4.example.com 587
SMTP TLS 587: openssl s_client -4 --starttls smtp -showcerts -connect ipv4.example.com:587 -servername mail.example.co.uk
IMAP SSL 993: openssl s_client -4 -showcerts -connect ipv4.example.com:993 -servername mail.example.co.uk
POP3 SSL 995: openssl s_client -4 -showcerts -connect ipv4.example.com:995 -servername mail.example.co.uk
SMTP 25: telnet ipv6.example.com 25
SMTP TLS 25: openssl s_client -6 --starttls smtp -showcerts -connect ipv6.example.com:25 -servername mail.example.co.uk
POP3 110: telnet ipv6.example.com 110
IMAP 143: telnet ipv6.example.com 143
SMTP SSL 465: openssl s_client -6 -showcerts -connect ipv6.example.com:465 -servername mail.example.co.uk
SMTP TLS 587: telnet ipv6.example.com 587
SMTP TLS 587: openssl s_client -6 --starttls smtp -showcerts -connect ipv6.example.com:587 -servername mail.example.co.uk
IMAP SSL 993: openssl s_client -6 -showcerts -connect ipv6.example.com:993 -servername mail.example.co.uk
POP3 SSL 995: openssl s_client -6 -showcerts -connect ipv6.example.com:995 -servername mail.example.co.uk
Port 587 “Submission” needs a EHLO command and starttls before it should encrypt (“–starttls smtp” does this).
It should be possible to connect with telnet before encryption.
Port 465 “Submissions” should be encrypted from start – i.e. implicit encryption (RFC8314)
Telnet should return binary stuff.
(Dependent on your openssl version you may need to remove the “-6/-4” parameters which enforce it using the version of TCP/IP: the ipv4/ipv6 hostnames are setup especially for this testing and only have the corresponding AAAA for iPv6 and A for IPv4 entries. Of course, you might prefer to use the IP addresses directly yourself).
THE ONLY commands to fail were:
telnet ipv6.example.com 587
openssl s_client -6 --starttls smtp -showcerts -connect ipv6.example.com:587 -servername mail.example.co.uk
[nb don’t try checking on checktls.com – it turns out despite their UI implying otherwise “Test IPv6 MX hosts. Currently our hosting company Digital Ocean blocks this.” Opening the “Instructions/Info” on https://www.checktls.com/TestReceiver and then “More Options fields” did it reveal this – meaning I had to start all my tests from scratch to confirm the findings]
I had my first response at 09:02 the next day (I submitted the issue at 19:22 the night before and didn’t select “Priority”) just to say they were looking into it.
At 13:20, I go the following reply:
I have consulted our Operations team and it appears, as our IPv6 is in beta, we only have select ports open in our IPv6 firewall and these can not be altered.
What I can offer you though is offer you a migration to our new hosting platform secure.iomart.io Our IPv6 offering is more complete and has no port restrictions.
With a migration your server will me migrated in it’s current state including keeping it’s IPv4 address.
However you would have to change the IPv6 address as we can not migrate this. More info can be found here https://docs.memset.com/cd/Migration-from-Memset-Cloud-VPS-to-OnApp-(iomart.io).316735529.html
So it looks like I’m going to be migrating to a new datacentre shortly. I have been a member of the Memset IPv6 Beta program since December 2012 and kind of hoped they had all the problems worked out by now (11.5 year later).