Mailserver configuration issue

[Vitus]

Good morning,

I have a flightsim unrelated problem and hope some of you might be able to help me out. As you know, I launched my website a few weeks ago. And I noticed that emails sent through my server almost always end up in the spam folder of the recipient. You can try that yourself by signing up to my newsletter on the bottom right corner of the website - the server will sent you a confirmation mail and I bet it too will end up in your spam or even be blocked by your email host.

My setup is the following: To host my page, I rented a vServer and run an installation of plesk 12 on a linux centos os. For the emails, I'm using one of the solutions plesk provides - Dovecot for imap and Postfix as my smtp server.

To help me debugging, I tried the website mxtoolbox.com, since it seems to be a very thorough tool. When I try the SMTP test on the site, I get the OK on all but one items. It tells me that the SMTP Valid Hostname test failed, saying that Reverse DNS is not a valid Hostname. Oddly enough the SMTP Reverse DNS Mismatch test returns the right url. Everything seems to check out...

I already spent a lot of time reading up on the issue, but I can't find a solution. I hope someone here would know something about this issue.

Thanks for reading and have a great day!
Vitus

 

[Vitus]

This problem still bugs me. I spent the whole morning between the linux shell, mxtoolbox, strato and random webpages about email servers.

I did some changes to my DMARC, PTR and changed the SSL certificate. I find it hard to test though. Could some of you volunteer for me and provide me with your email address? I'd love to have a small selection of email hosts with addresses to which I can send test emails. If you're willing to help out, please pm me with an email address I can use. The only thing I require from you afterwards is a short feedback to see if my mail went through OK, ended up in the spam folder or didn't arrive at all.

 

[n4gix]

n4gix -at- comcast -dot- net

 

[edphipps]

doublebaccn at initials of America On Line period com

 

[Vitus]

Thank you! I sent a test email.

 

[Heretic]

Hope your spam filter is any good, Bill. Posting e-mail addresses in the open is a big internet "no no"

 

[n4gix]

Thank you! I sent a test email.

Received email and replied. Your Wing42 email landing in my Personal folder just as it was supposed to.

 

[Vitus]

It drives me nuts. One made it, the other one didn't.

 

[Ronald]

And I noticed that emails sent through my server almost always end up in the spam folder of the recipient.

1. YOU are not responsible for how Internet Service Providers ENROUTE and/or WebMail providers FLAG your [sent] emails.
2. YOU can only learn how to configure your OWN SERVER to the best of your knowledge (and improve that skillset on the way)
3. YOU are not responsible for how others receive your emails,
That is fully up to them and/or their own configuration of their (web) email-clients and/or (local) spam-filters.

You can try that yourself by signing up to my newsletter on the bottom right corner of the website - the server will sent you a confirmation mail and I bet it too will end up in your spam or even be blocked by your email host.

2. "Assumption is the mother of all f*ckups" -> Computers and configuring email servers is an exact science, no guess work needed here.
So get the basics and facts right first and create a TEST-PLAN and carefully execute your plan and evaluate the results.
Only then and only then you can come to solid conclusions, nothing else will help you..

3. Here is what happened when I registered for your email-newsletter:
- I subscribe to your newsletter
>> website replied with: "A verification email has been sent. Please check your inbox."
- I checked my yahoo WEB-email account
>> Confirmation was delivered into my Yahoo SPAM folder of my inbox
- I clicked on the confirmation link
>> website showed succesfull "Thank you for subscribing to our newsletter." message

Conclusions drawn so far:so far
A - Your email server works perfectly (for this part)
B - Confirmation link and process works fine as well
C - Mailserver sends email to my Yahoo (web) email-address

D - Error location YAHOO WEBMAIL
Yahoo (and perhaps many other providers) have her reasons to flag your incoming email as spam. This needs to be your point of focus right now
Ask yourself: HOW does the (YAHOO) ANTI-SPAM software (at the web-mail-provider) makes the decision to flag your email-domain as spam?

 

[Ronald]

It tells me that the SMTP Valid Hostname test failed, saying that Reverse DNS is not a valid Hostname. Oddly enough the SMTP Reverse DNS Mismatch test returns the right url. Everything seems to check out...

4. Somehow .. in the chain of all email servers between you (as the sender) and the other persons mailserver, something goes wrong.

From what I (yet) know about it, is that most ANTI_SPAM Software packages communicates with the other server(s) upstream in the email delivery chain.
If the recipients email server can not (in some way) contact the ORIGINAL SENDER email_server.. it internally generates an (sender) error and thereby flags your mails as "spam" That is how (Anti_spam-stuff) it roughly works:
- so you need to focus on the configuration option in your email server which allow other email servers to verify/check with your email server.
- and also (double) check all DNS names in your email server configuration.

Since email-servers (amongst each-other) and/or email-spam-firewalls use each-other DNS_DOMAIN_NAMES to contact each-other for the verification process.

I rented a vServer and run an installation of plesk 12 on a linux centos os.

5. Some other angles to solve this matter (since its rootcause can also be beyond the scope of your own control)!
Contact the technical support staff of your server rental company and ask them to send proof that they have configured their part of the server correctly.
If they have screwed up the configuration of yoru server, you get screwed too

6. Locate a email/dns-domain-blacklist tool online (like the ne below) and check your domain
- https://www.ultratools.com/tools/spamDBLookup
- http://whatismyipaddress.com/blacklist-check
- https://www.dotcom-tools.com/blacklist-check-email.aspx

This helps you to check and identify if your (or your server-rental-companies) domainname has already been FLAGGED / BLACKLISTED for sending spam before you rented the server.
These (spam sending) blacklists are used by (most) ISP's and Web-email providers as INPUT for their automated spam-filter software, so your emails can get automatically dumped
in (almost everyone's) spam_box without you (or your server-rental-firm) knowing about it (and who reported your domain to the blacklist in the first place :-} )

EDIT_2:
7. PING and TRACE-ROUTE the entire email delivery route from your server provider.
It _is_ also technically possible that everything on your own server is configured ok, but that somewhere along the delivery pipeline on of the follow errors occur:
- Mis-configured and/or not-up-todate DNS server(s) elsewhere (at local/remote ISP's or Germany's country level Internet backbone infra_structure providers).
- Firewalls, gateways are put in place (which _can_ block antispam dns_request between email-servers)
- Other ISP / Email server hosters who have technical issues with their (own servers).

8. Be sure to systematically check your (entire) TCP/IP connection BOTTOM-UP according to the ISO model below

If on of the ISO-layers is blocked / fails.. you got a connectivity challenge to solve Vitus.



I hope my explanation (of what is going on in the background) will help you so far, to get your email server configuration up and running.
If not, just keep posting your progress over here..

EDIT: (15:25 GMT+1)
QUESTION-1: Is (https://cronon.net/) your hosting provider?
QUESTION-2: Are these the products you are using? (so I form a better background picture of your situation):
- https://www.centos.org/ - Hardware base OS
- http://linux-vserver.org/Welcome_to_Linux-VServer.org - Virtual Server software
- https://www.plesk.com/ - Website build and manage console
- https://www.dovecot.org/ - Email IMAP server software
- http://www.postfix.org/ - Email SMTP server software

 

[Vitus]

Hello Ronald,

thank you so much for your detailed analysis! I'd say you're spot on with your conclusions. Ultimately, the problem lies with the recipients server configuration, which is of course out of my reach. However emails don't just get flagged at random, the recipient's algorithms are looking for a set of parameters that need to be matched. I did check the blacklists for email addresses/servers and got a tick from all of them. My server is not blacklisted with any of the services (btw. mxtoolbox has that check included in their analysis as well!).

To answer your questions about the software in use:
The server is hosted at Strato, Berlin. It's a vServer, i.e. I share the server's resources with other people. It's running a centOS linux 7.4.1708 with Plesk 12.5. That is a software bundle that came pre-configured from Strato, which saved me from a few headaches. Plesk is great, because it takes care of all underlying packages. Within plesk you can host and manage multiple users, domains etc. and it configures all the services for you. Everything from Apache, mySQL, Antivirus, FTP, etc.etc.

And one feature that I am using is their email configuration. I choose postfix and dovecot and it worked straight of the box. With the only exception being spam-settings, mainly because it's also a matter of configuring the DNS properly.

I am running SPF, DKIM and set a DMARC entry as well. And I think that SPF and DKIM work just fine. DMARC seem to have a problem and I might be on to something with the DNS settings. Meanwhile I contacted Strato again to see if they can help me sort it out. I wait for their answer and give an update when I have more information.

Thanks again!

 

[Vitus]

Here's a small update:
My hosting provider couldn't help me much, but their support forwarded me to a linux/server/etc forum and I got some clues what to look for. I did solve one issue that was indicated by MXToolbox: the RDNS. Turns out that wing42.com is not a fully qualified domain name (FQDN), whereas www.wing42.com is

DMARC seems to be a problem with my provider that can't be solved right now. But it might not have as much of an impact on the spam filters of gmail and co.

I hope that the problem is solved now, but I need to do some more testing with some willing participants

@Ed: I sent another test mail, since your server was the picky one last time.

 

[edphipps]

Check your inbox. Your test was successful.

 

[Ronald]

I'm glad everything is slowly moving forward Vitus. I've located a few background resources regarding FQDN, IMAP and POP3 in case you might need them
RFC's (very useful internet standards mother document to get "syntaxes" right):
Domain names - https://tools.ietf.org/html/rfc1035
SMTP--------------- https://tools.ietf.org/html/rfc5321#section-2.3.5
POP3 -------------- https://www.ietf.org/rfc/rfc1939.txt
IMAP --------------- https://www.ietf.org/rfc/rfc3501.txt

FQDN
- https://en.wikipedia.org/wiki/Fully_qualified_domain_name
- http://www.omnisecu.com/tcpip/fully...-and-partially-qualified-domain-name-pqdn.php

HOW TO's
- EMAIL1 - http://computer.howstuffworks.com/e-mail-messaging/email.htm
- EMAIL2 - https://blog.woodpecker.co/cold-email/smtp-imap/
- POP3 - https://technet.microsoft.com/en-us/library/cc737236(v=ws.10).aspx

DMARC whitepapers (help you fully understand how stuff works under the hood):
- http://cert.europa.eu/static/WhitePapers/Updated-CERT-EU_Security_Whitepaper_DMARC_17-001_v1_2.pdf (see links on page 9,10,10)
- https://static.ondmarc.com/whitepapers/OnDMARC Whitepaper.pdf

DMARC (other) background resources:
- https://en.wikipedia.org/wiki/DMARC
- https://dmarc.org/
- https://blog.returnpath.com/how-to-explain-dmarc-in-plain-english/
- https://dmarc.globalcyberalliance.org
- http://hub.dyn.com/i/328635-a-guide-to-email-authentication-dmarc/1?

(Email) AUTHENTIFICATION:
- https://blog.returnpath.com/authentication-101-the-fundamentals/
- https://blog.returnpath.com/trust-in-email-begins-with-authentication/

DKIM:
- https://blog.returnpath.com/protecting-your-brand-from-phishing-how-to-create-a-dkim-record/
- https://blog.returnpath.com/how-to-explain-dkim-in-plain-english-2/

SPF:
- https://blog.returnpath.com/how-to-explain-spf-in-plain-english/
- https://blog.returnpath.com/protecting-your-brand-from-phishing-how-to-create-your-spf-record/