Greylisting -- requesting enhancements

Hi,

We have a script that can add many of the common mail systems automatically.

The docs for that is at: https://documentation.cpanel.net/display/ALD/Greylisting#Greylisting-AddcommonmailserviceIPaddressestotheTrustedHostslist

As for the SPF, we enabled by default to reduce the delay. But I am interested in getting more details on how it is affecting you.

Thanks,

Travis

Travis,

Admittedly I am spoiled because I have been able to use whitelisting by PTR for years with smf-grey-milter on other boxes. But i'll give you a single example below how lack of whitelisting by PTR is a major inconvenience.

If you've ever heard of SpamExperts spam filtering, their outbound servers are geographically spread out, on different IP networks and such. If I want to whitelist all of those servers, I have to determine all of the IP addresses of all of those servers [currently about 32 of them, which can and do change over time]. But all of their servers reverse-resolve to "delivery.antispamcloud.com".

So, if I had the ability to whitelist by PTR, I would simply do something [in SMF-Grey-Milter] such as:

WhitelistPTR delivery.antispamcloud.com

Then it would not matter if they added/removed servers, and I would never have to update my whitelist.

I assure that over the years I have determined that many mail providers [not just the big ones that your script handles] should be whitelisted by PTR / partial-PTR / wildcard PTR for best effectiveness and have done so.

M

As far as the suggestion I made about not whitelisting mail that passes SPF checks by default, I understand why you have chosen to enable that. After more thought, I'd agree with why you all do that by default. People [admins] should understand though that with that enabled, they will be exempting a lot of spam from greylisting because spammers nowadays are using SPF [and DKIM, and probably DMARC].

M

1. The first time you enable Greylisting, the system adds the most common mail service IP addresses and ranges to the Trusted Hosts list:

   GoogleAppleMicrosoftYahooAOL

   You can use the /scripts/setup_greylisting_db script to automatically add several common IP addresses to the Trusted Hosts list.

Is this a one-time thing? Or does it check daily with these providers to see their current list of mail server IPs?

- Scott

At this time, the strings are hardcoded. We are looking to make this more intelligent in the future by having the system dynamically scrape the SPF includes. We got in a time crunch though and released this version. We wanted to give you at least a starting set.

We didn't do Domain name whitelisting in this version since it is not recommended in the RFC for Greylisting. The fastest way to get the IP addresses for systems is to query the SPF records. It is pretty efficient. An example of how to do this can be found at https://support.google.com/a/answer/60764

I hope this helps,

Travis

Querying SPF records to get an idea of the IP space that needs whitelisted is only useful if the company / accountowner has properly set up SPF records to begin with. That's a leap.

I guess I'll have to take a look at the RFC. I only know that I've been whitelisting by rDNS/PTR for years, and it has worked out great for me. I can only guess that the reason that whitelisting based upon PTR might not be recommended is because PTRs can be forged, or that resolution of a PTR takes more effort than a simple SPF query. But still, whitelisting on a greylisting platform has been done for years and I'd have to say its a successful method, regardless of what the author of the RFC suggests.

I'm using the cPanel provided greylisting on one machine now and am extremely happy with it thus far. I posted a comment in a thread on the forum on Jun 21 2015 about my experience thus far.

Please do consider the addition of rDNS/PTR whitelisting. I'd also suggest that cPanel not even bother to prepopulate the whitelisting because then you are on the hook to do it for the foreseeable future. If you can't thoroughly whitelist mail from the large providers [it is difficult, and you guys missed many IP blocks for many of the big providers], it's better to not do it and force the server admins to do this.

Thanks for the great work and effort put into greylisting!

M

I think what monarobase was getting at is that real mail servers resend messages, which is exactly what greylisting assumes legitimate mail servers would do, which is why subsequent messages from the same server are allowed through.

Which means a spammer with a real mail server would be getting through anyway, since it would be resending. So disabling the SPF whitelisting wouldn't make a difference in that case, since those are supposedly the types of servers that have SPF enabled.

You can easily import the whole sqlite database from your existing server to the new server. Once the database is copied over, cpgrey should work with the copied DB.

Hi Travis. Can you expand on that? What is the name of the sqlite db? Can you give some sample commands and stuff?

Travis: Are you saying that just by downloading and uploading in another server the file file /var/cpanel/greylist/greylist.sqlite should be enough to transfer the whitelisted entries to other servers?

Wow, thanks for that. This is an important piece of information that certainly ought to be prominently placed (or turned into a GUI feature) on the Greylisting page in WHM.

I found this out when I went to whitelist one of Comcast's servers, 68.87.31.167. I noticed it is already on cPanel's list of IPs in their docs (last updated June 26, 2015), but on my server, which just got the 11.50 update on July 4, 2015, it did not have that Comcast IP... so there was an obvious discrepancy.

Just ran the command 5 minutes ago on mine. Thanks! :)

We are looking to improve the use of this functionality in 11.54. We will likely make this into a GUI tool.

Please consider expanding the SPF checkbox to the following list of checkboxes:

----------------

Exempt from greylisting domains matching the following criteria (select all checkboxes that apply):

• Valid PTR
• No PTR
• Invalid PTR

• Passes DMARC
• No DMARC
• Fails DMARC

• Valid SPF
• No SPF
• Invalid SPF

• Valid DKIM
• No DKIM
• Invalid DKIM

• Specific service providers
• - AOL
• - Apple
• - Comcast
• - cPanel
• - Google
• - Hotmail
• - Microsoft
• - Outlook
• - Road Runner
• - Verizon
• - Yahoo

----------------

Travis: I noticed that if once a week I run

1. /scripts/setup_greylist_db --trust aol --trust apple --trust comcast --trust cpanel --trust google --trust hotmail --trust microsoft --trust outlook --trust roadrunner --trust verizon --trust yahoo

new IPs are added to the whitelist every time, ever if I didn't whitelisted new ones and opt not to whitelist those using SPF.

Hence, there is a list of good IP's being updated, right? If it is so, why can't cPanel auto-add those IP's to the Trusted list?

I'm surprised running it weekly would add anything, since their documentation page listing the IPs says it has not been updated for a month: "Created by Doc User, last modified on Jun 26, 2015"

FYI, I opened a ticket with cPanel a couple days ago, and my email confirmation was delayed by over an hour. Looking at the logs, I found two surprising things:

1) The IP that it came from is NOT listed on their documentation as one of the cPanel IPs to whitelist.

2) After the first attempt by cPanel to deliver the mail to our server was deferred (due to greylisting), their server did not try to redeliver for 60 MINUTES. Per Wikipedia, the default retry interval for most major MTAs is 15 mins, so this was pretty surprising.

I opened another ticket with cPanel to let them know about these things, and they said they'd "pass it on".

- Scott

That 'last modified' date probably refers to the date that the documentation page (rather than the list of IP addresses) was last updated.

It would be nice to have a list of checkboxes in WHM, one for each provider (AOL, Apple, etc) as I had suggested 3 posts up.

cPanel could then auto-update the list of whitelisted IPs (not just adding but also removing outdated entries for each provider) on a nightly basis, perhaps as part of the /scripts/upcp cron job.

In addition to the nightly updates, the whitelist would also be updated on demand whenever the checkbox selection is changed by the server admin.

Agree with all your suggestions. FYI, the IP addresses ARE on the documentation page... so I'm just saying that the IPs, on the documentation page, have not been updated in a month. I guess it's possible (perhaps likely) that the IPs are being updated for the script, but nobody is updating the documentation page (in which case the IPs should be removed from the documentation page)

I just ran the script (again) and it added a bunch of new IPs, so thanks for the tip. One of the IP ranges covers the cPanel.net IP address that I commented on earlier. That IP is still not on their Documentation page, so this proves that the Documentation page is not being kept up to date. Sounds like a cron is in order to re-run this script from time to time.

Sorry for the confusion -- I'm completely wrong. The IP or IP range from cPanel.net was NOT added, and while the script claims to have added a bunch of new IPs, further tests reveal that these IPs are the same ones already added when I first ran the script a couple weeks ago. Zero new entries were added. I verified this by checking the number of hosts/ranges were under Trusted Hosts prior to running the script, then I checked again after running the script (refreshing WHM) and there is no change.

P.S. the cPanel.net range where the ticket email came from is: 208.74.120.0/21. Verified via ARIN that cPanel owns this range, so I felt safe to just add the whole range to my Greylist.

Yep, a weekly cron task could do the trick. Only thing I wouldn't love to wait until 11.54.

Another thing this feature will likely need in the NEAR future is the ability to whitelist well known mail providers of the countries you select, or at least,...

1) add a switch to "UPDATE ALL" of the available mail providers.

2) extend the list of known providers with others well known too: GMX (widespread in Europe), Terra (widespread in Latam), Yandex and mail.ru (widespread in Russia), fastmail, icloud (maybe same IP's than "Apple" ip ranges?), Sina.com and 163.com (for those in Chinese market).

I doubt cPanel is going to want the work of keeping this list updated. My guess is that we will either have to pay a fee to subscribe to some service that is responsible for keeping such a list updated, or maybe there will be a way to check SPF records for these major providers and suck in those hosts & ranges automagically.

Alright, I have a ton of updates for this.

We have been developing a new tab for Greylisting. The Common Mail Providers tab will allow you to subscribe to updates for IPs of systems like Outlook and Gmail. We will also allow you to subscribe to new systems that we may add in the future. This does include updates to the full list.

We will be removing the prepopulated IP Addresses that we entered upon activation of Greylisting and putting them in this table instead.

cPanel will be updating this list out of band of regular build updates and as such we have created logic that will check for updates to the common mail provider IP addresses and will update them as needed. The update is only done if you subscribe to updates for that specific mail provider. We will also send the Server Admin a message when we remove entries from the list. That email will include a list of IP address we are removing so that you may manually add them yourselves if you wish.

Additionally we have added support for exporting and importing Trusted Hosts lists.

This will all be included in 11.52.

Will be be able to add our own suppliers to this list ?

Travis, this looks great! In case you missed my previous comment(s), greylisting has been working fantastically. As in a 'spam fighting miracle'. Very few complaints, certainly some time involved tweaking, but the results are just stunning. Major thanks to you and the cPanel crew for this feature!

- Scott

@monarobase

This list is maintained by cPanel. If you want to add your own suppliers, you will need to use the Trusted Hosts tab.

@Scott

Thanks for the awesome feedback. I'm pretty proud of how it is performing.

Well, that's weird. Maybe there is a configuration in your server preventing it from working. In my case, it just did a great job deferring tons of spammers.

The only thing I'm concerned about is that when your Trusted Host list becomes bigger and bigger... will it make greylisting/exim to consume more and more CPU or RAM? I deal with customers with mission critical tasks, who cannot wait 10-15 minutes of delayed emails to arrive, so that I need to whitelist a bunch of IP addresses (and I'm saying 500-700). May this cause the server to underperform?

Before Greylisting, my queues would routinely fill up with 1000+ messages, mostly auto-replies and bounces of undelivered spam that we accepted. After we implemented Greylisting, it's nearly zero. I just spot checked four of my servers, and they all have about 50 or less messages, and not all of those are spam related. This is a dramatic improvement and I'm very thankful for cPanel implementing this feature. We've seen very few complaints, once we added IPs for some of the big mailers that weren't in the provided whitelists. We also tweaked the Greylisting settings, allowing mailers to retry sooner than 10 minutes, and also we are keeping successful triplets whitelisted a lot longer than the defaults, to minimize future mail delays.

@mtindor, how many total rules do you have listed under Trusted Hosts? We are around ~225 so far, between cPanel provided rules plus ones we've added ourselves. Sure sounds like something is broken with your config -- I hope it can be sorted out. Let us know what the result is (if you ever figure it out!)

- Scott

Greylisting was fantastic for us, but after a few days in use we had to disable it. We had lots of complains about long delays or e-mails that never reached customer mailboxes. After investigating logs we found tons of situations from mails providers that don´t use same IP for every deliver attemp and triplet was never found even after dozens of deliver attemps. I tried tweking settings and also spent a long time looking for logs and adding IP blocks to whitelist but day by day our help-desk was just receiving same complaints, until we stop to use it. Our whitelist was over 1000 entries.

Regards,

Jeff

SpamExperts whitelists 255 IP's at a time and not a single IP adress when an e-mail makes it through. They also do the same for any outgoing IP's. I believe it mainly prevents the issue you subscribe. While causing some e-mails to make it through that shouldn't it also means that most of the time you don't notice the greylist at all.

@Jefferson Zardo, this is definitely an issue... where large providers have a bank of mail servers, and every delivery attempt comes from a different IP. Those are the ones that we had to spend time whitelisting and I'm sure we still have some work to do. That may be a feature that cPanel could offer... "consider IP in same /24 as a match during retries when looking for triplets" or something along those lines. I realize that many providers change IPs ranging even more broadly than that, but this would still help.

@Scott, We looked into doing that, and may in the future. I would recommend opening another Feature request for that specific functionality.

As it turns out, this issue that I was ranting above was an an issue I caused all by myself, by adding an improper IP range to the whitelist [I had to have mistyped something].

cpgreylistd is working fab!

1. As it turns out, this issue that I was ranting above was an an issue I caused all by myself, by adding an improper IP range to the whitelist [I had to have mistyped something].

   cpgreylistd is working fab!

mtindor: thanks for getting back to us on that! I'm glad to know greylisting is working for you.

Re: https://features.cpanel.net/topic/greylisting-requesting-enhancements#comment-45860

I don't think a feature request has been opened for this as @Travis suggested, so I've just opened one:

https://features.cpanel.net/topic/greylisting-flexibility-for-dynamic-sender-ip-addresses

Please upvote if you would like a solution to the problem of greylisting senders who use multiple IP addresses.

I second this. The ability to block just the IP or a /24 or /16 range like you can do actually, must remain optional. I track spammers' IP's and I've widely seen that some of them use just one given IP belonging to one compromised or purposedly configured VPS, and sometimes is the hosting company itself providing mailing services to grey companies, and as such, the problematic IP range will be more wide.

In the other hand, I also have to whitelist large companies who send mission critical messages, where a delay in delivery may represent a business problem. Those companies sometimes use a /24 range, or even worst: sometimes they use a single IP from a VPS belonging to a hosting company who allowed sending spam from their other IP's.

That's why this blocking option must always be optional, and cPanel needs to assure their system can decode all kind of CIDR's, like /26 or /18 when inputted manually.

Not at all......

You could whitelist a C class, but only for same sender and same recipient...

This way, for that sender/recipient, you may accept mails whitelisted from the C class, but not other emails...

If the sender/recipient is different, the C class is not considered.

Could you check to see if those additional IPs that you had to manually allow through also happen to share the same PTR (either full or partial wildcard) as the original IPs?

If so, implementing the PTR whitelisting as suggested by mtindor would (as a bonus) negate the need to perform the IP range checks.

This is a massive change in functionality. Can you please open another user story for this requests?

Travis, should there be a feature request for each point (1 and 2) ? or can point 1 continue here ?

If point 1 is true (haven't tested yet) then it would make grey listing almost unusable. The point being that users should not notice it once they have been using it for a short time. Once an IP is known to be a real mail server, no e-mails from this serveur should require further grey listing validation.

Please add each as it's own feature request.