Exim Batch sending
As a server administrator, I would like to see the possibility to configure some kind of batch sending for outgoing emails.
Example:
User is sending 1 email to 200 recipients, currently what cpanel does, is random batching, this means it might split up the recipients into 3-5 batches, some batching containing 40 emails others containing 100, and some 30 e.g.
I would like to see following feature:
1: If user send email to x recipients, start a rate limit.
2: Exim will say, okay, I need to send to 200 recipients, lets batch it, to 20 recipients per batch.
3: Every minute exim will send those 20 emails.
So instead of sending 100 emails at same second (or few seconds), we rate limit to 20 emails per minute (or whatever we define).
This would help improving delivery, since it will look less spammy for email hosts, that they receive 100 emails within 3-4 seconds.
Another way of doing it, could be, you queue up all emails send by the user, and rate limit them to sending 1 email every 2 or 3 seconds. (This is what many email providers do).
Please make this feature, so we don't need to use external mail providers for getting this functionality.
I like this idea 
cPanel, WebHost Manager and WHM are registered trademarks of cPanel, L.L.C. for providing its computer software that facilitates the management and configuration of Internet web servers.
There are no current plans at this time for cPanel & WHM to take on the specific roles and responsibilities that have traditionally been held by the batch sending software. It is the usual expectation of the software sending the emails (whatever mailer software you're utilizing) to provide support for queue/rate limiting.
I am confused by a few of the itemized justifications you've listed, as well.
1) Exim does not randomly "batch". It has a concept of 'batch_max' (default 100, configurable through WHM -> Exim Configuration Editor -> Advanced Editor). Note that this has nothing to do with multiple emails being sent, only multiple recipients. Simply put, if you have a *single* email with > 100 recipients (or whatever your batch_max is), it will split the send up into multiple SMTP transactions.
2) Rate limiting is something that you can handle through Exim ACL's. This again would be through the Exim Configuration Editor under the Advanced Editor via use of custom ACLs (specifically, review the "ratelimit" functionality within Exim's documentation).
#1 is easily sorted if you would prefer different handling.
#2 is a bit more complex given it's a proper ACL, but like #1 is completely supported within the product via the editor. The argument can be made for this to be provided in a more friendly UI way with preset ACLs written by cPanel & WHM. However, I would have serious concerns about the added complexity and confusion this may add by having this layered on top of the existing "max sent per hour" limits as well as that this would further grow mail queues on systems. I would argue that this limiting should occur at the mail sending app level, not the MTA (Exim) level given that the per-hour limit already exists at the MTA level.
I feel that a lot more discussion is needed on this before it can be considered.
There are no current plans at this time for cPanel & WHM to take on the specific roles and responsibilities that have traditionally been held by the batch sending software. It is the usual expectation of the software sending the emails (whatever mailer software you're utilizing) to provide support for queue/rate limiting.
I am confused by a few of the itemized justifications you've listed, as well.
1) Exim does not randomly "batch". It has a concept of 'batch_max' (default 100, configurable through WHM -> Exim Configuration Editor -> Advanced Editor). Note that this has nothing to do with multiple emails being sent, only multiple recipients. Simply put, if you have a *single* email with > 100 recipients (or whatever your batch_max is), it will split the send up into multiple SMTP transactions.
2) Rate limiting is something that you can handle through Exim ACL's. This again would be through the Exim Configuration Editor under the Advanced Editor via use of custom ACLs (specifically, review the "ratelimit" functionality within Exim's documentation).
#1 is easily sorted if you would prefer different handling.
#2 is a bit more complex given it's a proper ACL, but like #1 is completely supported within the product via the editor. The argument can be made for this to be provided in a more friendly UI way with preset ACLs written by cPanel & WHM. However, I would have serious concerns about the added complexity and confusion this may add by having this layered on top of the existing "max sent per hour" limits as well as that this would further grow mail queues on systems. I would argue that this limiting should occur at the mail sending app level, not the MTA (Exim) level given that the per-hour limit already exists at the MTA level.
I feel that a lot more discussion is needed on this before it can be considered.
There are no current plans at this time for cPanel & WHM to take on the specific roles and responsibilities that have traditionally been held by the batch sending software. It is the usual expectation of the software sending the emails (whatever mailer software you're utilizing) to provide support for queue/rate limiting.
I am confused by a few of the itemized justifications you've listed, as well.
1) Exim does not randomly "batch". It has a concept of 'batch_max' (default 100, configurable through WHM -> Exim Configuration Editor -> Advanced Editor). Note that this has nothing to do with multiple emails being sent, only multiple recipients. Simply put, if you have a *single* email with > 100 recipients (or whatever your batch_max is), it will split the send up into multiple SMTP transactions.
2) Rate limiting is something that you can handle through Exim ACL's. This again would be through the Exim Configuration Editor under the Advanced Editor via use of custom ACLs (specifically, review the "ratelimit" functionality within Exim's documentation).
#1 is easily sorted if you would prefer different handling.
#2 is a bit more complex given it's a proper ACL, but like #1 is completely supported within the product via the editor. The argument can be made for this to be provided in a more friendly UI way with preset ACLs written by cPanel & WHM. However, I would have serious concerns about the added complexity and confusion this may add by having this layered on top of the existing "max sent per hour" limits as well as that this would further grow mail queues on systems. I would argue that this limiting should occur at the mail sending app level, not the MTA (Exim) level given that the per-hour limit already exists at the MTA level.
I feel that a lot more discussion is needed on this before it can be considered.
There are no current plans at this time for cPanel & WHM to take on the specific roles and responsibilities that have traditionally been held by the batch sending software. It is the usual expectation of the software sending the emails (whatever mailer software you're utilizing) to provide support for queue/rate limiting.
I am confused by a few of the itemized justifications you've listed, as well.
1) Exim does not randomly "batch". It has a concept of 'batch_max' (default 100, configurable through WHM -> Exim Configuration Editor -> Advanced Editor). Note that this has nothing to do with multiple emails being sent, only multiple recipients. Simply put, if you have a *single* email with > 100 recipients (or whatever your batch_max is), it will split the send up into multiple SMTP transactions.
2) Rate limiting is something that you can handle through Exim ACL's. This again would be through the Exim Configuration Editor under the Advanced Editor via use of custom ACLs (specifically, review the "ratelimit" functionality within Exim's documentation).
#1 is easily sorted if you would prefer different handling.
#2 is a bit more complex given it's a proper ACL, but like #1 is completely supported within the product via the editor. The argument can be made for this to be provided in a more friendly UI way with preset ACLs written by cPanel & WHM. However, I would have serious concerns about the added complexity and confusion this may add by having this layered on top of the existing "max sent per hour" limits as well as that this would further grow mail queues on systems. I would argue that this limiting should occur at the mail sending app level, not the MTA (Exim) level given that the per-hour limit already exists at the MTA level.
I feel that a lot more discussion is needed on this before it can be considered.
Hi @Brian,
First of all, thanks for the response!
Because I mentioned 'Randomly' batch, is because, often when I see emails going out, it's in batches of sometimes 30 recipients and next batch is maybe 40 recipients, eventho it's the same email that gets sent.
I also completely agree that the rate limit should be implemented in the 'mail sending app', and I guess many people will see this as very optimistic, because it rarely happens.
It might also be, that the 'mail sending app' is a email client like Outlook, Apple Mail, Thunderbird, whatever exists, some sends emails to maybe 100-200 recipients this way, and sadly the mail applications doesn't do rate limiting.
Therefore I see it as a 'nice to have', (Same reason why I created a 'feature request' in first place), is because sadly we don't live in an ideal world, where every application follow best practice, and not every client hosted, follow best practice as well.
So from a business perspective, I find it nice, that hey, you help your clients making sure you don't spam email providers too fast, by doing some kind of rate limiting, and at same time, it might save you for spending time getting your IP's whitelisted due to someone sent a large amount of emails within a very short period of time.
If emails is handled right, there's no problem sending a lot of emails within 1 hour if you distribute the sending a bit, and follow whatever receiving rate limits the different providers have.
The future indeed exist in Exim, sadly I haven't been able to figure out how to get it working.
A friendly UI with the preset ACLs would be cool, I would assume it surely not should be enabled by default, also because of the per-hour-limit existing, but I think, people should have a simpler way to configure this, it would be beneficial for a lot of people I guess.
I can always advise clients to use third party services like mandrill or sendgrid etc, but if it could be prevented it would be nice.
I also know that queues might grow because of this feature, for same reason it should not be enabled by default.
Again, it's a nice to have - and I would love to see it in cPanel.
If people don't think it's a good idea, then you guys can skip it for sure :)
Hi @Brian,
First of all, thanks for the response!
Because I mentioned 'Randomly' batch, is because, often when I see emails going out, it's in batches of sometimes 30 recipients and next batch is maybe 40 recipients, eventho it's the same email that gets sent.
I also completely agree that the rate limit should be implemented in the 'mail sending app', and I guess many people will see this as very optimistic, because it rarely happens.
It might also be, that the 'mail sending app' is a email client like Outlook, Apple Mail, Thunderbird, whatever exists, some sends emails to maybe 100-200 recipients this way, and sadly the mail applications doesn't do rate limiting.
Therefore I see it as a 'nice to have', (Same reason why I created a 'feature request' in first place), is because sadly we don't live in an ideal world, where every application follow best practice, and not every client hosted, follow best practice as well.
So from a business perspective, I find it nice, that hey, you help your clients making sure you don't spam email providers too fast, by doing some kind of rate limiting, and at same time, it might save you for spending time getting your IP's whitelisted due to someone sent a large amount of emails within a very short period of time.
If emails is handled right, there's no problem sending a lot of emails within 1 hour if you distribute the sending a bit, and follow whatever receiving rate limits the different providers have.
The future indeed exist in Exim, sadly I haven't been able to figure out how to get it working.
A friendly UI with the preset ACLs would be cool, I would assume it surely not should be enabled by default, also because of the per-hour-limit existing, but I think, people should have a simpler way to configure this, it would be beneficial for a lot of people I guess.
I can always advise clients to use third party services like mandrill or sendgrid etc, but if it could be prevented it would be nice.
I also know that queues might grow because of this feature, for same reason it should not be enabled by default.
Again, it's a nice to have - and I would love to see it in cPanel.
If people don't think it's a good idea, then you guys can skip it for sure :)
Replies have been locked on this page!