It's handy to be able to save disk space using Dovecot's built-in zlib Maildir compression plugin, which transparently compresses and decompresses mail on disk. This just requires that the 'zlib' plugin be added to mail_plugins at the top level and in the pop3 and imap stanzas.
While this can be added manually, having it as a configuration switch within cPanel would be preferable to avoid the config being dropped when Dovecot is upgraded - even editing the main.local template file is not foolproof because cPanel sometimes overwrites/deletes this file and causes havoc.
It is probably fine to make it an enable-only operation, as disabling requires all mail files to first be decompressed and renamed.
I like this idea 
The bug preventing this from working has been resolved, and this is now available in v66 in the CURRENT tier! Get a quick overview of all the improvements on the version 66 Release Site. If you have questions, feel free to join me for the next few hours in this public hipchat room:
https://www.hipchat.com/gQ7xUUD7i
The bug preventing this from working has been resolved, and this is now available in v66 in the CURRENT tier! Get a quick overview of all the improvements on the version 66 Release Site. If you have questions, feel free to join me for the next few hours in this public hipchat room:
https://www.hipchat.com/gQ7xUUD7i
This is something we'd be very interested in seeing as well - Dovecot's storage compression doesn't even rely on cpanel's introduction of mdbox as it works just as well with maildir.
Enabling it for new message deliveries only involves a couple of minor adjustments to Dovecot's config file (as with the OP, on systems where we manually use compression already we just do so via a .local template file currently) with literally 4 lines added and 1 more changed we typically see a 30-40% reduction in storage usage for mail, which is HUGE.
Enabling compression in Dovecot will cause newly-stored mail to be compressed (which is a great start) but doesn't compress existing stored mail, ideally we'd also like to see the feature include a mailbox conversion tool to compress existing stored mail (the process for doing so is again trivial, we use our own script for the purpose and the process is fully documented on Dovecot's wiki)..
This is something we'd be very interested in seeing as well - Dovecot's storage compression doesn't even rely on cpanel's introduction of mdbox as it works just as well with maildir.
Enabling it for new message deliveries only involves a couple of minor adjustments to Dovecot's config file (as with the OP, on systems where we manually use compression already we just do so via a .local template file currently) with literally 4 lines added and 1 more changed we typically see a 30-40% reduction in storage usage for mail, which is HUGE.
Enabling compression in Dovecot will cause newly-stored mail to be compressed (which is a great start) but doesn't compress existing stored mail, ideally we'd also like to see the feature include a mailbox conversion tool to compress existing stored mail (the process for doing so is again trivial, we use our own script for the purpose and the process is fully documented on Dovecot's wiki)..
What would the CPU hit be for having all the email be compressed though?
What would the CPU hit be for having all the email be compressed though?
Revisiting this after a few months... The manual method of enabling this feature is still working well with cPanel v64 (although I had to modify the new .local dovecot file after the upgrade to v64 because that file changed and reverted back to the default .main file).
The CPU hit would depend on the level of compression you specify (on a scale of 1 to 10). I use the default of 6 and have no issues here. Again, this would be a very nice feature for cPanel to decrease disk space usage, and it would be very simple to officially add because all of the functionality is already there, it's just a matter of adding the zlib plugin to a few places in the dovecot file.
Revisiting this after a few months... The manual method of enabling this feature is still working well with cPanel v64 (although I had to modify the new .local dovecot file after the upgrade to v64 because that file changed and reverted back to the default .main file).
The CPU hit would depend on the level of compression you specify (on a scale of 1 to 10). I use the default of 6 and have no issues here. Again, this would be a very nice feature for cPanel to decrease disk space usage, and it would be very simple to officially add because all of the functionality is already there, it's just a matter of adding the zlib plugin to a few places in the dovecot file.
This is currently being investigated. The work has been assigned case COBRA-4455
This is currently being investigated. The work has been assigned case COBRA-4455
Hi Nick,
Great to see this is now underway - Could I ask whether the intention here is for cPanel to handle mail compression only when mdbox is in use, or to also support it when using maildir? (We'd much prefer the latter, and Dovecot itself appears to essentially not care which mailbox format is in use for the purposes of its compression).
Hi Nick,
Great to see this is now underway - Could I ask whether the intention here is for cPanel to handle mail compression only when mdbox is in use, or to also support it when using maildir? (We'd much prefer the latter, and Dovecot itself appears to essentially not care which mailbox format is in use for the purposes of its compression).
This is feature is now available in EDGE under WHM, Service Configuration, Mailserver Configuration
This is feature is now available in EDGE under WHM, Service Configuration, Mailserver Configuration
Is there an automated option or script to compress existing messages?
Is there an automated option or script to compress existing messages?
Same question.. existing messages be compressed when this feature is enable?
Same question.. existing messages be compressed when this feature is enable?
If there is not an automatic way to compress old/existing emails, I'm pretty sure that converting to/from the maildir/mdbox will compress the old ones. (Whichever one you use, just convert to the other one and then back).
If there is not an automatic way to compress old/existing emails, I'm pretty sure that converting to/from the maildir/mdbox will compress the old ones. (Whichever one you use, just convert to the other one and then back).
This feature will only work with new incoming emails. The work to compress existing messages is a bit too much to be included in the existing system. We'll discuss internally if there are other (similar to the one @jadafields mentioned).
This feature will only work with new incoming emails. The work to compress existing messages is a bit too much to be included in the existing system. We'll discuss internally if there are other (similar to the one @jadafields mentioned).
Not sure if intended but it seems like this work has already been done in a roundabout way Benny - We've come to testing this now and while simply enabling the option to compress messages doesn't compress existing messages (or new messages when using maildir, we've raised a bugreport) jandafields is right that the mailbox conversion process does handle the compression.
So all existing mail can be compressed currently by enabling the compress messages option then converting all mailboxes from your current chosen storage method to the other, then back again.
Not sure if intended but it seems like this work has already been done in a roundabout way Benny - We've come to testing this now and while simply enabling the option to compress messages doesn't compress existing messages (or new messages when using maildir, we've raised a bugreport) jandafields is right that the mailbox conversion process does handle the compression.
So all existing mail can be compressed currently by enabling the compress messages option then converting all mailboxes from your current chosen storage method to the other, then back again.
Have we established this feature will work with both mdbox and maildir ?
And will it affect the cpanelsolr indexing for search ?
Have we established this feature will work with both mdbox and maildir ?
And will it affect the cpanelsolr indexing for search ?
The bug preventing this from working has been resolved, and this is now available in v66 in the CURRENT tier! Get a quick overview of all the improvements on the version 66 Release Site. If you have questions, feel free to join me for the next few hours in this public hipchat room:
https://www.hipchat.com/gQ7xUUD7i
The bug preventing this from working has been resolved, and this is now available in v66 in the CURRENT tier! Get a quick overview of all the improvements on the version 66 Release Site. If you have questions, feel free to join me for the next few hours in this public hipchat room:
https://www.hipchat.com/gQ7xUUD7i
Replies have been locked on this page!