Make cpbackup exclude more efficient

We have a cpanel server where a particular user, until very recently, had a large number of static cache files below the home directory. It's a (slightly misconfigured) Yii CFileCache with around 5 million files in three nested levels of subdirectories.

The head folder that contains all of these subdirectories and cache files was excluded from backup via ~/cpbackup-exclude-conf

We have verified (tar tvf) that the containing folder, and certainly none of its contents, never appearead in the backup archive.

However, the daily backup for this user was taking upwards of 5 hours and logged a load average of 10+ and made the whole system unresponsive. This has been going on for weeks. The "copying homedir" part of the backup log produced around 1750 progress dots each morning.

On Friday, we decided to move the whole structure of static cache files out of the home directory, into a subfolder of /var/cache/, and connect the ends again with a single symlink.

Since then, we have seen three daily backups that finished in ~40 minutes each, with associated load averages going no higher than 3.2.

The "copying homedir" part of the backup log is now down to 13 progress dots, too.

At this point we have strong reasons to suspect that the exclude directive does not actually skip the matched files from inspection, but only from final storage. Our exclude pattern has no wildcards. It's one plain directory name per line. Glob matching should never be necessary to find out if a directory is part of the pattern set or not.

Long story short, we believe a straightforward exclude directive on a folder should skip as much processing of that folder as possible. The backup system should avoid listing/statting contents altogether.

The behavior we observed before and after our "workaround" strongly indicates that an exclude directive on a folder located under a user's home does not actually achieve this desirable early-skipping behaviour. We speculate that the excluded folder still gets processed completely, its entire content tree listed, and the exclude pattern only gets applied on a per-file basis after a full, exhaustive list of files under the homedir is created.