new backup system needs comprehensive logging
Hi ya'll,
The new backup system is awesome, but there is a section where this is decidedly less logging occurring,and it would be nice to have that back for troubleshooting.
The legacy backup system included FTP information in the backup log. I understand that logging of the transferring of backups to remote locations is now done in /usr/local/cpanel/logs/cpbackup_transporter.log. However, a great deal of information is no longer available.
I'd like to see log entries to show:
- started transferring account X via Y transport
- transfer of account X via Y transport [successful | failed]
it would be great if the reason for the failure was logged, if that's discoverable. If this is too much processing, just redirecting (all) the output from the child processes to the logs would be helpful too. :)
Thanks for considering it!
I like this idea 
cPanel, WebHost Manager and WHM are registered trademarks of cPanel, L.L.C. for providing its computer software that facilitates the management and configuration of Internet web servers.
additionally, logging of when the ftp connection hits the timeout threshold
additionally, logging of when the ftp connection hits the timeout threshold
I'm seeing the following in the cPanel error log on a server running 11.38.2 (build 7):
[2013-10-03 02:43:38 -0500] info [backup] Queueing transport of file: /home/backup/2013-10-03/accounts/xxxx.tar.gz
however, it would be nice if this information was logged to the transporter log instead of the cPanel error log.
I'm seeing the following in the cPanel error log on a server running 11.38.2 (build 7):
[2013-10-03 02:43:38 -0500] info [backup] Queueing transport of file: /home/backup/2013-10-03/accounts/xxxx.tar.gz
however, it would be nice if this information was logged to the transporter log instead of the cPanel error log.
For reference, some of the information I requested is now going to the cPanel error log:
[2013-12-08 02:00:01 +0800] info [backup] Started at Sun Dec 8 02:00:01 2013
[2013-12-08 02:00:01 +0800] info [backup] Validating remote transports
[2013-12-08 02:00:01 +0800] info [backup] Setting I/O priority to reduce system load: best-effort: prio 3
[2013-12-08 02:00:01 +0800] info [backup] Hard Linking available on "/backups"
[2013-12-08 02:00:01 +0800] info [backup] Running dir & file backup with target : /backups/2013-12-08/system
[2013-12-08 02:05:50 +0800] info [backup] Running account backup with target : /backups/2013-12-08/accounts
[2013-12-08 02:05:50 +0800] info [backup] Queueing prune operation for remote destination daily backups
[2013-12-08 02:05:50 +0800] info [backup] Completed at Sun Dec 8 02:05:50 2013
For reference, some of the information I requested is now going to the cPanel error log:
[2013-12-08 02:00:01 +0800] info [backup] Started at Sun Dec 8 02:00:01 2013
[2013-12-08 02:00:01 +0800] info [backup] Validating remote transports
[2013-12-08 02:00:01 +0800] info [backup] Setting I/O priority to reduce system load: best-effort: prio 3
[2013-12-08 02:00:01 +0800] info [backup] Hard Linking available on "/backups"
[2013-12-08 02:00:01 +0800] info [backup] Running dir & file backup with target : /backups/2013-12-08/system
[2013-12-08 02:05:50 +0800] info [backup] Running account backup with target : /backups/2013-12-08/accounts
[2013-12-08 02:05:50 +0800] info [backup] Queueing prune operation for remote destination daily backups
[2013-12-08 02:05:50 +0800] info [backup] Completed at Sun Dec 8 02:05:50 2013
Verbose logging specific to each type of transfer would be especially useful. For example amazon s3 support has asked for logs containing the full HTTP Request and Response headers from the Backup Client interacting with S3. Verbose ftp, and sftp logs would also be useful.
Verbose logging specific to each type of transfer would be especially useful. For example amazon s3 support has asked for logs containing the full HTTP Request and Response headers from the Backup Client interacting with S3. Verbose ftp, and sftp logs would also be useful.
My beef is primarily with the new backup system's *transfer* log. An excerpt:
Transport errors encountered.
The system encountered errors during transport of the backup files.
Transport
Backup Label
Message
Backup FTP site
account1
Unable to send “/backup/2016-01-11/accounts/account1.tar.gz” to destination “Backup FTP site”
Backup FTP site
account2
Unable to send “/backup/2016-01-11/accounts/account2.tar.gz” to destination “Backup FTP site”
That's it. "Unable to send" is not descriptive enough to debug: is the source file missing, is the NIC broken, is there no remote FTP server, was there no diskspace, etc?
I had to sift through the /usr/local/cpanel/logs/cpbackup_transporter.log log file to find "warn [cpbackup_transporter] Upload attempt failed: 183142215: No space left on device". A pretty common issue which should IMO be reported in the mailed log.
Can't those warning-/errormessages be included in the transporter log?
My beef is primarily with the new backup system's *transfer* log. An excerpt:
Transport errors encountered.
The system encountered errors during transport of the backup files.
Transport
Backup Label
Message
Backup FTP site
account1
Unable to send “/backup/2016-01-11/accounts/account1.tar.gz” to destination “Backup FTP site”
Backup FTP site
account2
Unable to send “/backup/2016-01-11/accounts/account2.tar.gz” to destination “Backup FTP site”
That's it. "Unable to send" is not descriptive enough to debug: is the source file missing, is the NIC broken, is there no remote FTP server, was there no diskspace, etc?
I had to sift through the /usr/local/cpanel/logs/cpbackup_transporter.log log file to find "warn [cpbackup_transporter] Upload attempt failed: 183142215: No space left on device". A pretty common issue which should IMO be reported in the mailed log.
Can't those warning-/errormessages be included in the transporter log?
Replies have been locked on this page!