BackupPC 4.4.0 release

This release contains several new features and some bug fixes. New features include:

• any full/filled backup can be marked for keeping, which prevents any expiry or deletion
• any backup can be annotated with a comment (eg, "prior to upgrade of xyz")
• added metrics CGI (thanks to @jooola) that replaces RSS and adds Prometheus support
• tar XferMethod now supports xattrs and acls
• rsync XferMethod now correctly supports xattrs on directories and symlinks
• nightly pool scanning now verifies the md5 digests of a configurable fraction of pool files
• code runs through perltidy so format is now uniform (thanks to @jooola, with help from @shancock9 and @moisseev)

New versions of BackupPC::XS (0.62) and rsync-bpc (3.0.9.15, 3.1.2.2 or 3.1.3beta0) are required.

Thanks to Jeff Kosowsky for extensive testing and debugging for this release, particularly around xattrs.

Here are the more detailed changes:

• Merged pull requests #325, #326, #329, #330, #334, #336, #337, #338, #342, #343, #344, #345, #347, #348, #349
• Filled/Full backups can now be marked as "keep", which excludes them from any expiry/deletion. Also, a backup-specific comment can be added to any backup to capture any important information about that backup (eg, "pre-upgrade of xyz").
• Added metrics CGI, which adds Prometheus support and replaces RSS, by @joola (#344, #347)
• Tar XferMethod now supports xattrs and acls; xattrs should be compatible with rsync XferMethod, but acls are not
• Sort open directories to top when browsing backup tree
• Format code using perltidy, and included in pre-commit flow, by @joola (#334, #337, #342, #343, #345). Thanks to @joola and @shancock9 (perltidy author) for significant effort and support, plus improvements in perltidy, to make this happen.
• Added $Conf{PoolNightlyDigestCheckPercent}, which checks the md5 digest of this fraction of the pool files each night.
• $Conf{ClientShareName2Path} is saved in backups file and the share to client path mapping is now displayed when you browse a backup so you know the actual client backup path for each share, if different from the share name
• configure.pl now checks the per-host config.pl in a V3 upgrade to warn the user if $Conf{RsyncClientCmd} or $Conf{RsyncClientRestoreCmd} are used for that host, so that the new settings $Conf{RsyncSshArgs} and $Conf{RsyncClientPath} can be manually updated.
• Fixed host mutex handling for dhcp hosts; shifted initial mutex requests to client programs
• Updated webui icon, logo and favicon, by @moisseev (#325, #326, #329, #330)
• Added $Conf{RsyncRestoreArgsExtra} for host-specific restore settings
• Language files now all use utf8 charsets
• Bumped required version of BackupPC::XS to 0.62 and rsync-bpc to 3.0.9.15.
• Ping failure message only written to stdout only if verbose
• BackupPC_backupDelete removes partial v3 backup in HOST/new; fixes #324 reported by @thalueng
• BackupPC_backupDelete adds -f option to over keep, if set
• BackupPC_nightly: change -f to -F when running BackupPC_refCountUpdate during fsck
• Better formatting of eval cmd result in cmdSystemOrEvalLong()
• Remove zero count entries in per-backup and per-host poolCnt files
• Add a fake version parameter to the CSS URL to overcome caching
• Fixed ALRM typo in BackupPC_dump, by @rhansen (#348)
• Fixed command name in error message in BackupPC_archive, by @rhansen (#349)
• lib/BackupPC/Xfer/Rsync.pm skips adding --iconv option when $Conf{ClientCharset} is utf8 or empty, to avoid a long-standing rsync bug
• lib/BackupPC/Xfer/Rsync.pm provides a more informative warning when the client rsync exits with an IO error, and also includes the IO error in the xferErr count
• lib/BackupPC/Xfer/Smb.pm recognizes NT_STATUS_LOGON_FAILURE error
• Increase text length in config editor fields, by @moisseev (#336)
• Typo fix in config.pl, by @cure (#338)
• Minor documentation updates, removing old SourceForge links

BackupPC 4.3.2 release

This release contains several bug fixes and a few minor features:

• Merged pull requests #278, #281, #289, #295, #297, #307, #308, #311, #312, #313, #314.
• Added per-host read/write exclusivity to worker programs, so command-line programs don't collide with backups or other operations; reported by @hamster65 (issue #299)
• Added $Conf{RsyncIncrArgsExtra} (issue #284)
• Added $Conf{ClientShareName2Path}, which allows mapping of share names/path to real paths on the client (eg, to backup a snapshot that's in a different directory to the share name path)
• Fixed v3 digest calculation in bin/BackupPC_backupDuplicate and bin/BackupPC_migrateV3toV4; reported by @palmtop (issue #291)
• Improved handling of BackupPC_nightly running for more than 24 hours: drop next queued run, and provide better log messages; reported by @guestisp (issue #303)
• Improved error checking on $Conf{BackupPCNightlyPeriod}, and update $Info->{NightlyPhase} if it's larger than $Conf{BackupPCNightlyPeriod}; reported by @guestisp (issue #304)
• Fixed warnings in bin/BackupPC, submitted by @moisseev (#278)
• Avoid rrd updates using the same time stamp, submitted by @moisseev (#311, issue #305)
• Removed unused counting of renamed files, submitted by @moisseev (#281)
• Make tar xfer output parsing compatible with FreeBSD tar, submitted by @haarp (#289)
• Fix daemon stdin open to read mode, submitted by @moisseev (#308)
• Hosts config editor table layout CGI fix, submitted by @steven-de-wit (#297)
• Fixes to French translation, submitted by @pjoubert- (#295)
• Fixes to Italian translation, submitted by @guestisp (#314)
• Updated comments for Apache 2.4 config in httpd/src/BackupPC.conf, submitted by @NotAProfessionalDeveloper (#307)
• Documentation update for SCGI prerequisite, submitted by @guestisp (#312)
• Documentation update for nginx config, submitted by @guestisp (#313)

BackupPC 4.3.1 release

This release contains several bug fixes.

• Merged pull request #240.
• Fixed CGI host summary columns for new Comment value; reported by @karlis-irmejs (#245).
• Added support to BackupPC_tarCreate for PAX headers to allow large metadata values to be correctly encoded; reported by @seblu (#254).
• Fixed regexp in BackupPC so that versions like rsync-bpc 3.1.2beta0 are matched. PR submitted by Adrien Ferrand (#240).
• Added more details to error messages when failing to connect to BackupPC server; reported by @dmak (#261).
• bin/BackupPC Main_Initialize() checks if a BackupPC server is running by trying to connect to it, in addition to existing check that PID exists; reported by @toggoboge (#264).
• Fixed utf8 encoded share names in deletion of orphan RsyncShareNames, reported by @Yann79 (#266).
• Fixed %Conf passing to perl code version of $bpc->cmdSystemOrEval; reported by Jeff Kosowsky.
• BackupPC_backupDelete removes files with BPC_FTYPE_DELETED from filled merged backup; reported by Jeff Kosowsky.
• added Data::Dumper SortKeys(1) to lib/BackupPC/Storage/Text.pm so that config hash writes have repeatable order; suggested by @kr4ut (#275).
• bin/BackupPC_archive: use $bpc->openPCLogFile() instead of manual LOG file rotation
• lib/BackupPC/CGI/Archive.pm: create client directory if it doesn't exist

BackupPC 4.3.0 release

This release includes a couple of new features (eg, checking inode usage on the BackupPC filesystem) and a few improvements (eg, html table sorting), and several bug fixes:

• Merged pull requests #200, #208, #216, #217, #229, #230, #231, #232, #233, #235.
• Added checking of file system inode usage, with new configuration settings $Conf{DfInodeUsageCmd} and $Conf{DfMaxInodeUsagePct}.
• bin/BackupPC_tarExtract: fixes to hardlink handling on incrementals, plus cleanup
• bin/BackupPC_backupDuplicate: fixed directory creation for copying v3 backups.
• bin/BackupPC_backupDelete: couple of inode-related bug fixes; also do fsck if $Conf{RefCntFsck} != 0.
• bin/BackupPC: fixed handling of dhcp hosts, reported by Jean-Marc.
• bin/BackupPC: improved version check of rsync_bpc to allow period before "beta", reported by @ktenney in issue #214.
• Added $Conf{EMailAdminSubject} to allow the admin email subject to be set.
• Changed default $Conf{CgiDateFormatMMDD} to 2 (YYYY-MM-DD), submitted by @moisseev (#231).
• Allow multiple groups in $Conf{CgiAdminUserGroup}, submitted by @moisseev (#235).
• Improvements to html table sorting, and made more tables sortable, submitted by @brenard and @moisseev (issue #199, PRs #200, #229, #230, #232).
• Add action type column to queue status page (#208), submitted by @brenard.
• lib/BackupPC/CGI/Summary.pm: host summary only reports age of completed backups (not partial or active), suggested by Michael Selway.
• lib/BackupPC/CGI/HostInfo.pm: show active backup as partial if there is no job running (to handle case of abnormal exit of BackupPC), reported by @8666 in issue #220.
• systemd/src/backuppc.service: added ExecReload, submitted by @opoplawski (#233).
• lib/BackupPC/Xfer/Rsync.pm: empty $Conf{RsyncdPasswd} is no longer fatal; removed remaining mentions of RsyncdAuthRequired from conf/config.pl and lib/BackupPC/Config/Meta.pm; reported by @jooola in issue #224.
• bin/BackupPC_dump: moved alarm($Conf{ClientTimeout}) to after all the pre-backup code (including expiry, duplication etc of backups, which could be slow) reported by Michael Selway.
• bin/BackupPC_refCntUpdate: added a couple of bpc_progress_state prints to improve status reporting during nightly (-m), reported by Michael Selway.
• lib/BackupPC/CGI/RSS.pm: added Disabled value to host info (copied from $Conf{BackupsDisable}); suggested by @danielmotaleite (issue #222).
• lib/BackupPC/Xfer/Rsync.pm: added checks for a couple of common rsync file error and error exit messages that now increment the xferErrs count.
• bin/BackupPC_tarExtract: errors related to a particular file are counted as Xfer errors, rather than considered fatal.
• bin/BackupPC_dump: added check that $LogFd is defined before using it.
• lib/BackupPC/Lib.pm: replaced exit() with POSIX::_exit() so that object destruction is skipped in the child process. This fixes corruption of the compressed XferLOG file if the exec of $Conf{DumpPostUserCmd} fails.
• conf/config.pl: updated project home page to https://backuppc.github.io/backuppc in email message footers.

BackupPC 4.2.1 release

This release allows the new backup delete CGI feature to be disabled for users and/or admins, plus a couple of minor bug fixes:

• Merged pull request #195.
• Added new config variable $Conf{CgiUserDeleteBackupEnable} that sets whether users and admins can delete backups via the CGI interface. The default value is 0, which means it's disabled for regular users but available for admins.
• Fixed delete backup bug in bin/BackupPC_Admin_SCGI reported by Julian Zielke in issue #193.
• Added check to dirCacheFlush() in bin/BackupPC_tarExtract to skip files that don't have attributes; reported by Tarak Patel.
• Removed extraneous duplicate variable assignment in lib/BackupPC/CGI/GeneralInfo.pm, from @moisseev (#195).

BackupPC 4.2.0 release

Adds a new feature that allows backups to be deleted from the CGI interface.

Full list of changes:

• Merged pull requests #160, #190.
• Backups can now be deleted via the CGI interface, written by @moisseev (#160).
• bin/BackupPC_backupDelete: added -L option that puts output into client LOG file, to support CGI backup deletion.
• Added support for a user-editable comment per host, via a new ClientComment config parameter, requested by @andrewmaksymowsky.
• bin/BackupPC_tarExtract: Added support for pax headers, which smbclient uses for long file names
• bin/BackupPC_backupDelete: make sure directory exists when renaming tree from deleted backup.
• lib/BackupPC/Xfer/Smb.pm: ignore empty output lines from smbclient; fixes issue #159.
• bin/BackupPC: improved several of the exit error messages.
• lib/BackupPC/Xfer/Rsync.pm: added shareName to RsyncArgs* argument substitutions; suggested by Alex Kobel.
• conf/BackupPC_stnd.css: removed import url('....') suggested by @MartijnRas in issue #174.

BackupPC 4.1.5 release

Fixes bug related to hardlinks not being correctly stored in certain cases, raised in issues #123 and #124. Changes are:

• Changed required BackupPC::XS version from 0.56 to 0.57.
• bin/BackupPC_dump now updates inodeLast for share being backed up.
• bin/BackupPC_refCountUpdate: inodeLast is checked and updated during fsck; needs BackupPC::XS 0.57.

Thanks to @SebastianS90 and others for all the help providing detailed debug information, and testing the fixes.

BackupPC 4.1.4 release

BackupPC 4.1.4 fixes various bugs:

• Merged pull requests #99, #121, #125, #131, #133, #134, #137, #148, #149, #150 #151, #152, #153, #155, #157, #167

• lib/BackupPC/Xfer/Smb.pm: made pipeSMB non-blocking to avoid a reported deadlock when BackupPC's select() returns ok for reading, but there are no bytes to read from the client tar's log/stdout output. Parallel change to lib/BackupPC/Xfer/Tar.pm in 4.1.3.

• bin/BackupPC_tarCreate and bin/BackupPC_zipCreate: untaint the host name so they work with setuid under CGI; fixes empty tar or zip files downloaded via CGI interface (fixes issue #156)

• bin/BackupPC: fixed BackupPC::XS min version checking and error message, from @moisseev (#152)

• bin/BackupPC: added more detailed startup information (perl and BackupPC version) to log, from @moisseev (#157)

• bin/BackupPC_rrdUpdate: fixed empty pools hiding from @moisseev (#167)

• lib/BackupPC/Xfer/Smb.pm: now ignores additional debug messages from smbclient, and flags lines in the XferLOG it doesn't recognize.

• lib/BackupPC/CGI/Browse.pm: default display now has the last, rather than first, share opened.

• Replaced submit with button so that Enter doesn't activate the Delete button. Fixes issue #161, reported by Philippe-M.

• removed commented-out settings for some ftp args (eg, port#) in lib/BackupPC/Xfer/Ftp.pm; reported by Adam W.

• bin/BackupPC_backupDelete: only print delta counts if LogLevel is >= 5

• bin/BackupPC_tarExtract: fix existing file size count and size

• lib/BackupPC/CGI/EditConfig.pm: fixed masking of subheadings in config editor.

• config/config.pl: added -mSMB3 to $Conf{SmbClientIncrCmd} and $Conf{SmbClientRestoreCmd}, from @SvenBunge (#99)

• lib/BackupPC/Xfer/Rsync.pm: improved cleanup of orphan rsyncTmp files

• In bin/BackupPC_dump, added "share" to bpc_progress_state message so it is 'backup share "$shareName"'. Patch #150 by @guestisp (issue #143)

• added share name to log message in lib/BackupPC/CGI/Restore.pm for tar and zip restore.

• makeDist: fixed exit code from @moisseev (#153)

• Added Travis CI configuration from @moisseev (#155) and enabled travis

• Replaced "Homepage" with "Github" in config.pl and configure.pl from @moisseev (#121)

• Spelling fixes, mainly in comments from @ka7 (#125).

• Fixed comment in config.pl (zh_CH -> zh_CN) from @patch (#131)

• Fixed German translations from @mainboarder (#133, #134)

• Fixed minor comment typo in config.pl from @pbe-axelor (#137)

• Fixed comments in systemd/README from @schuetzm (#138)

• Fixed Italian translations from @guestisp (#148, #149; issue #142)

• Fixed incorrect hash key in German translations from @moisseev (#151)

BackupPC 4.1.3 release

BackupPC 4.1.3 fixes several bugs:

• Merged pull requests: #109, #114

• Fixed editing of compound menu variables (eg: BackupFilesOnly).

• Made tarPipe in lib/BackupPC/Xfer/Tar.pm non-blocking to avoid a reported deadlock when BackupPC's select() returns ok for reading, but there are no bytes to read from the client tar's log/stdout output. Thanks to Matt Bedynek for running various tests and providing debugging insights to track this down.

• Better error checking when using $f->read() on pool files. Thanks to Cody Jackson for tracking down this issue, related to reading corrupted compressed pool files. There's also an additional fix in backuppc-xs (version 0.54).

• Cleans up any orphan temporary pool writing files.

• Fixed utf-8 output in SCGI.

• Fixed a reference counting bug in BackupPC_tarExtract.

• Fixed rsync restore transfer byte total, reported by Alexander Moisseev

• Replaced logo href with https://backuppc.github.io/backuppc.

• On a v3->v4 upgrade, remove the new --one-file-system flag from the new RsyncArgs if it wasn't there before.

• Added /usr/local/bin to search path in configure.pl from Alexander Moisseev (#109).

• Avoid missing or extra quotes when replacing misused undef or empty string values in configure.pl from Alexander Moisseev (#114).

• Chasing down a still unsolved bug with help from Lano and Dieter Fauth where newly added pool files in uncompressed backups get removed by BackupPC_refCountUpdate during a long-running backup, or if BackupPC_migrateV3toV4 is running. Two workarounds added in this release: BackupPC_migrateV3toV4 will now exit if BackupPC is running, and BackupPC_refCountUpdate only removes pool files that are more than a week old.

BackupPC 4.1.2 release

BackupPC 4.1.2 fixes several bugs in 4.1.1. The changes include:

• Merged pull requests: #93, #94, #97, #102
• Fixed NetBios lookup of hosts, reported by Doug Lytle.
• Fixed bin/BackupPC_tarExtract and lib/BackupPC/Xfer/Ftp.pm in case where a directory tree is no longer present in a new backup. Reported by Jens Potthast and Matt Bedynek.
• Fixed SCGI when BackupPC is run in non-daemon mode (which is the systemd default starting in 4.1.1).
• $Conf{ClientNameAlias} can now be an array of hostnames; the first one that succeeds ping is used for the backup or restore. From martintamare (#94).
• Fixed status link in SCGI server, improved the navigation tab (now it doesn't scroll) and some css/html cleanup; from Nicholas Hall (#97).
• Fixed logic in bin/BackupPC_dump for $Conf{FixedIPNetBiosNameCheck} to prevent netbios name check; from Nicholas Hall (#102).
• Fixed config editor bug in sub-entries of hash config variables, related to SCGI.
• Fixed rsync restore of a top-level directory when the share is "/", reported by Ray Frush.
• Increased $Conf{FullKeepCnt}[0] by 1 in expiry calculations, so the most recent filled backup effectively isn't counted for expiry. This improves the behavior, particularly when $Conf{FullKeepCnt} = 1.
• Rsync transfers now use --timeout=$Conf{ClientTimeout} instead of using an alarm based on rsync log output, suggested by ACR.
• added --delete-excluded and --one-file-system to $Conf{RsyncArgs}.
• Removed extra quotes from $Conf{CgiURL} in configure.pl from Alexander Moisseev (#93). Also added $DestDir to some print messages in configure.pl.
• BackupPC_dump and BackupPC_restore now support the -p option to turn off progress reports. The old -p (inplace) option to BackupPC_tarExtract is now -P.
• Backup directory mtime is now set to the backup endTime.
• Updated FSF address in cgi-bin/BackupPC_Admin, reported by TomCat42 (#91).