How can I perform automatic database backups and cleanups of Polyspace Access from R2020b?

67 views (last 30 days)
How can I perform automatic database backups and cleanups of Polyspace Access from R2020b?

Accepted Answer

MathWorks Support Team
MathWorks Support Team on 6 Sep 2022 at 4:00
Edited: MathWorks Support Team on 15 Sep 2022 at 10:07
Here is an example of how to do both backup and cleanup as once-a-week cronjobs:
1) 'crontab' should look like follows (run 'crontab -e' as root):
0 0 * * 0 /var/lib/polyspace/cleanup_access.sh > /var/backups/polyspace/cleanup_access.log 2>&1
0 1 * * 0 /var/lib/polyspace/backup_access.sh > /var/backups/polyspace/backup_access.log 2>&1
2) If /var/backups/polyspace does not exist, create this folder using the shell command:
mkdir /var/backups/polyspace
Warning: the folder /var/backups/polyspace should contain enough free disk space to store the copy of the database. Otherwise, you could experience problems during the backup.
To know the size of the database, enter the shell command:
  • For Polyspace R2022a and beyond:
docker exec -i polyspace-access-db-0-main psql -a -b -U postgres prs_data -t -c "SELECT pg_size_pretty(pg_database_size('prs_data'))"
  • For Polyspace R2020b through R2021b:
docker exec -i polyspace-access-db-main psql -a -b -U postgres prs_data -t -c "SELECT pg_size_pretty(pg_database_size('prs_data'))"
To know the disk space available for /var/backups/polyspace, enter the shell command:
df -h /var
If there is not enough space on /var, instead of var/backups/polyspace you could choose another folder for the backup file and update the cronjob accordingly.
3) Put the following 3 files in /var/lib/polyspace:
  •  cleanup_access.sh:
#!/bin/sh
set -e
BASEDIR=$(dirname "$0")
# Set the ACCESS_STORAGE_DIR variable according your Polyspace installation path.
# The path of the Storage directory is available in the Cluster Settings of the Cluster Admin interface.
# Alternatively, you can find this path in the settings.json file by searching for etlStorageDir.
ACCESS_STORAGE_DIR=/local/home/dtstrunner/ps_access/appdata/polyspace-access/storage
date
echo ""
echo "Cleanup starts..."
$BASEDIR/make_clean_commands.pl
echo ""
echo "To clean:"
cat $BASEDIR/cleanup.pscauto
echo ""
echo "Requesting clean in $ACCESS_STORAGE_DIR..."
cp $BASEDIR/cleanup.pscauto $ACCESS_STORAGE_DIR/.
  • backup_access.sh:
For Polyspace R2022a and beyond:
#!/bin/sh
set -e
date
echo "Pausing services..."
docker stop polyspace-access-etl-0-main polyspace-access-web-server-0-main
echo "Backing up database..."
time docker exec polyspace-access-db-0-main pg_dumpall -U postgres | gzip > /var/backups/polyspace/backup_access.gz
echo "Resuming services..."
docker start polyspace-access-etl-0-main polyspace-access-web-server-0-main
For Polyspace R2020b through R2021b:
#!/bin/sh
set -e
date
echo "Pausing services..."
docker stop polyspace-access-etl-main polyspace-access-web-server-main
echo "Backing up database..."
time docker exec polyspace-access-db-main pg_dumpall -U postgres | gzip > /var/backups/polyspace/backup_access.gz
echo "Resuming services..."
docker start polyspace-access-etl-main polyspace-access-web-server-main
  •  make_clean_commands.pl:
#!/usr/bin/perl
# 1. Please change the command to list projects according to your needs (e.g. path to binary, hostname, login, protocol, password)
my $ps_access_command = '"/usr/local/Polyspace_Server/R2020b/polyspace/bin/polyspace-access" -host access.mycompany.org -protocol http -list-project -login admin -encrypted-password ABABABABABABABABABA';
# 2. Please change the location of the output file
my $pscauto_filename = './cleanup.pscauto';
# 3. 
$value = qx($ps_access_command);
@lines = grep(/ProjectsWaitingForDeletion\/.*/, split('\n', $value));  # use this for recursive delete
#@lines = grep(/ProjectsWaitingForDeletion\/.* RUN_ID \d+$/, split('\n', $value));  # use this to only delete leaf nodes
@lines = map { s/(.+?)( RUN_ID .*)?$/delete_project \"\1\"/g; $_ } @lines; 
open(FH, '>', $pscauto_filename) or die $!;
print FH join("\n", @lines);
close(FH);
print "\n$pscauto_filename successfully written!\n\n";
Details that must be adjusted to the installation:
  1. in make_clean_commands.pl: path, username, and password (note: encrypted password as in other server commands)
  2. in cleanup_access.sh: ACCESS_STORAGE_DIR according to the Polyspace Access installation path
  3. in case any paths are changed, backup_access.sh and crontab must be adjusted, too
Please note:
  • There should be several GB of free disk space in /var/backups since the database (even compressed) can be quite large.
  • A backup can take quite a while and the database is unavailable during the operation, therefore this should be done overnight or at the weekends.
Note that this is very simplistic, but it works (tested with R2020b on Debian and Ubuntu). For advanced use, log rotation and rotating backups might be interesting (https://rotate-backups.readthedocs.io/en/latest), as well as NFS/AFS/Samba/... backups instead of putting them locally into /var/backups. Users might also want to distribute these files via puppet/ansible/chef, in case they have multiple Access instances. For more information about 'puppet', please see: https://puppet.com/blog/automating-puppet-cron-jobs-and-scheduled-tasks/
More information regarding the 'crontab' syntax can be found here: 
  2 Comments
Martin Becker
Martin Becker on 24 May 2022
Note that since R2022a the container names have changed. It is now: polyspace-access-etl-0-main, polyspace-access-web-server-0-main and polyspace-access-db-0-main (the zero is new)

Sign in to comment.

More Answers (0)

Tags

No tags entered yet.

Products


Release

R2020b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!