Preliminary information
The backup scripts are stored in /u3/scripts;
The log files for these scripts are stored in /u3/scripts/logs; These logs are rotated on dafnenfs from logrotate (check last point);
- The script u2_backup.sh is responsible to create a backup of /u2 in /u3/u2_backup and launch nfs_rsync.sh script to rsync data to nascoda1. The u2_backup.sh script is launched by crontab on host dafnenfs every day (see Table 2 for details). The script sends an e-mail to andrea.michelotti@lnf.infn.it francesco.galletti@lnf.infn.it to notify the exit status.
- The script nfs_rsync.sh is responsible to rsync the data from the host where it is launched to nascoda1. It sends an e-mail to andrea.michelotti@lnf.infn.it francesco.galletti@lnf.infn.it to notify the backup exit status. It is launched by crontab on host danteweb and vldafneafp every day (see Table 2 for details);
- The host dafnenfs share /u3/scripts; danteweb and vldafneafp mount that folder via NFS on boot (folder present in /etc/fstab);
- The host dafnenfs has a custom configuration for logrotate present in /etc/logrotate.d/nfs_rsync to rotate log files generated by these scripts. It will rotate all log files every day, keeping them compressed in /u3/scripts/logs/old folder for 5 days.
Git repository for backup scripts
Link / Path | Note |
---|---|
link: https://baltig.infn.it/lnf-da-control/dcs-backup-scripts branch : main | Script in repository:
|
...
Data
Control system data was saved in a Sun Storagetek 2550 Disk Array (LAT room) connected with two fiber optic paths to beatrix paths to beatrix and virgil, the old production cluster. Now it's all stored and exported to us from the IT department of LNF; These are the folders in which data is organized:
Host | Folder | Backup Folder (nascoda1/2) | Description | ||||
---|---|---|---|---|---|---|---|
dafnenfs | /etc | server_data/dafnenfs/etc | Configuration files | ||||
dafnenfs | /data/diskless | diskless | Filesystem for VMIC diskless clients | ||||
dafnenfs | /data/export/exec/Solaris_9_sparc.all/usr | force_usr | User Filesystem for Force diskless clients | ||||
dafnenfs | /data/export/root | force_root | Root Filesystem for Force diskless clients | ||||
dafnenfs | /data/export/swap | force_swap | SWAP for Force diskless clients | ||||
dafnenfs | /data/u1 | u1 | |||||
dafnenfs | /data/u2 | u2 | Control system, home folders (dafne, vmic e VMs) and other files | ||||
dafnenfs | /data/u3 | u3 | LVTemp, /data/u2 backup and other sensible files | ||||
dafnenfs | /siddharta/dataFromSiddharta | siddharta/dataFromSiddharta | Data written from Siddharta experiment useful for DAFNE | dafnenfs | /siddharta/dataFromDafne | siddharta/dataFromDafne | Data written from DAFNE useful for Siddharta experimentand from DAFNE |
dafnenfs | /data/backup_linacslmonitor | - | vllinacctr001 VM backup | ||||
danteweb | /var/lib/mysql | mysql | Database files (backup disabled; to be re-enabled soon) | ||||
danteweb | /var/www | apache | Web files (backup disabled; to be re-enabled soon) |
Table 1 - data folders
Configurations
Configuration files to be saved:
slmonitor | Folder for A. Stella | |||||||
danteweb | Server | Folder | Backup Folder (nascoda1/2) | Description|||||
---|---|---|---|---|---|---|---|---|
/etc | server_data/danteweb/etc | Apache, MySQL, Memcached config files and all needed for a rollbackConfiguration files | ||||||
danteweb | /opt | server_data/danteweb/opt | Service scripts and data, including a crontab | snapshootsnapshot | ||||
danteweb | /etcvar/lib/mysql | server_data/dafnenfsdanteweb/etcmysql | Configuration Database files (never stored) | |||||
vlmonitor01danteweb | / | etcvar/ | nagioswww | server_data/ | vlmonitor01danteweb/ | etcvar/ | nagioswww | Nagios configurationWeb files |
vlmonitor01vldafneafp | /var/spool/nagiosetc | server_data/vlmonitor01/var/spool/nagios | Nagios home (with public/private keys) | vldafneafp/etc | Configuration files | |||
vldafneafp | /srv | vlmonitor01 | /opt/nagios_custom_plugins | server_data/vlmonitor01/opt/nagios_custom_plugins | Nagios custom plugins | vldafneafp/srv | APPLE DEVIL's FILESYSTEM data shared via AFP |
Table 1 - data Table 2 - configuration folders
Backup planning
Backup crontab are planned as Table 3 2 reports.:
Server | Backup frequency | Time | Description | ||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
dafnenfs | daily | 01:30 | Create a backup of u2 in u3/u2_backup and then launch rsync script | ||||||||
danteweb | To be implemented | dafnenfs | To be implemented | vlmonitor01 | To be implemented | daily | 01:10 | Launch rsync script | |||
vldafneafp | daily | 02:01 | Launch rsync script |
Table 2 Table 3 - Backup planning
Snapshot and replica
Backups have to be Backups are planned on NAS nascoda1.lnf.infn.it (LAT room) and rsynced on NAS nascoda2 nasda.lnf.infn.it (Control Lab, Building 2, Room 117) with following planning (including single volume snapshots):
Source NAS | Destination NAS | Operation | Frequency | Time | Description | |||
---|---|---|---|---|---|---|---|---|
nascoda1 | local | Data volume snapshot | Daily | 00:05 | Snapshot kept fpr 1 month | |||
nascoda1 | nascoda2 | NAS system volume snapshot and replica | Daily | 00:55 | Local NAS system volume snapshot and replica on nascoda2 | |||
nascoda1 | local | Configuration volume snapshot | Weekly (Sunday) | 01:05 | Snapshot kept fpr 1 month | |||
Smart Versioning | - | Hourly snapshots: 24 Daily snapshots: 7 Weekly snapshots: 5 Monthly snapshots: 12 | ||||||
nascoda1 | nasda | Data | nascoda1 | nascoda2 | Data and configuration volume sync | Daily | 04:00 | rsync data from nascoda1 to nascoda2 of all volumes |
nascoda2 | local | Data volume snapshot | Daily | 00:05 | Snapshot kept fpr 1 month | |||
nascoda2 | nascoda1 | NAS system volume snapshot and replica | Daily | 00:55 | Local NAS system volume snapshot and replica on nascoda1 | |||
nascoda2 | local | Configuration volume snapshot | Weekly (Sunday) | 01:05 | Snapshot kept fpr 1 month |
Table 4 - Snapshot and replica planning
Montly manual backup
nasda/nascoda folder |
Table 3 - Snapshot and replica planning
Recover data from backup
Read this documentation page.
Backup on disaster recovery USB disk
To backup DCS from nasda to the disaster recovery disk, this is the procedure to be executed:
- Connect the disk to the front usb port of nasda and wait for the blue usb led to become steady on;
- Keep pressed the "Copy" button in front of nasda for about 3 seconds, then release the button and check that the blue usb led begins to blink and a single beep coming from nasda. When the backup is finished, the blue usb led will be steady on;
- To safely disconnect the disk, keep pressed the "Copy" button for about 6 seconds, until the blue usb led begins to blink and two beeps come from nasda; then release the button and wait for the blue usb led to turn off. Then, disconnect the disk;
- Write the date and the name of who made the backup on the "backup notes" present inside the disk box and keep the disk in a safe place;
There are two videos showing how to start the backup and how to disconnect the disk:
View file | ||||
---|---|---|---|---|
|
View file | ||||
---|---|---|---|---|
|
Excerpt Include | ||||
---|---|---|---|---|
|