Data on tape need to be copied on a disk buffer to be accessed.
The buffer is a disk (detached and generally different from the actual disk) that serves as a temporary platform for files that must be migrated or have been recalled from tape. This is not a static disk but once it is full, the oldest and already migrated files are deleted by the garbage collector (however the files remain on tape).
Tape area path will be provided by CNAF. In particular, you can check the list of storage areas at the following page: https://www.cnaf.infn.it/~usersupport/StorageAreas.html where you can find the correspondence between the storage area and the real path on the filesystem.
Check if the file is on the disk (using local POSIX commands)
To know if a file is on the disk, when the user is provided with POSIX access, one can check the file dimension using a command like this:
-bash-4.2$ ls -ls /storage/gpfs_archive/argo/argo-cnaf/testkaren/provaka1_1.txt 0 -rw-r----- 1 argoprod argo 27 Dec 15 2010 /storage/gpfs_archive/argo/argo-cnaf/testkaren/provaka1_1.txt
If the file has null dimension (field number 1 of the output), it is not physically present on the disk, whereas if the actual size (field number 6) is not 0, the file is on tape.
Check if the file is on the disk (with Grid tools using VO based authentication)
To check if the file is on disk, on tape or both, a user can use the following command (to be used with a valid VOMS Proxy):
clientSRM ls -l -v NIG -e <endpoint> -s <file-SURL>
For example:
clientSRM ls -l -v NIG -e httpg://storm-fe-archive.cr.cnaf.infn.it:8444/ -s srm://storm-fe-archive.cr.cnaf.infn.it:8444/ams/${your_file}
Based on the information shown in the output, it is possible to locate the file by checking the value of the fileLocality line:
-[0] fileLocality=0 the file is on disk
-[0] fileLocality=1 the file is on tape
-[0] fileLocality=2 the file is both on disk and tape
Migrate files on tape
As previously explained, when a file has to be moved to tape, one needs to put it into the buffer disk. From there, data will be automatically migrated to tape after a certain time.
It is important to notice that files with white spaces in the name cannot be migrated on tape.
The user can write a file on the buffer disk with a command like this:
gfal-copy your_file srm://storm-fe-archive.cr.cnaf.infn.it:8444/ams/your_file
The file "your_file" will be automatically migrated to tape. To check if the file has been migrated, a user can use a command like the following:
clientSRM ls -l -v NIG -e httpg://storm-fe-archive.cr.cnaf.infn.it:8444/ -s srm://storm-fe-archive.cr.cnaf.infn.it:8444/ams/your_file
If the output contains a line like this:
[0] fileLocality=0
the file has not been migrated yet. If the output contains a line like this:
[0] fileLocality=2
the file has already been migrated to tape.
Recall files from tape (using Grid tools with VO-based authentication)
If a user needs a file that is only on tape, it has to be recalled back to disk. This can be done directly with a clientSRM bol, where "bol" stands for Bring On Line, with a command like this:
clientSRM bol -e httpg://storm-test.cr.cnaf.infn.it:8444 -s srm://storm-test.cr.cnaf.infn.it:8444/tape/file_test
then the output returns a requestToken (for example: requestToken="ea8b525d-1b12-47a5-b8d5-6935ebc53003"), which it can be used to check the status of the Bring On Line operation:
clientSRM sbol -e httpg://storm-test.cr.cnaf.infn.it:8444 -t "ea8b525d-1b12-47a5-b8d5-6935ebc53003"
When the operation is completed, the user gets a Site URL (SURL), that one can exploit to access the recalled file. Whereas, If a user needs to recall multiple files, it can be issued a multiSURL clientSRM bol command:
clientSRM bol -e httpg://storm-test.cr.cnaf.infn.it:8444 -s srm://storm-test.cr.cnaf.infn.it:8444/tape/file_test1 srm://storm-test.cr.cnaf.infn.it:8444/tape/file_test2
After being recalled, the files get pinned. If the user is provided with POSIX access to the filesystem, they can check extended attributes of a file, for example:
getfattr -d /storage/gemss_test1/tape/file_test # file: storage/gemss_test1/tape/file_test user.TSMRecD="1606379599" user.TSMRecR="0" user.storm.migrated user.storm.pinned="1606638797"
Default pin time is 3 days on storm-test, configurable by StoRM admins. After that time, the file could be removed from disk buffer. It can be specified a custom pin lifetime in seconds with -b option of the clientSRM bol command, for example:
clientSRM bol -b 86400 -e httpg://storm-test.cr.cnaf.infn.it:8444 -s srm://storm-test.cr.cnaf.infn.it:8444/tape/file_test
so, this time pin lifetime will be 1 day only. In any case, the user can check extended attributes of the file with gfal-xattr:
gfal-xattr srm://storm-test.cr.cnaf.infn.it:8444/tape/file_test user.replicas = gsiftp://transfer-test.cr.cnaf.infn.it:2811//storage/gemss_test1/tape/file_test user.status = ONLINE_AND_NEARLINE srm.type = StoRM
The user can see that the recalled file is ONLINE_AND_NEARLINE, i.e. it's both on disk and tape. Alternatively to clientSRM bol and clientSRM sbol, they can use, respectively, clientSRM PtG and clientSRM SPtG.
Recall files from tape (without Grid tools)
To recall files from tape, it is necessary to provide the list of the file to be recalled. CNAF will recall them.
Please, contact the User Support team at the mailing list user-support@lists.cnaf.infn.it.