| Table of Contents |
|---|
Submission utility
To ease the transition to the new cluster we implemented a solution based on environment modules. It will set all environment variables needed to correctly submit to both the old and new cluster.
Once logged into any UI this utility can be activated by executing:
$ source /opt/exp_software/opssw/budda/modules/enableWe strongly encourage you to insert this source into your ~/.bashrc file.
Once the utility has been sourced you will be available. You can list all the available modules using:
...
| Code Block | ||||||
|---|---|---|---|---|---|---|
| ||||||
apascolinit1@ui-tier1 ~ $ source /opt/exp_software/opssw/budda/modules/enable apascolinit1@ui-tier1 ~ $ module avail -------------------------------------------------------- /opt/exp_software/opssw/budda/modules/modulefiles --------------------------------------------------------- htc/auth htc/ce htc/local use.own Key: modulepath default-version |
...
- htc/local - to be used once you want to submit/query local schedds sn-02 or sn01-htc (HTCondor23 scheddsubmit/query local schedds sn-02 or sn01-htc (HTCondor23 schedd), supports variables specification:
variable values description ver 9 connects to the old HTCondor cluster and local schedd (sn-02) 23 connects to the new HTCondor cluster and local schedd (sn01-htc) Code Block language bash theme Midnight title Usage of htc/local module apascolinit1@ui-tier1 ~ $ module switch htc/local ver=9 apascolinit1@ui-tier1 ~ $ condor_q -- Schedd: sn-02.cr.cnaf.infn.it : <131.154.192.42:9618?... @ 04/17/24 14:58:44 OWNER BATCH_NAME SUBMITTED DONE RUN IDLE HOLD TOTAL JOB_IDS Total for query: 0 jobs; 0 completed, 0 removed, 0 idle, 0 running, 0 held, 0 suspended Total for apascolinit1: 0 jobs; 0 completed, 0 removed, 0 idle, 0 running, 0 held, 0 suspended Total for all users: 50164 jobs; 30960 completed, 1 removed, 12716 idle, 4514 running, 1973 held, 0 suspended apascolinit1@ui-tier1 ~ $ module switch htc/local ver=23 apascolinit1@ui-tier1 ~ $ condor_q -- Schedd: sn01-htc.cr.cnaf.infn.it : <131.154.192.242:9618?... @ 04/17/24 14:58:52 OWNER BATCH_NAME SUBMITTED DONE RUN IDLE HOLD TOTAL JOB_IDS Total for query: 0 jobs; 0 completed, 0 removed, 0 idle, 0 running, 0 held, 0 suspended Total for apascolinit1: 0 jobs; 0 completed, 0 removed, 0 idle, 0 running, 0 held, 0 suspended Total for all users: 0 jobs; 0 completed, 0 removed, 0 idle, 0 running, 0 held, 0 suspended suspende
- htc/auth - helps to setup authentication methods for Grid submission
variable values description auth
GSI sets up GSI authentication (old cluster only)
SSL
sets up SSL authentication (new cluster only) SCITOKENS
sets up SCITOKENS authentication Code Block language bash theme Midnight title Usage of htc/auth module apascolinit1@ui-tier1 ~ $ module switch htc/auth auth=SSL Don't forget to voms-proxy-init! apascolinit1@ui-tier1 ~ $ module switch htc/auth auth=SCITOKENS Don't forget to "export BEARER_TOKEN=$(oidc-token <client-name>)"!
- htc/ce - eases the usage of condor_q and condor_submit commands setting up all the needed variables to contact ous CEs
our CEsvariable values description num 1,2,3,4 connects to ce{num}-htc (new cluster) 5,6,7 connects to ce{num}-htc (old cluster) auth GSI,SSL,SCITOKENS calls htc/auth with the selected auth method Code Block language bash theme Midnight title Usage of htc/ce module apascolinit1@ui-tier1 ~ $ condor_q Error: ...... apascolinit1@ui-tier1 ~ $ module switch htc/ce auth=SCITOKENS num=2 Don't forget to "export BEARER_TOKEN=$(oidc-token <client-name>)"! Switching from htc/ce{auth=SCITOKENS:num=2} to htc/ce{auth=SCITOKENS:num=2} Loading requirement: htc/auth{auth=SCITOKENS} apascolinit1@ui-tier1 ~ $ export BEARER_TOKEN=$(oidc-token htc23) apascolinit1@ui-tier1 ~ $ condor_q -- Schedd: ce02-htc.cr.cnaf.infn.it : <131.154.192.41:9619?... @ 04/17/24 15:48:24 OWNER BATCH_NAME SUBMITTED DONE RUN IDLE HOLD TOTAL JOB_IDS .......... .......... ..........
...
- Submitting a job to the cluster.
Code Block language bash theme Midnight title Executable and Submit file apascolinit1@ui-tier1 ~ $ cat sleep.sh #!/bin/env bash sleep $1 apascolinit1@ui-tier1 ~ $ cat submit.sub # Unix submit description file # subimt.sub -- simple sleep job batch_name = Local-Sleep executable = sleep.sh arguments = 3600 log = $(batch_name).log.$(Process) output = $(batch_name).out.$(Process) error = $(batch_name).err.$(Process) should_transfer_files = Yes when_to_transfer_output = ON_EXIT queue
Code Block language bash theme Midnight title Submission and control of job status apascolinit1@ui-tier1 ~ $ module switch htc/local ver=23 apascolinit1@ui-tier1 ~ $ condor_submit submit.sub Submitting job(s). 1 job(s) submitted to cluster 15. apascolinit1@ui-tier1 ~ $ condor_q -- Schedd: sn01-htc.cr.cnaf.infn.it : <131.154.192.242:9618?... @ 03/18/24 17:15:44 OWNER BATCH_NAME SUBMITTED DONE RUN IDLE TOTAL JOB_IDS apascolinit1 Local-Sleep 3/18 17:15 _ 1 _ 1 15.0 Total for query: 1 jobs; 0 completed, 0 removed, 0 idle, 1 running, 0 held, 0 suspended Total for apascolinit1: 1 jobs; 0 completed, 0 removed, 0 idle, 1 running, 0 held, 0 suspended Total for all users: 1 jobs; 0 completed, 0 removed, 0 idle, 1 running, 0 held, 0 suspended
...