and RSCC
Getting Started
instruction is here. Note that you need to be a RIKEN trainee as indicated in that instruction.
http://ccjsun.riken.go.jp/ccj/
Finding PHENIX data
http://ccjsun.riken.go.jp/ccj/doc/phenix-data/index.html
setup at CCJ
http://ccjsun.riken.go.jp/ccj/Wako-Sys/
PHENIX DB at CCJ
http://ccjsun.riken.go.jp/ccj/doc/phenix-data/phenix-db/
to check the status:
/ccj/u/script/postgres/psql-status-prod.csh
to update:
/ccj/u/script/postgres/psql-update-prod.csh
Run 8 DST
LSF
http://ccjsun.riken.go.jp/ccj/doc/LSF/Primer.html
http://www-cdf.fnal.gov/offline/runii/fcdfsgi2/lsf_batch.html
use "bpeek" to check the output when running
run8pp production
login : phnxreco@pc2cn003
prod db
psql -h ccjams001.riken.go.jp -d BadEvents
tcsh
http://www.grymoire.com/Unix/Csh.html
Unlike other shell, impossible to have double-quote inside double quote. But can do this
echo "The date command returns "'"'`date`'"'
HPSS, hsi
http://www.nersc.gov/nusers/resources/HPSS/hsi/Doc/
http://www.sdsc.edu/us/resources/hpss/hsi.html
MRTG document
http://oss.oetiker.ch/mrtg/doc/mrtg-reference.en.html
Link to RSCC
rsh
the error "poll: protocol failure in circuit setup"
https://secure-support.novell.com/KanisaPlatform/Publishing/466/3077223_f.SAL_Public.html
"do_ypcall: clnt_call: RPC: Timed out"
don't know what is the cause yet.
some SQLs
SELECT filename,status2,jobend-jobstart FROM prodjob_run6 WHERE runnumber=198179 ORDER BY filename; SELECT outfilename,status FROM aggjob_run6 WHERE runnumber=198169 ORDER BY outfilename; SELECT runnumber,count(filename) FROM prodjob_run6 GROUP BY runnumber ORDER BY runnumber; SELECT status2,filename,retcode from prodjob_run6 where status2<>1 order by filename;
Transfer with gridftp/srm
from CCJ to dCache@RCF
- need a doegrids certificate
- need to map the public key at RCF side
- need .pem files on the machine where the transfer will be started
- run grid-proxy-init
- use globus-url-copy or srmcp to do the transfer
source /vdt/setup.csh grid-proxy-init globus-url-copy -vb -p 10 file:///ccj/w/data05a/yokkaich/temp2GB.dat gsiftp://phnxgftp.rcf.bnl.gov/pnfs/rcf.bnl.gov/phenix/user/rind/ccjtest3.dat.1
set X509_USER_PROXY to a file so that other processes can use the proxy
http://polzope.in2p3.fr:8081/PHENIX/computing/didacticiel-srm-dcache
https://www.racf.bnl.gov/Facility/dcache/globus_url_copy.html
common tasks
- check disk space on RSCC nodes
- check disk space on data36/ccjbox5,6,7,8
- check job time, should be ~2h for a 2 GB PRDFF
data production
- make prdf file list - make_prdf_lists.csh
- populate db with prdf filenames for reconstruction jobs - db-prepare.pl
setup for reco
- set pro build
- set some env variables
- copy all macros
- run luxor linker
- set PRDFNAME
setenv PRDFNAME EVENTDATA_P00-0000256450-0000.PRDFF
unsetenv OFFLINE_MAIN
unsetenv ONLINE_MAIN
unsetenv ROOTSYS
source /opt/phenix/bin/phenix_setup.csh pro.82
setenv PGUSER phnxrc
setenv ODBCINI `ls -l /opt/ccj/etc/production/odbc/odbc.ini.ccj.prod.p | awk '{print ($11)}'`
setenv PGHOST ccjams001.riken.go.jp
setenv PGPORT 5432
/afs/rhic/phenix/software/calibration/data/LuxorLinker.pl 11
cp -f ~/ccjDP/run8pp_v01/production_macro/* .
all nodes:
TOP (PHCompositeNode)/
DCM (PHCompositeNode)/
dDchDCM (PHIODataNode)
DST (PHCompositeNode)/
EventHeader (PHIODataNode)
Sync (PHIODataNode)
TrigLvl1 (PHIODataNode)
L2Decision (PHIODataNode)
Lvl2OutArray (PHIODataNode)
PreviousEvent (PHIODataNode)
BbcRaw (PHIODataNode)
BbcOut (PHIODataNode)
ZdcRaw (PHIODataNode)
ZdcOut (PHIODataNode)
SmdOut (PHIODataNode)
ErtOut (PHIODataNode)
fclRawSouth (PHIODataNode)
fclRawNorth (PHIODataNode)
fclOutSouth (PHIODataNode)
fclOutNorth (PHIODataNode)
T0Out (PHIODataNode)
dPc1Raw (PHIODataNode)
dPc2Raw (PHIODataNode)
dPc3Raw (PHIODataNode)
dPc1Cluster (PHIODataNode)
dPc2Cluster (PHIODataNode)
dPc3Cluster (PHIODataNode)
Pc1Raw (PHIODataNode)
Pc2Raw (PHIODataNode)
Pc3Raw (PHIODataNode)
Pc1Cluster (PHIODataNode)
Pc2Cluster (PHIODataNode)
Pc3Cluster (PHIODataNode)
MpcRaw (PHIODataNode)
TecOutShort (PHIODataNode)
TecOut (PHIODataNode)
TecHitOut (PHIODataNode)
dTofReconstructed (PHIODataNode)
TofOut (PHIODataNode)
DchHitLineTable (PHIODataNode)
DchTrack (PHIODataNode)
dCrkHit (PHIODataNode)
CrkHit (PHIODataNode)
AccCluster (PHIODataNode)
PHCentralTrack (PHIODataNode)
PHMuoTracksOO (PHIODataNode)
SpinDataEventOut (PHIODataNode)
HardPhCglList (PHIODataNode)
HardPhPhotonList (PHIODataNode)
VtxOut (PHIODataNode)
AccRaw (PHIODataNode)
AccHit (PHIODataNode)
TofwRaw (PHIODataNode)
TofwHit (PHIODataNode)
emcTowerContainer (PHIODataNode)
emcClusterContainer (PHIODataNode)
CglTrack (PHIODataNode)
CglTrackBack (PHIODataNode)
PHTrackOut (PHIODataNode)
PHTrackOutBack (PHIODataNode)
PHDchTrackOut (PHIODataNode)
CrkRing (PHIODataNode)
CrkRingBack (PHIODataNode)
TMutHit (PHIODataNode)
TMuiHitO (PHIODataNode)
TMuiRoadO (PHIODataNode)
TMui1DRoadO (PHIODataNode)
TMuiClusterO (PHIODataNode)
TMuiPseudoLL1 (PHIODataNode)
TMutClus (PHIODataNode)
TMutCoord (PHIODataNode)
TMutGapCoord (PHIODataNode)
TMutStub (PHIODataNode)
TMutTrk (PHIODataNode)
TMutVtx (PHIODataNode)
lpcRaw (PHIODataNode)
PHGlobal (PHIODataNode)
PHGlobal_CENTRAL (PHIODataNode)
PHGlobal_MUON (PHIODataNode)
EWGCentralTrack (PHIODataNode)
RUN (PHCompositeNode)/
RunHeader (PHIODataNode)
PdbCalBankSave (PHIODataNode)
TrigRunLvl1 (PHIODataNode)
TrigRunLvl2 (PHIODataNode)
Flags (PHIODataNode)
DetectorGeometry (PHIODataNode)
MWGCuts (PHIODataNode)
PAR (PHCompositeNode)/
dBbcRawHitPar (PHIODataNode)
BbcCalibPar (PHDataNode)
mPadDetGeo (PHDataNode)
TecGeometry (PHIODataNode)
TecCalibration (PHDataNode)
TofGeometry (PHDataNode)
dDchRecoPar (PHIODataNode)
dDchUnpackPar (PHIODataNode)
dCrkRawHitPar (PHIODataNode)
dCrkCal (PHIODataNode)
AccGeometry (PHDataNode)
dDchFastSimPar (PHIODataNode)
dDchDCMPar (PHIODataNode)
dDchEvalPar (PHIODataNode)
dDchPerfPar (PHIODataNode)
TofwGeometry (PHDataNode)
mEmcGeometry (PHIODataNode)
cglDetectorGeo (PHDataNode)
PRDF (PHDataNode)
BBC (PHCompositeNode)/
dBbcRaw (PHIODataNode)
dBbcOut (PHIODataNode)
ZDC (PHCompositeNode)/
PAD (PHCompositeNode)/
dPc1RawClus (PHIODataNode)
dPc2RawClus (PHIODataNode)
dPc3RawClus (PHIODataNode)
TEC (PHCompositeNode)/
TOF (PHCompositeNode)/
dTofRaw (PHIODataNode)
dTofRawRec (PHIODataNode)
DCH (PHCompositeNode)/
dDchHit (PHIODataNode)
dDchTracks (PHIODataNode)
dDchFEM (PHIODataNode)
dDchRaw (PHIODataNode)
dDchGhitRaw (PHIODataNode)
dDchGhitHits (PHIODataNode)
dDchTracksExt (PHIODataNode)
dDchTracksExtPerf (PHIODataNode)
dDchTracksPerf (PHIODataNode)
DchRawTablev1 (PHIODataNode)
DchHitLineTablev1 (PHIODataNode)
CRK (PHCompositeNode)/
dCrkRaw (PHIODataNode)
dCrkRawReCal (PHIODataNode)
DCHDO (PHCompositeNode)/
DchHisto (PHDataNode)
DchDAO (PHDataNode)
DchDGO (PHDataNode)
DchDCO (PHDataNode)
DchDNO (PHDataNode)
DchEvent (PHDataNode)
DCHTMP (PHCompositeNode)/
DchCandidate (PHDataNode)
PerfectDchTrackCandidate (PHDataNode)
DchTrackCandidate (PHDataNode)
DchRawInfo (PHDataNode)
DchHitInfo (PHDataNode)
DchTrackInfo (PHDataNode)
DchTrackBestInfo (PHDataNode)
DchHitLineLists (PHDataNode)
DchPc1HitLists (PHDataNode)
EVA (PHCompositeNode)/
dDchNibbleGhit (PHIODataNode)
dcghit (PHIODataNode)
fkin (PHIODataNode)
EMC (PHCompositeNode)/
CGL (PHCompositeNode)/
dPHDchTrack (PHIODataNode)
MUTOO (PHCompositeNode)/
TMutHitMap (PHDataNode)
mMutUnpackPar (PHDataNode)
mMutZeroSupPar (PHDataNode)
mMutCalibratePar (PHDataNode)
TMutClusMap (PHDataNode)
TMutCoordMap (PHDataNode)
TMutGapCoordMap (PHDataNode)
TMutStubMap (PHDataNode)
TMutTrkMap (PHDataNode)
TMutVtxMap (PHDataNode)
mMutMuiRoadPar (PHDataNode)
mMutFindClusPar (PHDataNode)
mMutFitClusPar (PHDataNode)
mMutMatchCoordPar (PHDataNode)
mMutStubFitPar (PHDataNode)
mMutFindTrackPar (PHDataNode)
mMutKalFitPar (PHDataNode)
mMutRejectTrackPar (PHDataNode)
mMutBPFitPar (PHDataNode)
mMutFindVtxPar (PHDataNode)
mMutBPVertexPar (PHDataNode)
mMutFitVtxPar (PHDataNode)
MUIOO (PHCompositeNode)/
TMuiHitMapO (PHDataNode)
mMuiRawUnpackPar (PHDataNode)
TMuiRoadMapO (PHDataNode)
TMui1DRoadMapO (PHDataNode)
TMuiClusterMapO (PHDataNode)
TMuiPseudoLL1Map (PHDataNode)
mMuiClusterFinderPar (PHDataNode)
mMuiRoadFinder1Par (PHDataNode)
mMuiFastRoadFinderPar (PHDataNode)
mMuiFindRoadPar (PHDataNode)
mMuiBLTEmulatorPar (PHDataNode)
All triggers:
(MUIDLL1_N1D&MUIDLL1_S1D)&BBCLL1 (MUIDLL1_N1D|MUIDLL1_S1D)&BBCLL1 (MUIDLL1_N1H|MUIDLL1_S1H)&BBCLL1 BBCLL1&(ZDCN|ZDCS) BBCLL1(>0 tubes) BBCLL1(noVertexCut) Clock ERTLL1_2x2&BBCLL1 ERTLL1_4x4a&BBCLL1 ERTLL1_4x4b ERTLL1_4x4b&BBCLL1 ERTLL1_4x4c&BBCLL1 ERTLL1_E ERTLL1_E&BBCLL1 Laser(ZDC) MPC_4x4A MPC_4x4B MPC_4x4C&(ERTLL1_4x4c|ERTLL1_4x4a) MPC_4x4C&(ERTLL1_4x4c|ERTLL1_4x4c) MUIDLL1_N1D&ERTLL1_2x2 MUIDLL1_N1H&(ERTLL1_4x4c|ERTLL1_4x4a)&BBCLL1 MUIDLL1_N2D&BBCLL1 MUIDLL1_N2D|MUIDLL1_S2D MUIDLL1_S1D&ERTLL1_2x2 MUIDLL1_S1H&(ERTLL1_4x4c|ERTLL1_4x4a)&BBCLL1 MUIDLL1_S2D&BBCLL1 Noise PPG(Laser) PPG(Pedestal) PPG(Test Pulse) RxNP ZDCLL1narrow ZDCLL1wide ZDCN||ZDCS
NFS disks at CCJ
ccjnfs10
/ccj/w/data23 (/dev/dsk/c2t0d0s2 ):133559776 blocks 2086856 files
/ccj/w/data24 (/dev/dsk/c3t3d0s2 ):636433296 blocks 9944253 files
/ccj/w/data25 (/dev/dsk/c10t0d0s2):35766784 blocks 558831 files
/ccj/w/data26 (/dev/dsk/c11t3d0s2): 7608576 blocks 118855 files
/ccj/w/data27 (/dev/dsk/c14t0d0s2):314431744 blocks 4912982 files
/ccj/w/data28 (/dev/dsk/c15t3d0s2):864271504 blocks 13504214 files
/ccj/w/data30 (/dev/dsk/c17t3d0s2):1659218208 blocks 25925260 files
ccjnfs11
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/ndst_vg-ndst_lv
8.9T 8.7T 179G 99% /ccj/w/data40
/dev/sdb1 1.7T 1.4T 261G 85% /ccj/w/data41
/dev/sdc1 1.7T 1.5T 225G 87% /ccj/w/data42
/dev/sdd1 1.7T 13M 1.7T 1% /ccj/w/data43
/dev/sde1 1.7T 13M 1.7T 1% /ccj/w/data44
ccjnfs12
/dev/sdc1 1.7T 35M 1.7T 1% /ccj/w/data45
/dev/sdd1 1.7T 244M 1.7T 1% /ccj/w/data46
/dev/sde1 1.7T 360G 1.3T 22% /ccj/w/data47
/dev/sdf1 1.7T 1.5T 168G 90% /ccj/w/data48
/dev/sdg1 1.7T 1.3T 347G 80% /ccj/w/data49
/dev/sdh1 1.7T 1.5T 192G 89% /ccj/w/data50
ccjnfs13
Filesystem Size Used Avail Use% Mounted on
/dev/sdc1 1.7T 1.2T 535G 69% /ccj/w/data51
/dev/sdd1 1.7T 1.4T 278G 84% /ccj/w/data52
/dev/sde1 1.7T 1.3T 349G 80% /ccj/w/data53
/dev/sdf1 1.7T 1.5T 151G 92% /ccj/w/data54
/dev/sdg1 1.7T 324G 1.4T 20% /ccj/w/data55
/dev/sdh1 1.7T 359G 1.3T 22% /ccj/w/data56
/dev/sdc1 1.7T 1.2T 535G 69% /ccj/w/data51
/dev/sdd1 1.7T 1.4T 278G 84% /ccj/w/data52
ccjnfs14
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/ndst_vg1-ndst_lv1
8.9T 8.5T 441G 96% /ccj/w/data31
/dev/mapper/ndst_vg2-ndst_lv2
8.9T 8.0T 913G 90% /ccj/w/data32
ccjnfs15
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/ndst_vg5-ndst_lv5
8.9T 6.8T 2.2T 77% /ccj/w/data35
/dev/mapper/ndst_vg6-ndst_lv6
8.9T 6.6T 2.4T 74% /ccj/w/data36
help message from HPSS
HPSS File and Directory Commands -------------------------------- get, cget, mget, recv - Copy HPSS file to local directory put, cput, mput, replace, save, store, send - Copy a local file to HPSS cp, copy - Copy a file within HPSS delete, mdelete, erase, rm - Remove a file from HPSS ls, list - List directory pwd - Print current directory find - Traverse a directory tree looking for a file ln - Create symbolic link in HPSS mv, move, rename - Rename an HPSS file mkdir, md, add - Create an HPSS directory rmdir, rd, remove - Delete an HPSS directory cd, cdls - Change current directory touch - Update last access time Local File and Directory Commands --------------------------------- lcd, lcdls - Change local directory lls - List local directory lpwd - Print current local directory !<command> - Issue shell command File Administrative Information ------------------------------- chgrp - Change group id of file or directory chmod - Change permissions of file or directory chown - Change owner of file or directory umask - Set file creation permission mask Storage Hiearacy Commands ------------------------- lscos - List available Classes of Service Keywords -------- set, default - Change keyword value COSLIST - COS list name to use for subsequently created files DIRn - Set working directory TALK - Turns on/off verbose mode BACKUP - Keep a copy of overwritten files Saved Keysets ------------- adopt - Load a keyset free - Delete a keyset keep - Save a keyset show - Display a keyset HSI commands ----------- help - Display help file quit, exit, end - Terminate HSI glob - Toggle wildcard expansion hash - Toggle file transfer hashmarks in - Read commands from a local file out - Write HSI output to a local file log - Write all HSI commands and responses to a local log file bell - Toggles terminal bell prompt - Toggles HSI prompting for mget, mput, and mdelete HSI is a command driven interface to the HPSS Mass Storage System. HSI can accept input several different ways: From a command line. Example: hsi Single line execution. Example: hsi "mkdir foo; cd foo; put data_file" From a command file. Example: hsi "in command_file" HSI can read from standard input and write to standard output. Command line example: tar cvf - . | hsi put - : data.tar hsi get - : data.tar | tar xvf - Interactive example: put "|tar cvf - *.c" : data.tar get "|tar xvf -" : data.tar WARNING: For 'get' and 'put' operations, HSI uses a different syntax than FTP to identify the local file name. The syntax uses a ':' to identify the local file name. See section 5.8 of the online documentation for details. Example: put local_file : hpss_file Recursive operations are allowed for the following commands: cget, chgrp, chmod, chown, cput, delete, get, ls, mdelete, mget, migrate, mput, purge, put, rm, stage, touch Wildcards are supported. See section 5.9 of the online documentation. The complete HSI manual is online at http://www.sdsc.edu/Storage/hsi
http://www.nersc.gov/nusers/resources/HPSS/hsi/Doc/
http://www.sdsc.edu/us/resources/hpss/hsi.html
HPSS User's guide
http://www.hpss-collaboration.org/hpss/users/user_doc.jsp
dCache
the dCache book: http://www.dcache.org/manuals/Book/
