Grid Access

From HLRS Dgrid
(Redirected from Grid Technology)
Jump to: navigation, search

bwGRiD Portal

If you don't like the command line, if you don't want to install much additional software on your computer, or just want to try out the bwgrid, you might want to try out our bwGRiD-Portal at portal.bwgrid.de You need a Grid User Certificate in your Firefox and you must have joined the bwGRiD VO before. You also need a grid-proxy, which you can create using the grid-proxy plugin (a Java Applet) offered inside the portal. Note: The grid-proxy manager plugin for firefox does not work anymore with up-to-date versions of firefox.

Globus Toolkit Version 4 (GT4) for Linux only

The Globus Toolkit (GT) is an open source software toolkit used for building Grid systems and applications. D-Grid offers Globus and also an Overview about the features. Check the D-Grid Access section for instructions how to access D-Grid resources using Globus.

DFN Certificate

Get a DFN Certificate and join a virtual organization (VO) following the steps described in the section Access prerequisites. Export/Backup your certificate from the browser into a .p12 file. This file can be converted to *.pem using the following commands:

user@host:~> openssl pkcs12 -nocerts -in DFNcertificate.p12 -out userkey.pem
user@host:~> openssl pkcs12 -clcerts -nokeys -in DFNcertificate.p12 -out usercert.pem

Those files containing user certificate and private key have to be moved to the folder ~/.globus and given proper access rights:

user@host:~> mkdir -p ~/.globus
user@host:~> mv usercert.pem ~/.globus/
user@host:~> mv userkey.pem ~/.globus/
user@host:~> chmod 700 ~/.globus
user@host:~> chmod 600 ~/.globus/*

When using Windows, place the file in a subfolder ".globus" below your user directory (i.e. "C:\Documents and Settings\{username}\.globus" or "C:\users\{username}\.globus" for windows 7 or "C:\benutzer\{username}\.globus" in the German version). A version of OpenSSL for Windows can be downloaded from http://www.slproweb.com/products/Win32OpenSSL.html. OpenSSL is then installed as "C:\OpenSSL-Win32\bin\openssl.exe" or alike)

You might also have to integrate the CA certificates.

Precompiled bundles of gsissh

The most important tools like grid-proxy-init, gsissh and gsiscp are provided as precompiled bundles at the following URL: http://www.lrz.de/services/compute/grid_en/software_en/gsissh_static_en

GSI OpenSSH

NCSA provides a version of OpenSSH which contains a patch to add GSI support: http://grid.ncsa.illinois.edu/ssh/

SecureNetTerm

InterSoftInternational offeres a windows client called SecureNetTerm http://www.securenetterm.com/

Installation of a GT4 client system

The GT download page offers binary installers for several Linux distributions as well as the source code or a Java version.

Important: Globus Toolkit Version 5 is currently NOT supported by our grid frontend server! We recommend to install Globus Toolkit Version 4.0.8! However, if you only want to use gsissh, grid-ftp and the grid proxy - these components turned out to be (at least partly) compatible between the different major releases of globus toolkit.

Linux in general

The binary installers from the GT download page require a java compiler and the install tool ant. If your distribution is not available, it might work to choose a similar one, e.g. for Ubuntu the Debian installer (we assume the tgz-file is in the $HOME directory:

sudo apt-get install ant
sudo apt-get install openjdk-6-jdk
export JAVA_HOME=/usr/lib/jvm/java-6-opendjk
export PATH=$JAVA_HOME/bin:$PATH
tar xzvf gt4.08.-x86_deb_3.1-installer.tar.gz
cd gt4.08.-x86_deb_3.1-installer 
./configure --prefix=/usr/local
make
sudo make install
cd
export GLOBUS_LOCATION=/usr/local

... and add the export statements and a statement

source $GLOBUS_LOCATION/etc/globus-user-env.sh

e.g. to your .bashrc or .profile (the steps are very similar to compiling from source, except that most files are precompiled and contained in the installer.

Note: the gsi-enabled ssh also assumes that globus toolkit is installed, but this way the OpenSSH can be updated inside an already existing GT installation.

installing GT4 from source

This is quite similar to the procedure above with the binary installer, but all the packages are really compiled on your specific platform, so this should always work (provided that the necessary compilers, Java etc. are installed). Get the sources from http://www.globus.org/toolkit/ , install the program ant (e.g. sudo apt-get install ant, set up your $JAVA_HOME such that javac from your $PATH is contained under that path, e.g. by adding something like

export JAVA_HOME=/usr/lib/jvm/java-6-sun-1.6.0.15
export PATH=/usr/lib/jvm/java-6-sun-1.6.0.15/bin:$PATH

then, run configure; make; make install(this will take quite long...) If this works all fine up to here and upon callling gsissh you get errors like gsissh: error while loading shared libraries: libglobus_gss_assist_gcc64dbg.so.0: cannot open shared object file: No such file or directory you have to adjust your $LD_LIBRARY_PATH according to your install directory of GT4. finally set $GLOBUS_LOCATION in your environment, call source $GLOBUS_LOCATION/etc/globus-user-env.sh, then follow the instructions in the Manual Accessing with Globus.

installing via VDT

Another way to install a GT4 client on a supported Linux platform is achieved through the Virtual Data Toolkit (VDT) and a installer program called Pacman. Unpack the packman-bundle to some place where you would like to install it. cd out of the directory and run it once which will call the setup script. The "-pretend-platform" option can help you to persuade pacman to install a similar, but also working distribution (e.g. debian for Ubuntu). Check the error output of a failed installation attempt to detect the supported platforms and pacman -help for some more information about the installer. Consider also to install the "GSIOpenSSH" package.

There's also material from a Globus-Workshop held at the LRZ.

GSI-SSHTerm (Java) for Windows, Linux, etc

The GSI-SSHTerm Application package is an application written in Java that enables to you connect to a server running GSISSH using your grid certificate. There are multiple ways running the software:

Check the Java system policy file ({java.home}/lib/security/java.policy) and the Java user policy file in the home directory to grant the application sufficient Java security manager permissions.

For some Java installations you will be requested to download the Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files from www.oracle.com/technetwork/java/javase/downloads/index.html. Unpack the files and copy them into the $JAVA_HOME/lib/security.

Quite a nice documentation about GSI-SSHTerm is provided by LRZ: http://www.lrz.de/services/compute/grid_en/software_en/gsisshterm_en/

Windows additional tools

There's also a GridFTP.NET and GRAM.NET. GSISSHTerm (see above) works for Windows as well, an there is also a version of Putty available at http://ngs.ac.uk/research-and-development/gsi-putty

Other Issues

Firewall issues

The following outgoing TCP ports have to be opened, depending on the services you want to use:

  • 8443 (WS GRAM)
  • 2119 (GSI Gatekeeper)
  • 2811, 20000-25000 (GridFTP and also WS GRAM)
  • 2222 (gsissh)
  • 7512 (grid-proxy)
  • 8080, 8443 (Portal)

The following environment variables have to be set (e.g. within .bashrc)

  • GLOBUS_TCP_PORT_RANGE=25000

The hosts to which you might want to connect are

Frontends

grid01.hs-esslingen.de 
globus.bfg.uni-freiburg.de
bwgrid-cluster.scc.kit.edu
gtbw.grid.uni-mannheim.de
gridway.dgrid.hlrs.de
gt4.uni-tuebingen.de
koios.rz.uni-ulm.de

Storage

bwgrid-se.scc.kit.edu (has 6 IPs)

Portal

bwgp.rz.uni-ulm.de

for VO registration/information

dgrid-vomrs.fz-juelich.de Port 8443
dispatch.fz-juelich.de Port 8814


You can read more about DGrid firewall requirements for GT4 (in german) at http://www.d-grid.de/fileadmin/dgrid_document/Dokumente/FG3-5_Empfehlungen_Statischer_Firewalls_v121.pdf

Testing

After initializing the system with

source /opt/vdt/setup.sh
grid-proxy-init

(it is assumed here that vdt is installed at /opt and a bash is used; there's also a "setup.csh" suited for a C shell) it should be possible to run the first jobs like

globusrun-ws -submit -factory <hostname> -passive -streaming -print-fault-type -job-command /usr/bin/id

or

globusrun-ws -submit -F <hostname> -Ft PBS -passive -s -pft -c /bin/hostname

should work (the latter using a Batch job to send the command to a cluster node). The "-debug" option will give you some more information of what is happening and might be used, if the command doesn't work. Note that the "old GT2 commands" like globusrun use services (e.g. jobmanager), which are NOT installed on gridway.dgrid.hlrs.de.

Commands like id uncover your identity. (you might also use whoami) The logins look like stXXYYYY with st for Stuttgart, XX for your D-Grid communitiy (e.g. in for InGrid) and a number YYYY. Knowing your username and GSIOpenSSH installed, you also can login via this certificate based ssh:

gsissh -p 2222 gridway.dgrid.hlrs.de
exit

Note that you should always work on a node, not on the gatekeeper itself.

Working on the D-Grid Cluster using GT4

Prepare your environment

To prepare your working environment, you might first login again and get a node for an one hour interactive session afterwards using the Batch System:

gsissh -p 2222 gridway.dgrid.hlrs.de
qsub -I -l nodes=1:dgrid,walltime=1:00:00
module avail

"module avail" shows installed packages, which can be added with "module load" (it is assumed that you have already initialized the system (see grid-proxy-init above))

There's also a certificate based version of scp named gsiscp. But instead of "pushing" your files to the dgrid resource, you might also "pull" them from your site using an ordinary scp on <hostname>. You can quit the interactive session (earlier than after one hour) with exit, as well as the gsissh session.

Submitting jobs

Submitting a job with the job description needs a XML file, which contains the environment specifications and the commands to execute.

File Tansfer

To transfer files between different hosts in the grid, you can use globus-url-copy <src> <dest>, where files can be local, or gsiftp URLs.

 globus-url-copy gsiftp://<hostname>/~/flow.info_0000 file:/tmp/flow


D-Grid Globus Infrastructure

The WebMDS shows ressource providers and also a D-Grid and InGrid ServiceGroup Overview.