Using cyclone

Connection and first steps

Use ssh to connect

ssh -Y <username>@cyclone.hpc.uib.no

or just simply:

ssh -Y cyclone.hpc.uib.no

if you're connecting from a computer where you are logged in under the same name. When you are connecting for the first time, you will be asked to verify a signature. Reply "yes" to this question.

Home directories

cyclone.hpc.uib.no shares home directories with the new hexagon cluster. These home directories are separate from your standard UiB home directory. The contents of your home directories will not be synced, so you will manually need to transfer your scripts to the new system.

How to access software

In contrast to the old skd-cyclone, most scientific software is only available through software modules. Modules need to be loaded manually before the respective software can be used. We use software modules, because they provide are a clean and handy way to offer conflicting software or software versions: Instead of updating software and potentially breaking user scripts and programs new software is then made available in parallel to those that require it.

Start-up scripts

It might be handy to automatically load modules for software that you use frequently every time you login to cyclone. To accomplish that for bash, there are two relevant files, $HOME/.bashrc and $HOME/.bash_profile . .bash_profile should contain things that need to be defined at login time only, like PATH and other environment variables, startup programs, etc. You just need things once, not in every shell you open. In most cases, you also need the things from .bashrc in your login shell. That's why .bash_profile sources .bashrc as well, but .bashrc doesn't usually source .bash_profile. In contrast, .bashrc should typically contain things you want to set in every shell you open, like aliases, functions, etc. These are per shell session items that are not inherited from environment (source and detailed explanation).

An example .bash_profile for inspiration

module load cdo/1.6.9
module load ncarg/6.3.0-login

alias rm="rm -i"
alias cp="cp -i"
alias erai="cd /Data/skd/share/Reanalysis/ERA_INTERIM/6HOURLY/"

if [ -f ~/.bashrc ]; then
   . ~/.bashrc
fi

An example .bash_profile for inspiration

alias rm="rm -i"
alias cp="cp -i"
alias erai="cd /Data/skd/share/Reanalysis/ERA_INTERIM/6HOURLY/"

Available software

Below is a list of frequently used software and which module to load on the new cyclone. Relevant module names are printed like for matlab/2015b. Modules can be loaded by module load <module name>, and unloaded module unload <module name>. More details on the module system.

To list all available modules, use the command

module avail

Please be aware that some of these modules will be specific to hexagon and it's cray architecture. Hence module names containing "cray" might not work on cyclone, although some exceptions are known to exist, see netcdf and ncview. Vice-versa, if the module name contains "foss", the module will be specific to cyclone and might not work on hexagon.

Data analysis and plotting

• CDO

  cdo, cdo/1.6.2

• Dynlib

  See temporary solution for python. Requires in addition OpenBLAS/0.3.1-GCC-7.3.0-2.30 and ScaLAPACK/2.0.2-gompi-2018b-OpenBLAS-0.3.1.

• Matlab

  matlab, matlab/2015b

• NCL

  ncarg, ncarg/6.0.0-login

• nco

  nco

• ncview

  ncview, based on the cray-netcdf package.

• python, plus selection of scientific packages

  Temporary solution, use one of the two below commands to enable the virtual environment:

  source /Data/gfi/users/local/share/virtualenv/dynpie2/bin/activate for python 2.7.

  source /Data/gfi/users/local/share/virtualenv/dynpie2/bin/activate for python 3.4.

  For the pygrib module, grib_api/1.24.0-foss-2018b is required in addition.

• R

  R, R/3.2.5

Text editors

• emacs

  emacs

• gedit

  <no module required>

• gvim

  <no module required>

• vi(m)

  <no module required>

Compilers and libraries

• blas

  OpenBLAS/0.3.1-GCC-7.3.0-2.30

• gcc, gfortran

  GCC, GCC/7.3.0-2.30. Note the upper-case module name, there is also a small-case gcc.

• grib_api

  grib_api/1.24.0-foss-2018b

• lapack

  ScaLAPACK/2.0.2-gompi-2018b-OpenBLAS-0.3.1

• netcdf

  netCDF libraries installed in the folder /opt/cray/netcdf/4.4.1.1/GNU/5.1. You'll hence need to include -I/opt/cray/netcdf/4.4.1.1/GNU/5.1/include -L/opt/cray/netcdf/4.4.1.1/GNU/5.1/lib in your compile command.

• HDF5 libraries

  module load HDF5/1.10.2-foss-2018b

Accessing the data from your own machines

Note: From your personal machines you will need to be connected to the UiB VPN to be able to mount the following network drives.

On MacOS you can mount folders through Finder > Go > Connect to Server .... In the dialogue window that pops up, enter

smb://<user name>@leo.hpc.uib.no/gfi

as the Server Address. You might want to replace gfi by skd to access the skd disks if that is your primary affiliation.