Using cyclone

Connection and first steps

Use ssh to connect

ssh -Y <username>@cyclone.hpc.uib.no

or just simply:

ssh -Y cyclone.hpc.uib.no

if you're connecting from a computer where you are logged in under the same name. When you are connecting for the first time, you will be asked to verify a signature. Reply "yes" to this question.

Home directories

cyclone.hpc.uib.no shares home directories with the new hexagon cluster. These home directories are separate from your standard UiB home directory. The contents of your home directories will not be synced, so you will manually need to transfer your scripts to the new system.

Please be aware that your local home directory is not expected to exceed 30GB at any time - even in periods when sys.adms. do not practice quotas. If you need to store larger amount of data, please plan for using the GFI&SKD storage - also decribed in this wiki.

To monitor your home directory size, you can use these commands from cyclone command shell:

du -sh ~/

Or to include more details on both dot files/directories and non-hidden home directory subfolders:

cd; du -sh .[^.]* *

Monitoring your homefolder quota, you can run the following command:

- lfs quota -hp $(id -u) /shared/

or just

- lfs quota -hp $(id -u) $HOME

How to access software

In contrast to the old skd-cyclone, most scientific software is only available through software modules. Modules need to be loaded manually before the respective software can be used. We use software modules, because they provide are a clean and handy way to offer conflicting software or software versions: Instead of updating software and potentially breaking user scripts and programs new software is then made available in parallel to those that require it.

Start-up scripts

It might be handy to automatically load modules for software that you use frequently every time you login to cyclone. To accomplish that for bash, there are two relevant files, $HOME/.bashrc and $HOME/.bash_profile . .bash_profile should contain things that need to be defined at login time only, like PATH and other environment variables, startup programs, etc. You just need things once, not in every shell you open. In most cases, you also need the things from .bashrc in your login shell. That's why .bash_profile sources .bashrc as well, but .bashrc doesn't usually source .bash_profile. In contrast, .bashrc should typically contain things you want to set in every shell you open, like aliases, functions, etc. These are per shell session items that are not inherited from environment (source and detailed explanation).

An example .bash_profile for inspiration

module load cdo/1.6.9
module load ncarg/6.3.0-login

alias rm="rm -i"
alias cp="cp -i"
alias erai="cd /Data/skd/share/Reanalysis/ERA_INTERIM/6HOURLY/"

if [ -f ~/.bashrc ]; then
   . ~/.bashrc
fi

An example .bash_profile for inspiration

alias rm="rm -i"
alias cp="cp -i"
alias erai="cd /Data/skd/share/Reanalysis/ERA_INTERIM/6HOURLY/"

Available software

Below is a list of frequently used software and which module to load on the new cyclone. Relevant module names are printed like for matlab/2015b. Modules can be loaded by module load <module name>, and unloaded module unload <module name>. More details on the module system.

To list all available modules, use the command

module avail

Hexagon-specific modules are now no longer listed and available on cyclone. All packages listed with this command should work, so please report to support@hpc.uib.no if they do not.

Data analysis and plotting

• CDO

  CDO, CDO/1.9.5-foss-2018b

• Dynlib

  See temporary solution for python. Requires in addition OpenBLAS/0.3.1-GCC-7.3.0-2.30 and ScaLAPACK/2.0.2-gompi-2018b-OpenBLAS-0.3.1.

• ghostview

  <no module required>

• grads

  <no module required>

• Matlab

  Matlab, Matlab/2018a, Matlab/2012a

• ncdump and other netCDF utilities

  netCDF, netCDF/4.6.1-foss-2018b

• NCL

  NCL, NCL/6.5.0

• nco

  NCO, NCO/4.7.4-foss-2018b

• ncview

  ncview, ncview/2.1.7-foss-2018b. Load only when needed! The package is currently known to break some GUI programs like gedit and gvim.

• python, plus selection of scientific packages

  First choose and load the python module for the python version that you want to use, either Python/2.7.15-foss-2018b for python 2.7.15, or Python/3.7.0-foss-2018b for python 3.7.0.

  Then, as a temporary solution, many scientific python packages are installed in the following virtual environments:

  source /Data/gfi/users/local/share/virtualenv/dynpie2/bin/activate for python 2.7.15.

  source /Data/gfi/users/local/share/virtualenv/dynpie3-2021a/bin/activate for python 3.7.0.

  For the pygrib module, ecCodes/2.7.3-foss-2018b is required in addition.

• R

  <no module required>

Text editors

• emacs

  Is installed globally, but requires util-linux/2.32-GCCcore-7.3.0 to work.

• gedit

  <no module required>

• gvim

  <no module required>

• vi(m)

  <no module required>

Compilers and libraries

• blas

  OpenBLAS/0.3.1-GCC-7.3.0-2.30. Note: The library is called libopenblas, so you'll need to compile with -lopenblas.

• eccodes (replacement for grib_api)

  ecCodes/2.7.3-foss-2018b

• gcc, gfortran

  GCC, GCC/7.3.0-2.30

• grib_api

  grib_api/1.24.0-foss-2018b

• HDF5 libraries

  HDF5/1.10.2-foss-2018b

• lapack

  ScaLAPACK/2.0.2-gompi-2018b-OpenBLAS-0.3.1. Note: The library is called libscalapack, so you'll need to compile with -lscalapack.

• MPI

  OpenMPI/3.1.1-GCC-7.3.0-2.30

• netcdf

  For C-libaries: netCDF/4.6.1-foss-2018b,

  For Fortran-libraries netCDF-Fortran/4.4.4-foss-2018b, compile with option -I/shared/apps/netCDF-Fortran/4.4.4-foss-2018b/include

• proj

  PROJ/5.0.0-foss-2018b

Jupyter & Jupyterhub

Here is a step by step guide to setup and use Jupyter based on the dynpie3 environment.

File:Get started cyclone.pdf

Accessing the data from your own machines

Note: From your personal machines you will need to be connected to the UiB VPN to be able to mount the following network drives.

On MacOS you can mount folders through Finder > Go > Connect to Server .... In the dialogue window that pops up, enter

smb://uib.no;<user name>@leo.hpc.uib.no/gfi

as the Server Address. You might want to replace gfi by skd to access the skd disks if that is your primary affiliation.

On Ubuntu this is very similar: Files > Go > Enter Location .... Enter the address:

smb://uib.no;<user name>@leo.hpc.uib.no/gfi

and enter your password in the next pop-up window, while keeping "UiB" in the "Domain" field.

Alternatively you can mount the network drive with sshfs to a dedicated directory using sshfs (install with sudo apt install sshfs):

mkdir ~/cyclone_gfi
From UiB network: sshfs -o follow_symlinks <user name>@cyclone.hpc.uib.no:/gfi/ ~/cyclone_gfi/
From ext. networks: sshfs -o follow_symlinks <user name>@sync.uib.no:/gfi/ ~/cyclone_gfi/
Unmount with: fusermount -u ~/cyclone_gfi

On Windows you can mount by navigating to File Explorer > Map network drive and enter \\leo.hpc.uib.no\gfi. Make sure to enable Connect using different credentials and fill in <user name>@uib.no and your PW.

Displaying GUIs on your screen

The ssh-login command at the top of this wiki enables trusted X11 forwarding (the -Y option), which allows you to display Graphical User Interfaces (GUIs) on your screen. This way you may be able to run software on a high performance system in a more userfriendly way than by just operating via the terminal. To make this work properly you may have to edit the ssh configuration file (/etc/ssh/ssh_config on linux systems) by adding or uncommenting the following two lines:

X11Forwarding yes
ForwardX11Trusted yes

Editing this file requires superuser priviliges so you may have to ask the IT department for support. For Mac the solution should be pretty similar, windows requires additional software like PuTTY (for establishing an ssh-connection) and an XServer (e.g. VcXsrv).