GFI&SKD Storage

Organized via UiB ITA network storage (NetApp), GFI&SKD rents a total quota of 450TB disk storage (May 2024).

The storage areas are available from these main folder paths on cyclone (local access) and any other GFI&SKD linux-client connected to UiB std. cable network (network acces via nfs):

- /Data/gfi  and /Data/skd

For GFI&SKD windows-clients and mac-clients inside UiB-network, the storage areas are accessed via one servers and two different paths for GFI and SKD. The paths are like follows and you can mount subfolders of these:

- From Win10, backup folders use:
           \\klient.uib.no\felles\matnat\gfi and \\klient.uib.no\felles\matnat\skd

- From Macos, backup folders use:
              smb://uib.no;userid@klient.uib.no/felles/matnat/gfi (or change tail gfi to skd).
  Please include the entire domain path uib.no rather than uib only also on Macs.

Howto on connecting from Windows/Mac is published from UiB ITA (just change string for correct server and path):

* Win10: How to connect to a network share

* Mac OS X: Connecting to your network share

The storage is organized by priority on larger shared folders for modelling data, in group and projects folders with and without need for backup and folders for individual use with or without backup. Because backup of large quantities are connected with considerable annual cost, users and groups will need to consider which data need a daily backup and which are reproducible and have a reasonably security within a modern storage system without backup.

As a general rule, we recommend backup of all data files that cannot be reproduced by re-executing an analysis script, re-running a model or that can be downloaded from other locations. Examples for data files to be placed in backup folders are scripts for starting a model run, matlab and python scripts for data analysis, manuscripts, and so on. Examples for data files that can remain without backup are results from model runs where scripts are available to re-run, data files downloaded from other servers, and plots that can be recreated by running a script.

Most folders assigned for group access have general read-write access on group level. Owners of each folder and file may still have set exclusive individual access on these folders and files. If other members of the assigned group find this inconvenient, please contact the formal owner of the file or folder.

Individual quotas (inside /Data/gfi/users and /Data/skd/users) are set by default to 200GB and adjusted by request on individual needs - normally < 1TB, but still in respect for other users needs and limited by the max. size of the parent folder. Larger amount should be considered for their own project space. Project and group quotas are managed by configured maximal limits of main project/group folders.

Access on top level folders are often set read-only to keep folder conventions & structure strict and tidy. When in need for storage space and/or adjustments and customization, please submit issue by email to: driftATgfi.uib.no.

Please note that "the work-folder" is meant for short time storage - Any file or folder older than 30 days inside "the work-folder" will be deleted automatically and without further notice.

Sizes, quotas, purpose etc. for the variety of folders inside GFI&SKD storage are given below.

GFI&SKD Storage folders inside /Data/gfi/ - (from NetApp - felles.uib.no)
Subfolder	Purpose	Projid	Size limit	Group assign	User quota	Backup
share	shared model data - era5 included		200 TB	GFI & SKD
vindenergi	group share		30 TB	VINDENERGI
scratch	individual and group		60 TB	GFI
work	shared short time (60days) Symbolic link -> /Data/skd/work		30 TB	GFI & SKD
met	group share		15 TB	MET
spengler	group share		20 TB	SPENGLER
exprec	group share		5 TB	EXPREC
metno	daily&arch. met.no prognosis & obs		1.5 TB	GFI
cordex	group share		7 TB	CORDEX
Total	Size of GFI storage - (from leo.hpc.uib.no)		338.5TB	GFI & SKD

GFI&SKD Storage folders inside /Data/gfi/ - (from UiB SAN - felles.uib.no)
Subfolder	Purpose	Projid	Size limit	Group assign	User quota	Backup
users	individual with backup	None	20 TB	GFI	*	*
projects	various project shares (sum of subfolders)	None	4.25 TB	GFI		*
projects/metdata	project group share	None	2.5 TB	METDATA		*
projects/farlab	project group share	None	1 TB	FARLAB		*
projects/isomet	project group share	None	0.25 TB	ISOMET		*
Total	Size of GFI storage - (from UiB SAN - felles.uib.no)		25.25 TB	GFI & SKD

GFI&SKD Storage folders inside /Data/skd/
Subfolder	Purpose	Projid	Size limit	Group assign	User quota	Backup
share	Symbolic link ->/Data/gfi/share/ - shared model data	48		GFI & SKD
skd-share	Several subfolders /Data/gfi/share/* -> /Data/skd/skd-share/.	66	50 TB	GFI & SKD
scratch	individual and group share	67	20 TB	SKD
work	shared short time (30days)	50	30 TB	GFI & SKD
stormrisk	group share	83	5 TB	STORMRISK
projects	various project shares	49	15 TB	SKD
users	individual with backup - This is from UIB SAN felles.uib.no	None	5 TB	SKD	*	*
Total	Size of SKD storage		125 TB	GFI & SKD

The below is not not correct, we are waiting for ITA for new solution on quota monitoring.

To monitor your quota (on folders from leo.hpc.uib.no), you can run the following command from an ssh login on cyclone:

- lfs quota -hp <projid> /shared/

- example for /Data/gfi/scratch: lfs quota -hp 79 /shared/

To monitor your 30 GB homefolder quota on cyclone, you can run the following command from an ssh login on cyclone:

- lfs quota -hp $(id -u) /shared/

or just

- lfs quota -hp $(id -u) $HOME