iRODS
iRODS (integrated Rule Oriented Data System) is an open source data management software. On the cluster is to used to store and archive data which do not have to be accessed directly at least temporarily. It allows to create collections, files, and add metadata to them.
iRODS access
To have access to iRODS you have to ask for an account to one of the cluster administrators. You will have to create a ~/.irods/irods_environment.json file with a configuration depending on your cluster user name.
Once the access is available, you can initialize iRODS with the following command on pbil-deb:
iRODS usage
In iRODS, a "collection" is equivalent to a folder. Your personal collection should be at /lbbeZone/home/login.
List files and collections
You can list the files and collections on your iRODS account with:
Store local files and folders into iRODS
To store a single local file into an iRODS collection you can use iput:
# Put a file in your root collection
iput README.md
# Put a file in the existing "project_folder" collection
iput README.md project_folder/
To store a local folder recursively as a new iRODS collection, use iput -r:
# Put a folder in your root collection
iput -r my_project
# Put a folder in the "projects" collection
iput -r my_project projects/
Warning
If your local folder is made of many small files, it is recommended to store it as an archive file (tar.gz for example).
Get files or collections from iRODS
To get files or collections back from iRODS, use iget:
# Get a single file from the root collection
iget README.md
# Get a single file from a sub collection
iget project/my_project/README.md
# Get a collection recursively
iget -r project/my_project
Remove files or collections
To remove a file or collection stored on iRODS, use irm:
Removed files and collections are not deleted completely, they are moved to a trash collection. You can the content of your trash with:
To empty the trash, use:
Metadata
It is possible to add metadata to files and collections with imeta add. A metadata is defined by three arguments: an attribute name, a value, and an optional unit.
# Add metadata to a file
imeta add -d collection/README.txt date 2024-12-12
imeta add -d collection/README.txt length 35 lines
# Add metadata to a collection
imeta add -C collection date 2024-12-12
To display files or collections metadata, use imeta ls:
# List a file metadata
imeta ls -d collection/README.txt
# List a collection metadata
imeta ls -C collection/
It is possible to query metadata to find particular files or collections with imeta qu:
# Find files with an attribute `date` with value "2024-12-12"
imeta qu -d data = "2024-12-12"
# Find collections with a length less than 40
imeta qu -d length "n<" 40
Web interface
A web interface is available at:
https://adminlbbe.univ-lyon1.fr/metalnx/
Bash completion
It is possible to add tab completion for iRODS commands in bash:
cd ~
mkdir .bash_completions
curl https://raw.githubusercontent.com/irods/contrib/refs/heads/main/irods_completion.bash -o .bash_completions/irods_completion.bash
echo "source .bash_completions/irods_completion.bash" >> .bashrc
source .bashrc
After that you should be able to use tab completion on iRODS path in commands such as iput, iget or irm.