iCommands

iCommands#

This section shows how to download files and directories from the UNLOCK FDP using iCommands. iCommands is the oldest and most feature-rich of the iRODS-based access methods, but it requires familiarity with the command-line and can be more challenging to set up - especially from macOS or Windows environments.

For more information on how to use iCommands, refer to Chapter 5 of the iRODS beginner training:
irods/irods_training

GitHub repository:
irods/irods_client_icommands

Installation#

iCommands is only available through a Linux environment. There are ways to access a Linux environment from other operating systems. Select the tab corresponding to your system.

Windows

There are several ways to access iCommands. The recommended method is to use WSL to run a Linux environment from which iCommands can be installed.

Press the `Windows’ button, type PowerShell and run it as administrator.
Use the following command in PowerShell to install WSL:

wsl --install

Open Ubuntu from your Start Menu, or start up WSL in a terminal:

wsl

Set up a username and password. To install iCommands, follow the instructions in the Linux tab under the Ubuntu section.

Other options are to use another Virtual Machine or a containerized application, please follow the corresponding section below for instructions.

Mac

There are several ways to access iCommands: You can use the CyVerse-specific iCommands download:

Download the package file.
Run the installer with right click. It is possible to get a security warning. To bypass this, go to System Preferences, select Security & Privacy menu, and click the Open Anyway button.
Follow the installation prompts.
(Circumstantial) If iCommands do not function, this might be because newer versions of Mac use a different shell (zsh). Edit your .zshrc file and add the following lines at the bottom:

# add iCommands Path
export PATH="/Applications/icommands/:$PATH"
export IRODS_PLUGINS_HOME=/Applications/icommands/plugins/

(Circumstantial) Restart the terminal or use the following command: source ~/.zshrc.

Other options are to use either a Virtual Machine or a containerized application, please follow the corresponding section below for instructions.

Linux

It is possible to install iCommands directly in most Linux distributions:

Ubuntu / Debian

wget -qO - <https://packages.irods.org/irods-signing-key.asc | sudo apt-key add -
echo "deb [arch=amd64] <https://packages.irods.org/apt/> $(lsb_release -sc) main" | sudo tee /etc/apt/sources.list.d/renci-irods.list
sudo apt-get update
sudo apt-get install irods-icommands

Or, for Ubuntu 22.04+ where apt-key is deprecated:

wget -qO - <https://packages.irods.org/irods-signing-key.asc |  gpg --dearmor | sudo tee /usr/share/keyrings/irods-archive-keyring.gpg > /dev/null
echo "deb [signed-by=/usr/share/keyrings/irods-archive-keyring.gpg arch=amd64] https://packages.irods.org/apt/ $(lsb_release -sc) main" | sudo tee /etc/apt/sources.list.d/irods.list
sudo apt-get update
sudo apt-get install irods-icommands

CentOS / RHEL

sudo rpm --import https://packages.irods.org/irods-signing-key.asc
sudo wget -O /etc/yum.repos.d/renci-irods.repo https://packages.irods.org/yum/renci-irods.repo
sudo yum install irods-icommands

Vagrant#

Vagrant is the command-line utility for managing the lifecycle of virtual machines. Isolate dependencies and their configuration within a single disposable and consistent environment. Vagrant is distributed as a binary package for all supported platforms and architectures. You can also compile Vagrant from source, which is covered in the README.

Installation#

Installation instructions for Vagrant is available at:
https://developer.hashicorp.com/vagrant/install
Also install a “virtualization provider” such as VirtualBox:
https://www.virtualbox.org/wiki/Downloads

Initialize a virtual machine#

From the documentation at Vagrant…

The first step to configure any Vagrant project is to create a Vagrantfile. The Vagrantfile allows you to:

Mark the root directory of your project. Many of the configuration options in Vagrant are relative to this root directory.
Describe the kind of machine and resources you need to run your project, as well as what software to install and how you want to access it.

Create a directory

Mac/Linux: mkdir vagrant_getting_started
Windows: Use file explorer to create a folder

Initialize the project

Make sure you are inside your project folder using the command-line interface:

vagrant init generic/ubuntu2204

A `Vagrantfile` has been placed in this directory. You are now
ready to `vagrant up` your first virtual environment! Please read
the comments in the Vagrantfile as well as documentation on
`vagrantup.com` for more information on using Vagrant.

There are many other vagrant images available but for this documentation we stick to a relatively recent ubuntu 22.04 image. Other images can be found at:
https://app.vagrantup.com/

Boot an environment

Now that you have initialized your project and configured a box for it to use, it is time to boot your first Vagrant environment.

vagrant up

Bringing machine 'default' up with 'virtualbox' provider...
==> default: Box 'generic/ubuntu2204' could not be found. Attempting to find and install...
    default: Box Provider: virtualbox
    default: Box Version: >= 0
==> default: Loading metadata for box 'generic/ubuntu2204'
    default: URL: <https://vagrantcloud.com/api/v2/vagrant/generic/ubuntu2204>
==> default: Adding box 'generic/ubuntu2204' (v4.3.12) for provider: virtualbox (amd64)
    default: Downloading: <https://vagrantcloud.com/generic/boxes/ubuntu2204/versions/4.3.12/providers/virtualbox/amd64/vagrant.box>
Progress: 3% (Rate: 12.0M/s, Estimated time remaining: 0:03:08)

==> default: Machine booted and ready!

SSH into the machine

You will not actually see anything though, since Vagrant runs the virtual machine without a UI. To prove that it is running, you can SSH into the virtual machine:

vagrant ssh

vagrant@ubuntu2204:~$

Installing iCommands with Vagrant#

To install iCommands on your virtual machine, follow the above iCommands installation instructions in the Linux tab under the Ubuntu section.

Docker#

Docker is a containerized application that can create environments (including Linux environments).

Installation#

To install Docker, download and install the docker application from:
https://www.docker.com/products/docker-desktop

We host a Docker image with iCommands installed. To spin up this Docker container with you can use the following command:

docker run -it --entrypoint /bin/bash docker-registry.wur.nl/m-unlock/docker/irods:latest

However, because this is a clean Docker instance it does not contain any authentication configuration. You could create the required irods_environment.json file inside the docker instance but upon starting a new instance this information would be lost. Therefore, the best approach would be to exit the docker container and create a folder containing an irods_envrionment.json configuration file (you can also use the .irods folder in your home directory if you have previously created a configuration file).

Create an irods folder
Create an irods_environment.json file within the irods folder with the following content:

{
    "irods_user_name": "<USERNAME>",
    "irods_host": "data.m-unlock.nl",
    "irods_port": 1247,
    "irods_zone_name": "unlock",
    "irods_home": "/",
    "irods_authentication_scheme": "pam_password",
    "irods_encryption_algorithm": "AES-256-CBC",
    "irods_encryption_key_size": 32,
    "irods_encryption_num_hash_rounds": 16,
    "irods_encryption_salt_size": 8,
    "irods_client_server_policy": "CS_NEG_REQUIRE",
    "irods_client_server_negotiation": "request_server_negotiation"
 }

Make sure to fill in your USERNAME in the first field.

Now, when starting the Docker container the folder you’ve created should be mounted to the internal location of ~/irods/. This is achieved via the -v parameter:

docker run -it -v /PATH/TO/YOUR/IRODS/FOLDER:/root/.irods docker-registry.wur.nl/m-unlock/docker/irods:latest

When the docker container has succesfully started you can initialize an iCommands session with the following command, it will prompt you for your SRAM token:

iinit

The iCommands are now available for use. Skip the Authentication header and go to the Usage header for more information. Be mindful that with the above command, your container is not mounted to your local filesystem and any files you will download will end up in the container’s filesystem - which is wiped after stopping the container.

Authentication#

iCommands (like all iRODS clients) requires a configuration file, which ensures that your connection is encrypted and that you do not need to provide all access information each time you connect to the UNLOCK FDP. This file is typically stored in your home directory under a folder called .irods:

OS	Path to `.irods` folder
Windows	`C:\Users\<your-username>\.irods`
macOS	`/Users/<your-username>/.irods`
Linux	`/home/<your-username>/.irods`

Create a file named irods_environment.json in this folder. Make sure that the extension is correct, enter the following information in the file:

{
    "irods_user_name": "<USERNAME>",
    "irods_host": "data.m-unlock.nl",
    "irods_port": 1247,
    "irods_zone_name": "unlock",
    "irods_home": "/",
    "irods_authentication_scheme": "pam_password",
    "irods_encryption_algorithm": "AES-256-CBC",
    "irods_encryption_key_size": 32,
    "irods_encryption_num_hash_rounds": 16,
    "irods_encryption_salt_size": 8,
    "irods_client_server_policy": "CS_NEG_REQUIRE",
    "irods_client_server_negotiation": "request_server_negotiation"
 }

Make sure to fill in your USERNAME in the first field.
When your configuration file is set up you can login to iCommands with the following command, it will prompt you for your SRAM token:

iinit

If your configuration file was not found in the right place, this command will prompt you to fill in the connection information manually to craft a configuration file. This was another way to set up the file but it can be rather error-prone.

Usage#

Use ihelp for an overview of all iCommands. You will find many familiar UNIX commands that work on the remote filesystem such as icd, ipwd, ils, imkdir, icp, imv, etc.

Within the UNLOCK FDP the data structure is very similar to the metadata registration structure. Read access is granted to people that are associated to the project and/or the investigation. The root folder is the landing directory of the UNLOCK FDP, also called a zone. Within this zone, the project folders that are accessible to you can be seen with the following command (add -r flag to see files and folders recursively):

ils /unlock/home

Within this home folder you should find your project folder under wur.YOUR_PROJECT_NAME. In this project folder you will find the original excel sheet that was used for the data registration under a metadata subfolder (after logging in and validation through the FAIR Data Station).
To download a file you can use the iget command which will download the file into your current directory:

iget /unlock/home/wur.YOUR_PROJECT_NAME/some_file_to_download.txt

Conversely, iput will upload a file, for both of these commands the -r flag works to download / upload directories:

iput -r path/to/local/directory

iCommands

Contents

iCommands#

Installation#

Vagrant#

Installation#

Initialize a virtual machine#

Installing iCommands with Vagrant#

Docker#

Installation#

Authentication#

Usage#