Download files and directories using iBridges#

This sections shows how you can download stuff from iRODS using iBridges. These are just a few examples on how you could use it within Python.

For the full documentation and usage go to:
https://ibridges.readthedocs.io/en/stable/

git repository:
https://github.com/UtrechtUniversity/iBridges/

Authentication#

All iRODS clients (icommands and APIs) expect the above parameters to be stored in a special folder. This folder is called .irods and it lies in your home directory:

Mac: /Users/<user>/.irods
Linux: /home/<user>/.irods
Windows: C:\Users\<user>\.irods

You can store the irods_environment.json in that folder and make sure that its extension is json.

Again under Windows the text editors usually save files with the .txt extension. So please watch out for this. Below we provide a code snippet which saves your personal UNLOCK iRODS information in the right place.

# Set irods environment location and username

irods_env_dir = "~/.irods"
irods_env_file = "irods_environment.json"
username = <SRAM username>

If you already have an environment file in place, you can skip the next cell and go to “Start a session”

from pathlib import Path
import json

# CREATE above defined irods environment directory if not does exist yet
irods_env_dir = Path.expanduser(Path(irods_env_dir))
if not irods_env_dir.exists():
    irods_env_dir.mkdir()

# Set irods_environment.json file as save in the .irods folder.
env = {
    "irods_host": "unlock-icat.irods.surfsara.nl",
    "irods_port": 1247,
    "irods_user_name": username,
    "irods_zone_name": "unlock",
    "irods_authentication_scheme": "pam",
    "irods_encryption_algorithm": "AES-256-CBC",
    "irods_encryption_key_size": 32,
    "irods_encryption_num_hash_rounds": 16,
    "irods_encryption_salt_size": 8,
    "irods_client_server_policy": "CS_NEG_REQUIRE",
    "irods_client_server_negotiation": "request_server_negotiation"
}

env_file = Path.expanduser(Path(irods_env_dir)).joinpath("irods_environment.json")
with open(env_file, 'w') as write_json:
    #json.dump(env, write_json,indent=2)
    json.dump(env, write_json,indent=2)

if Path.is_file(env_file):
    print("Created environment file at", env_file)
else:
    print("Failed to created environment file at", env_file)

Start a session! It will ask you for the SRAM token#

from pathlib import Path
from ibridges import Session
from getpass import getpass

env_loc = irods_env_dir+"/"+irods_env_file
env_file = Path.expanduser(Path(env_loc))

password = getpass()
session = Session(env_file, password=password)

if session:
    print("Session succesfully established")

Downloading files#

investigation = <investigation>
study = <study>

Create a local download directory#

from ibridges import IrodsPath

# Define where to download files locally
download_path = "./unlock_downloads/"+investigation+"/"+study 

# Create the directory if it doesn't exist yet
download_dir = Path.expanduser(Path(download_path))
download_dir.mkdir( parents=True, exist_ok=True )

Download a single file or directory#

Use the full iRODS path

You will receive a dictionary with changes, which you can also retrieve beforehand with the option dry_run=True.
Existing local data will not be overwritten. Please use the option overwrite=True if you want to overwrite your local data

from ibridges import download

irods_file = Path("/unlock/home/wur.fdp/stu_bmock12_prjna496047/obs_bmock12_mocktest_cwl/sam_bmock12_synthetic_metagenome/metagenomic_other_illumina/asy_illumina_srr8073716/data/SRR8073716_1.fastq.gz")
download(session, irods_file, download_dir)

irods_dir = Path("/unlock/home/wur.fdp/stu_bmock12_prjna496047/obs_bmock12_mocktest_cwl/sam_bmock12_synthetic_metagenome/metagenomic_other_illumina/asy_illumina_srr8073716")
download(session, irods_dir, download_dir)