duma.md

Data Upload Manager - DUMA Quick Guide

{{TOC}}

Overview

DUMA is an IBM Aspera deployment at the LCSB-University of Luxembourg. DUMA supports end-to-end encrypted data transfer and can handle high data volumes e.g. several tera bytes.

Obtaining a DUMA access link

You need an access link to use DUMA. An access link is a temporary, password-protected space, much like a drop box, on LCSB's DUMA server. In order to obtain an access link, you should contact your research collaborator at the LCSB-University of Luxembourg. Once created, you will receive your access link and associated password (link) by e-mail.

IMPORTANT NOTE: Whenever the LCSB shares a password for a DUMA endpoint (collaborator), the password be reachable via a link, which will expire. Therefore you should make a record of your password once you view it.

An access link can be reached via standard web browsers. Data can be transferred to/from an access link in two ways:

• Through the web browser by visiting the link, which is our recommended way of data transfer, described in this section of the guide
• Through the use of a command line tool. If your data sits in an environment, where you can not launch a web browser, then you may use use a command line client tool to reach an access link. This process is described in this section of this guide.

The use of DUMA is mediated by LCSB's data stewards. If you require assistance in using DUMA, you should send an email to LCSB admin team.

Accessing DUMA via Web Interface

In the following steps we provide instructions on how to use DUMA web interface.

1. Once you receive your access link and password from the LCSB, visit the link using a standard web browser. Firefox 66.x (or higher) recommended, but others should also work. You will be prompted for your password (see below).
   

2. When you visit a DUMA access link for the very first time, you will be prompted to install IBM Aspera Connect client.

   • click Download or Download latest version links (see below).
     

   • Your browser will download the client installer.

     

   • Run the installer till you receive the completion message.

     

   • Once IBM Aspera Connect client is installed, refresh the access link. You will be prompted whether you want to open IBM Aspera Launcher, click Open.

     

3. The access link page will display a File Browser section. Depending on the settings per access link, users can create or delete folders in the File Browser and upload or download data.
   

4. Clicking Upload or Download will launch the IBM Aspera Connect client on your computer. You first will be asked whether you allow the client to connect to aspera.lcsb.uni.lu. Choose Allow.
   

5. At any time you can launch IBM Aspera Connect to display the status of uploads to or downloads from your computer.
   

Accessing DUMA via Command-Line Tool

In the following steps we describe instructions on how to use DUMA web interface.

1. Install the aspera command line client. The ascp tool, which can be found here as instructed.

2. Obtain tokens. You will need tokens while using ascp. You will need a download token for download and a separate upload token for upload. Your tokens can be viewed by clicking the Help icon on your access link (see below).
   
   
   
   

3. Obtain SSH key. You will also need an SSH private key to authenticate. This is the Aspera key that comes with the Aspera Connect installation named asperaweb_id_dsa.openssh. Below are locations where the ssh_key can be found:

   • On MAC

     • Local installation of Aspera connect:
       • /Users/username/Applications/Aspera Connect.app/Contents/Resources/asperaweb_id_dsa.openssh
     • System wide installation of Connect:
       • /Applications/Aspera Connect.app/Contents/Resources/asperaweb_id_dsa.openssh

   • On WINDOWS

     • Local installation of Aspera connect:
       • C:\\Users\username\AppData\Local\Programs\Aspera\Aspera Connect\etc\asperaweb_id_dsa.openssh
     • System wide installation of Connect:
       • C:\\Program Files (x86)\Aspera\Aspera Connect\etc\asperaweb_id_dsa.openssh

   • On LINUX

     • Local installation of Aspera connect:
       • /home/username/.aspera/connect/etc/asperaweb_id_dsa.openssh
     • System wide installation of Connect:
       • /opt/aspera/etc/asperaweb_id_dsa.openssh

4. Set necessary environment variables. KEY and TOKEN environment variables should be set as follows. If you do not have any of the tokens use None as the value.

   • KEY=<path to the ssh key>
   • DTOKEN=<<<<YOUR DOWNLOAD TOKEN>>>>>>
   • UTOKEN=<<<<YOUR UPLOAD TOKEN>>>>>>

5. Run the relevant ascp commands.

   • Upload:
     • To upload all png files to the root of the share:
       • ascp -d -i $KEY -P 33001 -W $UTOKEN *.png app_duma_public_read@aspera.lcsb.uni.lu:/
     • To upload document.txt to a subfolder:
       • ascp -d -i $KEY -P 33001 -W $UTOKEN document.txt app_duma_public_read@aspera.lcsb.uni.lu:/subfolder
   • Download:
     • To download subfolder and all it's content to the current directory:
       • ascp -d -i $KEY -P 33001 -W $DTOKEN app_duma_public_read@aspera.lcsb.uni.lu:subfolder

You can find more information also on the Aspera website.