Skip to content
Snippets Groups Projects

Compare revisions

Changes are shown as if the source revision was being merged into the target revision. Learn more about comparing revisions.

Source

Select target project
No results found

Target

Select target project
  • vilem.ded/howto-cards
  • yjarosz/labCards
  • sarah.diehl/howto-cards
  • jacek.lebioda/howto-cards
  • pinar.alper/howto-cards
  • maharshi.vyas/howto-cards
  • manuel.maidorn/howto-cards
  • roland.krause/howto-cards
  • miriam.fougeras/howto-cards
  • soraya.hezzaz/howto-cards
  • fasavanh.sanichanh/howto-cards
  • marie.fossepre/howto-cards
  • jennifer.behm/howto-cards
  • annegrat.daujeumont/howto-cards
  • jon.gales/howto-cards-jpg
  • sandy.thill/howto-cards
  • jenny.tran/howto-cards
17 results
Show changes
Showing
with 214 additions and 29 deletions
external/contribute/web-ide/img/EditedLine.png

13 KiB

external/contribute/web-ide/img/MergeRequestPage.png

138 KiB

external/contribute/web-ide/img/OpenWebIDE.png

15.8 KiB

external/contribute/web-ide/img/SelectBranch.png

27.4 KiB

external/contribute/web-ide/img/VSCode.png

535 KiB

external/contribute/web-ide/img/img1.png

23.1 KiB

external/contribute/web-ide/img/img10.png

7.86 KiB

external/contribute/web-ide/img/img3.png

4.81 KiB

external/contribute/web-ide/img/img4.png

461 KiB

external/contribute/web-ide/img/img6.png

7.16 KiB

external/contribute/web-ide/img/img7.png

357 KiB

external/contribute/web-ide/img/img8.png

25.3 KiB

external/contribute/web-ide/img/img9.png

201 KiB

......@@ -7,11 +7,13 @@ redirect_from:
- /external/cards/contribute:web-ide
- /contribute/web-ide
- /external/external/contribute/web-ide/
- /cards/contribute:add-edit-card
- /external/cards/contribute:add-edit-card
---
# Contribute using Gitlab Web IDE
This shows how to use Gitlab Web IDE to add or edit a howto-card:
This card shows how to use GitLab Web IDE to modify a How-to card.
Before you start, please make sure that you have the correct access rights to edit the repository.
......@@ -20,73 +22,82 @@ the repository maintainers, or use your own copy of the repository (called a `fo
Please follow [these instructions](https://docs.gitlab.com/ee/user/project/repository/forking_workflow.html#creating-a-fork) to
obtain your copy (fork). You can then [mirror the fork automatically]({{ '/?contribute:mirror-fork' | relative_url }}) to keep in sync.*
A clear difference between editing a file in a git-tracked repository compared to a live document is that you can make an edit directly, and then suggesting the change. You can make this edit on your own version (called a `branch`), and then request to merge your changes into the main repository (through opening a `merge request`).
## Select the development version (branch)
Before making an edit, please follow the instructions in the `README.md` file. Often, a standard development scheme is followed. For instance, there is a `master` branch (usually referred to as the stable version of the repository) and a `develop` branch (the unstable development version of the repository). There can also be lots of other versions in the same repository. In order to select the development version (`develop` branch), click on the drop-down menu and select `develop`
Before making changes, please follow the instructions in the `README.md` file. This file contains important information about the project contribution guidelines. Often, a standard development scheme is followed. Usually, there is a `master` branch (referred to as the stable version of the repository) and a `develop` branch (the unstable development version of the repository). There can also be lots of other branches (versions) in the same repository. In most of the cases you would like to add your changes on top of `develop` branch that contains the latest version. In order to select the development version (`develop` branch), click on the drop-down menu on the top left of the page and select `develop`:
<img src="img/img1.png">
<img src="img/SelectBranch.png">
The page will refresh and show the repository contents. On the top left of the page, you can see that the `develop` branch has been selected.
The page will refresh and show the repository contents on this branch. You can see that the `develop` branch has been selected.
<img src="img/img2.png">
<img src="img/DevelopBranch.png">
## Open the web editor (Web IDE)
You can now open the web editor (integrated development environment or IDE) by clicking on the Web IDE button:
Now you can open the web editor (integrated development environment or IDE) by clicking on the Edit button and selecting Web IDE:
<img src="img/img3.png">
<img src="img/OpenWebIDE.png">
Now, you can make changes to the file you wish. As an example, we will make a change to the file `privacy_policy.md`. Select the file `privacy_policy.md` from the list on the left.
Now, you can make changes to the file you wish. As an example, we will change the file `privacy_policy.md`. Select the file `privacy_policy.md` from the list on the left.
<img src="img/img4.png">
<img src="img/VSCode.png">
Now, edit the file the way you want. For instance. we add a line with `test` in the first section:
Now, edit the file the way you want. For instance, we added a line with `test` in the first section:
<img src="img/img5.png">
<img src="img/EditLine.png">
*Note: You can see a green line on the left side of it, indicating that a new line has been added. The vertical line turns blue if a line is edited.*
<img src="img/EditedLine.png">
## Committing your change
As already mentioned, a change has to be proposed, and will then be reviewed. The system used is through `merge requests`. Now, as you have edited the file, you can see that on the bottom left of the window, a blue button appeared:
Now, as you have edited the file, you can see that on the top left corner of the window, a circle with a number appeared on the `Source Control` button:
<img src="img/CommitWindow.png">
<img src="img/img6.png">
This number indicates how many files has been changed, how many modifications there are to be saved.
*Note: There is no `Save` button. You save your work by committing changes.*
When clicking on `Commit...`, the page changes to display the changes made:
To commit changes (save your modifications) click on `Source Control` button. It will open a commit dialog. Write a descriptive commit message to label the changes properly.
<img src="img/img7.png">
If you would like to see all changes made in a file, click on the file name to display the changes made:
<img src="img/CompareChanges.png">
You can see that a green line has been added on the right column. The left column displays the original file (of the `develop` branch).
On the bottom left, you can see a form for committing the change. Here, you have to select `Create a new branch`. This is **important**.
**Important:** You have to select `Commit to new branch` by clicking on the down arrow.
<img src="img/img8.png">
<img src="img/CommitToNewBranch.png">
You can also add a specific commit message to label the change properly. The name of the branch is generated automatically. You can, if you want, change it as well.
Once you click on the `Commit to new branch` button, type the name of your new branch and press `Enter` to accept or `Escape` to cancel.
Then, select `Start a new request` when you are ready to start the submission process and hit the `Commit` button.
<img src="img/CreateNewBranch.png">
## Submitting your merge request
After hitting the `Commit` button, you will be redirect to the submission page for your merge request:
After creating the commit, the following pop-up window on the bottom right corner of your screen is going to indicate you that your changes have been commited. Click on the `Create MR` button to start your `Merge Request`.
<img src="img/CreateMR.png">
You will be directed to the `New merge request` page. This page allows you to merge your changes with the target branch.
<img src="img/img9.png">
<img src="img/MergeRequestPage.png">
Please make sure that you see that you are proposing to merge your branch (version) into the `develop` branch. This is shown at the top of the page:
Please make sure you are proposing to merge your branch (on your `fork`) into the `develop` branch (on main repository). You can change your target branch by clicking on the `Change branches` button at the top of the page. The following screenshot represents a merge request from `kc-test` branch on `kaan.cimir` fork of `howto-cards` to `develop` branch on original repository `R3/howto-cards`:
<img src="img/img10.png">
<img src="img/ChangeBranchesInMR.png">
You can then fill our the merge request submission form. and select, for instance, an `Assignee`.
You can then fill our the merge request submission form and select, for instance, an `Assignee`.
Good practice is to select `Delete source branch when merge request is accepted.`
Under `Changes`, you can see the added line:
<img src="img/img11.png">
<img src="img/ChangesImMR.png">
All that is left now is to submit the merge request by clicking on `Submit merge request`. That is it.
All that is left now is to submit the merge request by clicking on `Create merge request` button. That is it.
If you would like to understand more about described workflow, please consult [Git Trilogy Part 1: Contribute using GitLab](https://courses.lcsb.uni.lu/2022-05-31_git-trilogy-part-1/stable/#/) slides.
......@@ -13,7 +13,7 @@ redirect_from:
{:toc}
> AsperaWEB will be deprecated in the future (the end of service is planned for January 2023). LCSB File Transfer (LFT) will be used instead ([LFT quick guide](https://howto.lcsb.uni.lu/?exchange-channels:lft)).
> AsperaWEB will be deprecated in the future (the end of service is planned for January 2023). LCSB File Transfer (LFT) will be used instead ([LFT quick guide]({{ '/?exchange-channels:lft' | relative_url }})).
## Overview
......
---
layout: page
permalink: /external/exchange-channels/atlas-hpc/
shortcut: exchange-channels:atlas-hpc
redirect_from:
- /cards/exchange-channels:atlas-hpc
- /external/exchange-channels:atlas-hpc
- /integrity/exchange-channels/atlas-hpc
- /external/external/exchange-channels/atlas-hpc/
- /cards/integrity/transfer:atlas-hpc
- /external/integrity/transfer:atlas-hpc
- /integrity/transfer/atlas-hpc
- /external/external/integrity/transfer/atlas-hpc/
---
# Data transfer between Atlas and UL HPC Clusters
A recommended storage pattern is to have the master copy of data on Atlas (project folder) and only store data on the UL HPC Clusters temporarily for the required practical duration of computational analysis. The derived data and results should hereafter be transferred back to Atlas. This How-to Card describes the different methods to transfer data between Atlas and the UL HPC Clusters. The three recommended methods to transfer data are:
1. [Via laptop with ```scp``` or ```rsync```](#1-via-laptop-using-scp-or-rsync)
2. [Via dedicated Virtual Machine (VM)](#2-via-dedicated-virtual-machine-using-rsync)
3. [Via Large File Transfer (LFT)](#3-via-large-file-transfer-lft)
Please refer to the dedicated knowledge bases to see how to [connect to UL HPC Clusters](https://hpc-docs.uni.lu/connect/access/) and to [mount Atlas](https://service.uni.lu/sp?id=kb_article_view&sysparm_article=KB0010233).
<img src="img/data-transfer-flow.png">
## 1. Via laptop using `scp` or `rsync`
When using the UL laptop to transfer data between UL HPC Clusters and Atlas, you must [mount Atlas via smb on laptop](https://service.uni.lu/sp?id=kb_article_view&sysparm_article=KB0010233) before using `scp` and `rsync` for the transfer. While both commands ensure a secure transfer of data between the UL HPC Clusters and Atlas, `rsync` may be much faster for handling large amounts of small files (which are transferred very quickly in batches), and for selective incremental updates of large datasets (it is capable to automatically transferring only the changed files, thus saving time).
* `scp`: transfers all files and directories.
* `rsync`: transfers only the files which differ between the source and the destination.
Please visit the [UL HPC documentation](https://hpc-docs.uni.lu/data/transfer/#data-transfer-tofromwithin-ul-hpc-clusters) to see how to use `rsync` and `scp`.
## 2. Via dedicated Virtual Machine using rsync
Data can be transferred via a dedicated VM, which can be requested via [ServiceNow](https://service.uni.lu/sp?id=sc_cat_item&table=sc_cat_item&sys_id=49956812db3fa010ca53454039961978).
Instead of transferring data between Atlas and UL HPC Clusters through the laptop as described above, the transfer will go through the dedicated VM. Once connected to the VM and mounted to Atlas, the ```rsync``` command can be used in the same way as described in the [UL HPC documentation](https://hpc-docs.uni.lu/data/transfer/#data-transfer-tofromwithin-ul-hpc-clusters). This method is recommended for **recurring transfers of very large datasets** that benefit from high-speed network connection between the VM and the HPC.
**Note**: For larger transfers between Atlas and UL HPC Clusters, you may want to run the operations in background using `screen` or `tmux`. These prevent interruption of data transfer in cases when your `ssh` connection gets interrupted.
## 3. Via Large File Transfer (LFT)
An alternative solution is to use LFT for transferring data between Atlas and UL HPC Clusters. This method can reliably transfer large data volumes (typically several terabytes). However, LFT can only be used if the data is already on LFT (e.g., received from external collaborators). In this case, you can make a copy of the data and directly download it to the UL HPC Clusters for computational analysis. Note that a **master copy of the data must still be manually uploaded to Atlas** for internal archival.
Please refer to the [dedicated How-to Card on LFT]({{'/?exchange-channels:lft' | relative_url }}) for detailed information.
**Note**: In cases when the analysis data are not already received via LFT, we strongly recommend to use one of the other (simpler) methods instead.
external/exchange-channels/atlas-hpc/img/data-transfer-flow.png

127 KiB

---
card_order: 500
layout: page
permalink: /external/exchange-channels/cryptomator/
shortcut: exchange-channels:cryptomator
redirect_from:
- /cards/exchange-channels:cryptomator
- /external/cards/exchange-channels:cryptomator
- /exchange-channels/cryptomator
- /external/external/exchange-channels/cryptomator/
---
# Cryptomator
Cryptomator is a cloud cryptor - a tool allowing to work on and to share encrypted data seamlessly. The main benefit of using Cryptomator is that the service is transparent - after proper setup, the user and all applications can work with the data exactly the same way as before while the data is more secured.
<img src="img/cryptomator-diagram.png" height="500px"><br>
Cryptomator protects your data by ingesting it in a so-called vault, which is mounted to a virtual drive and password protected. To access and view the data, you must unlock the vault with the created password. The data is stored on the cloud of your choice when creating the vault, but is encrypted automatically by the Cryptomator client. It is only from the virtual drive which act as a USB flash drive that you can access and view the data, as it shows the decrypted view of its content. Ingesting your files directly into the virtual drive permits Cryptomator to automatically encrypt the data with AES and 256-bit key length.
This How-to Card provides a guide on the below topics. For the full documentation on using Cryptomator, please visit the [official website](https://docs.cryptomator.org/en/latest/).
- [Download Cryptomator Desktop](#downloading-cryptomator-desktop)
- [Setting up Cryptomator Hub](#setting-up-cryptomator-hub)
- [Vault management](#vault-management)
- [Collaborating on a shared vault as a user](#collaborating-on-a-shared-vault-as-a-user)
`Note:` Cryptomator does not offer cloud storage but adds an extra protection to your data stored on the cloud by automatically encrypting the files in the vault.
## Downloading Cryptomator Desktop
The desktop version of Cryptomator is available both for macOS, Windows, and Linux. Please follow the [guide](https://docs.cryptomator.org/en/latest/desktop/setup/) on Cryptomator's official website to install the application on your operating system.
<img src="img/cryptomator-main.png" height="400px"><br>
## Cryptomator Hub
Cryptomator Hub adds key and access management to your vaults from a central component deployed on your infrastructure, permitting collaborative work. As an owner/administrator of a vault, you can share your vaults with other people, who can access the vault by authenticating with Keycloak. There is thus no need for creating new accounts nor sharing encryption passwords with collaborators which are easily stolen, copied, or misused, and no data is shared on any online services.
As a vault administrator it is important to manage the admin password properly. In case you lose or forget the admin password, a recovery key is needed to restore access to the vault. This recovery key is generated upon creating the vault (see guide below). In practical, it is the responsibility of the project manager and/or PI to manage and store the admin vault password securely. Please use passphrase and proper password management when creating the admin vault password. For further information, visit the How-to Card on [password management]( https://howto.lcsb.uni.lu/?access:passwords).
## Setting up Cryptomator Hub
**Prerequisite:** A Cryptomator license is required to use Cryptomator Hub and can be requested via the [Service Portal](https://service.uni.lu/sp?id=sc_cat_item&sys_id=d3766892db3fa010ca534540399619a8&sysparm_category=735620d2db3fa010ca53454039961940).
1. Go to the LCSB instance of [Cryptomator Hub](https://lcsb.cryptomator.cloud/).
2. Click on “LCSB Login”.
<img src="img/cryptomatorHub-login.png" height="400px"> <br>
3. Select "uni.lu" account to sign in.
<img src="img/uni-login.png" height="400px"> <br>
4. When signed in successfully, you will be presented with the Hub's main page. To create a new vault, click the “Add” button on the top right corner and then "create new".
<img src="img/hub-mainpage.png" height="350px"> <br>
5. Fill out vault name and admin password. The admin password is required to grant or revoke access to the vault. When the form is completed, click “Next”.
<img src="img/creating-vault.png" height="350px"> <br>
6. A recovery key is displayed on the page. Copy the recovery key and store it in a safe location (e.g. password management system like [Bitwarden](https://bitwarden.com/) or [Keepass](https://keepass.info/)). The recocery key is needed for restoring access to the vault data. Tick the checkbox and click “Create Vault”.
<img src="img/recovery-key.png" height="350px"> <br>
7. The vault has now been successfully created from the Hub. You can now mount the vault to your desired cloud storage location by clicking “download zipped vault folder”.
<img src="img/download-vault-template.png" height="350px"> <br>
8. Unzip the vault in the cloud storage location (e.g. ownCloud or OneDrive).
<img src="img/ownCloud-folder.png" height="300px"> <br>
## Vault management
The following steps shows how to share a vault with other people and can only be performed by the vault administrator.
1. From the Hub's main page, click on the vault you wish to share and then "Manage vault".
<img src="img/manage-vault.png" height="250px"> <br>
2. Enter the vault admin password and click "Manage".
<img src="img/vault-admin-password.png" height="200px"> <br>
3. You are now able to share the vault with other team members by simply searching for the user and clicking the “add” button.
<img src="img/add-member.png" height="450px"> <br>
4. Before the user can access your vault, the user has to register the vault to the user's local device (see section "Collaborating on a shared vault as a user). Only then can the vault administrator grant permission by clicking on “update permission”.
<img src="img/grant-access.png" height="150px"> <br>
## Collaborating on a shared vault as a user
1. Open the desktop application. Click the “add’ button and then “Existing vault”.
<img src="img/add-existing-vault.png" height="400px"> <br>
2. Click on “Choose”.
<img src="img/vault-file.png" height="400px"> <br>
3. Go to the shared vault folder on the cloud. Select "vault.cryptomator" and then "Open".
<img src="img/import-vault.png" height="400px"> <br>
4. Click "Unlock now".
<img src="img/add-vault.png" height="350px"> <br>
5. Cryptomator will automatically redirect you to the login page. Please authenticate yourself with Keycloak.
6. If it is the first time you connect your device to the shared vault, you have to register the device. In this case, you will be prompt to enter a device name and then click "Confirm". The admin can now grant access to the device.
<img src="img/register-device.png" height="200px"> <br>
7. Once the admin grants access, click “Reveal drive” to access the vault on the virtual drive.
<img src="img/unlock-vault.png" height="400px"> <br>
8. You can now view decrypted data and save files in the vault on the virtual drive.
<img src="img/vault-data.png" height="400px"> <br>
9. The cloud cannot see anything in the folder, as all data is encrypted - both folder/file name and files.
<img src="img/cloud-data.png" height="400px"> <br>
`Note:` Always ingest the files directly into the vault on the virtual drive, as it will be encrypted and stored properly. If the file is ingested directly in the folder of the cloud storage location, it will not get encrypted by Cryptomator!
external/exchange-channels/cryptomator/img/add-existing-vault.png

152 KiB