Rclone is a command-line program that helps manage files on cloud and remote systems. In particular, it is useful for transferring large sizes and amounts of files. This document details how to configure Rclone to transfer files on a local system to OURRstore, a tape archive manged by the University of Oklahoma open to all researchers at EPSCoR institutions (including the University of Kentucky).

Contents

Assumptions 

• You have already created accounts and sent tape drives to OURRstore.
• All of the files are between 1TB - 1GB in size (200 - 20 GB preferably).
• You have a SSH key pair, with the public key authorized for use on the OURRstore DTN (dtn2.oscer.ou.edu).

Setting Up an SSH Key Pair

If you need to generate a public-private key pair, run the following command to generate one. This command generates a RSA formatted key, which should be usable by almost all systems.

ssh-keygen -t rsa -b 4096 f ~/.ssh/<key-name>

You will have two files; a file with no extension (the private key) and a file with a .pub extension (the public key).

On the OURRstore DTN, you may need to create a ~/.ssh/authorized_keys file to let the system know your public key. SSH into the DTN using your OU credentials you created when setting up an accoutn and run the following commands to create this file and set the appropriate permissions. SFTP WILL NOT WORK without these permissions being set correctly.

mkdir ~/.ssh
chmod 700 ~/.ssh
vi ~/.ssh/authorized_keys           # copy public key into here
chmod 644 ~/.ssh/authorized_keys

Once the authorized_keys file is created on the DTN, copy the contents of the public key on your system into it.

Configuring Rclone 

Run the following command to begin setup for the OURRstore connection. Any options not specified in the following steps can usually be skipped; just hit "Enter" to set the default value of an option.

rclone config

1. Enter "n" to create a new connection.
   
2. When selecting the type of connection, select SFTP.
   
3. Enter a name for the connection, dtn2.oscer.ou.edu for the host, and the username for your OSCER account if its is different from your current one.
   
4. Enter the path for your private SSH key, usually located in .ssh.
   
5. If you haven't already, copy your public key from the computer where you configured rclone into ~/.ssh/authorized_keys on the DTN.

Transferring Files to OURRstore 

Run the following command to copy files to the tape archive on OURRstore.

rclone copy -P <files> <connection-name>:/archive/ourrstore/<groupname>/tape_2copies/

You can also copy files to your home directory or into /scratch with the following commands.

rclone copy -P <files> <connection-name>:
rclone copy -P <files> <connection-name>:/scratch/<username>