Data Avenue on MTA Cloud

 
 

Data Avenue is a data storage management service that enables to access different types of storage resources (including S3, sftp, GridFTP, iRODS, SRM servers) using a uniform interface. The provided REST API allows of performing all the typical storage operations such as creating folders/buckets, renaming or deleting files/folders, uploading/downloading files, or copying/moving files/folders between different storage resources, respectively, even simply using 'curl' from command line. Data Avenue automatically translates users' REST commands to the appropriate storage protocols, and manages long-running data transfers in the background.

Data Avenue is available on the MTA Cloud's official website (https://cloud.mta.hu/dataavenue). In the tutorial which is available under services menu, we establish a cluster with two nodes types. On the Data Avenue node the Data Avenue application will run, and on a predefined number of storage nodes an S3 storage will run, in order to be able to try Data Avenue file transfer software such as making buckets, download or copy files. We used Ceph and Docker components to build-up the cluster.

Publications:

Department

Manager

Ákos Hajnal

Laboratory of Parallel and Distributed Systems
hajnal.akos@sztaki.hun-ren.hu
+36 1 279 6076