1. Distributed Storage in Automated Data Transfer
Daniel Hong1, Jeremy Rogers2, Dr. Micah Beck2
Farragut High School1,, University of Tennessee2
Introduction
Distributed storage is the use of multiple storages servers in different locations to store files, scientific data, and other forms of media. Distributed storage provides an effective way to spread files evenly across an area (ex. The world, country, state). Proper utilization of distributed storage can be used to take advantage of a network’s full capacity, and distributed storage can also be used for other purposes. The REDDNET project aims to develop an effective way of managing files on distributed servers through tools such as the Logistical Distribution Network (LoDN) and the Logistical Runtime System (LoRS).
Fig. 1: A map of REDDNET organizations and servers
Research Topic
For the project, the automation of data transfer was explored as well as different applications of distributed storage for scientific purposes. Geospatial data was an example of a download from distributed storage servers which was automated through LoDN. Methods to transfer data to isolated edge servers, which are not connected to the global LoDN server but support similar file transfers, were also explored in this project.
Fig. 2: Visualization of a download through LoDN
Results
The files downloaded through the Java program accurately reflected the previews shown on the GloVis web interface. A series of identifiers (corresponding to locations on the map) which represented the state of Tennessee were fed into the program, and scaled-down results (shown below) were automatically downloaded.
Methodology
A tool was written in Java to grab files from the GloVis project (Global Visualization) sponsored by the United States Geological Survey and the National Aeronautics and Space Administration. These files contain geospatial photographs that scientists can use to find different The Java tool utilized the new DateTime API to schedule tasks at a given time, and user-given inputs are downloaded at regular intervals. The framework which was written for the download could also be applied to different types of data (not just geospatial).
Acknowledgements:
This work was supported primarily by the Engineering Research Center Program of the National Science Foundation and the Department of Energy under NSF Award Number EEC and the CURENT Industry Partnership Program.