Putting data on the Storage Resource Broker (SRB)

Simply "putting" data to the SRB is very straightforward. There are both commandline interfaces (Sput: See What's all this Scommands stuff?) and GUI interfaces available (e.g., InQ: See the BIRN Portal). Both individual files and multi-file collections can be moved easily.

However, these methods put the data anywhere the user indicates, without any capability to find it again over the long-term. Many investigators have had the experience of data proliferating without an organized method for finding it as the lab personnel who were responsible for it move on, etc.

The BIRN vision of mediated data sharing over the long term intends to be robust to individual user's idiosyncracies. Thus the FBIRN has developed Upload Scripts: generic Linux-based shell scripts which

• • identify the structure of the imaging datasets,
  • organize them according to a standard data structure,
  • copy the files with their XCEDE meta-data files in the appropriate structure on the SRB, and
  • link the location of the files to their appropriate entry in a database for future queries.

The original upload scripts were developed and refined on the various collaborative FBIRN studies. As a result of these repeated modifications, generic upload scripts are now being made available.

The location of the data on the SRB is a URI link; thus the data can be moved without needing to update all databases which point to it.

Getting data off the Storage Resource Broker

Download capabilities also exist through command-line or GUI methods. If the user knows exactly what files are wanted and what their SRB locations are, and if the user has appopriate permissions, the downloads can be as simple as using "Sget" (see What's all this Scommands stuff?).

However, the user may not know which data they want. Current query and download methods work with the FBIRN database schema, to find the data the user is looking for, identify how large it is, allow the user to have a "shopping cart" of data to download, and to download data in the background (since these large datasets coming from around the country may not arrive instantaneously).

Web API specifications for interfacing with XCEDE 2.0 for easier upload and download are scheduled for Fall 2007.