About the Data Store
Based on a technology called iRODS, CyVerse's easy-to-use data hosting gives you great flexibility and control over your data. Choose from different interfaces—from web services to mountable file systems to high-speed command-line transfers—to connect your data to every CyVerse platform. To learn how to use the Data Store, see Manage Data.
Hosting your data in CyVerse's Data Store allows you to:
- Use your preferred access method to upload, download, and access data from your computer to the Data Store.
- Share your data instantly with other users.
- Keep your data secure by setting permissions to keep your data private only to you, share it with specific collaborators, or make it accessible to the general public (so no account is required).
- Take advantage of parallel data transfers for fast uploads and can handle partial transfers—so if a transfer of 1,000 files or one very large file is interrupted, the process can resume in the same place later.
There are a number of different ways with which you can upload, import, and download private or public data in the Data Store. Most require a CyVerse account, but you can access public data anonymously using the Mirrors site (below) without an account, if you don't have one already set up.
If you prefer command line, you can use:
- iRODS iCommands, which gives you the ability to transfer files quickly using multithreaded transfers. See Setting Up iCommands for steps on downloading and configuring iCommands for your operating system.
- FUSE to mount a Data Store directory to a local directory, or view and navigate directories and directory contents, using the command line.
- An API (for programmers).
If you prefer direct access, you can access your data directly using other interfaces in Python, C, Java, and REST API.
Concerned about data security?
We’ve kept your data security in mind. You can set permissions to keep your files private, or share them with specific collaborators, or even with the general public—no account required.
Curious about data allocation?
Each user has access to 100 GB of data storage, with the ability to request more allocation. Learn all about our data allocation policy here.
Want to access data but don’t have an account?
You can download some data files that have shared with the general public using several different methods. Find out more here.
Want to know more about data performance?
Below are the performance numbers for moving 100 GB of data from UC Berkeley to the CyVerse Data Store. To put some of these numbers into perspective, look at the transfer rate for the file annotation.MYI :
- 22622 MB = ~22 GB
- 367 sec = 6.1 minutes
This is the equivalent of transferring 5 DVDs (or one Blu-ray) in 6 minutes.
While this is over the internet2 backbone between UC Berkeley and UA Tucson, there are ~70,000 students and thousands more faculty, staff, and administration. All told, 102 GB of data was transferred between Berkeley and the CyVerse Data Store in 30 minutes — or 1 GB in 18 seconds. The following table compares other copying/transfer methods.
|Berkeley Server (RAID10 7.2k)*||CyVerse Data Store||iRODS||18V sec|
|Mac Pro CD (50x)||Mac Pro HD (7.2k)||cp||5min: 20V sec|
|Mac Pro HD 7.2K||USB 2.0 HD||cp||36V sec|
|Mac Pro HD 7.2K||internal||cp||15V sec|
|Mac Pro UA||Berkeley Server||scp||2min: 30V sec|
|Mac Pro HD 7.2K||USB2.0 Flash (Patriot XT)||cp||48V sec|
|USB 2.0 Flash (Patriot XT)||Mac Pro HD7.2K||cp||30V sec|
*The Berkeley machine is located in the UC Data Center with 1G to their network.
- Berkeley's Server IP address: 126.96.36.199 (synteny.cnr.berkeley.edu)
- CyVerse Data Store IP address: 188.8.131.52 (data.iplantc.org)
Ready to get started?
- Manage your data basics
- Learn about using your data (CyVerse wiki)
- Learn how to store your data with CyVerse
- Downloading Data Files Without a User Account
- Learn about iCommands with CyVerse
- Learn about iRODS on their website
- Ask a question about the Data Store
- Request an increase to your Data Store allocation
- Review CyVerse's Data Management Policy