Home

Manage Data

About Managing Your Data

Whether your data is structured or unstructured data, or large or small — kilobytes or terabytes — CyVerse's Data Store can help you with your data storage needs. Different interfaces are available, from web services to mountable file systems to high-speed command-line transfers.

CyVerse's data hosting is easy to use. Based on a technology called iRODS, CyVerse's data hosting gives you great flexibility and control over your data. If you choose to host your data with CyVerse, you can upload and access your data from within various CyVerse services such as Atmosphere and the Discovery Environment. You can also access it directly using other interfaces in Python, C, Java, and REST API.

You can set permissions to keep your data private, share it with specific collaborators, or make it accessible to the general public (no account required). The Data Store natively supports parallel data transfers for fast uploads and can handle partial transfers, so if you interrupt a transfer of 1,000 files, or one very large file, the process can resume in the same place later. You also can use CyVerse to share data and analyses.

Data Allocations

Users have an initial allocation of 100 GB.

Additional allocation up to 1 TB can be requested by submitting an allocation increase using the Increase Your Data Store Allocation form. For allocations greater than 1 TB, please include in your request an estimate of your initial storage needs and estimated growth over time. Please be aware that extremely large requests may need to go through an allocation committee process. For questions, please contact CyVerse Support.

Access Methods

There are a number of different ways with which you can access private or public data in the Data Store. Most require a CyVerse account, but you can download public data using the Data Commons repositories, Community Contributed Data and CyVerse Curated Data, without a CyVerse account. For more information, see Downloading Data Files Without a User Account.

METHOD ACCESS POINT UPLOAD DOWNLOAD INSTALL & SETUP ACCOUNT REQUIRED MAX FILE SIZE
More Information
Discovery Environment (DE) Web Both No YES

Upload: 2 GB/file

Import: Up to max allocation and provider's limit

DCR (formerly Mirrors)

NO LOGIN REQUIRED
Web Download only No NO Up to 2 GB No login required. Access data files that have been shared by CyVerse users with the general public.
Cyberduck (Replaced iDrop Desktop) GUI Standalone desktop Both Yes Login or Public Very large > 10 GB
iCommands Command line Both

 
Yes Login or Public Very large > 10 GB

 

FUSE Mount Command line Both Yes Yes 1 GB/file
iDrop Desktop (Replaced by Cyberduck)
About Firewalls

Transfers to the Data Store require firewalls to allow TCP port 1247 and TCP and UDP ports 20000-20399 to be open.

If you notice that long running transfers are terminated abruptly: Some institutional firewalls have policies governing the duration allowed for idle connections that may interfere with data transfers. 

CyVerse staff can provide additional networking and subnet information for firewall policies upon request. For more information, contact CyVerse Support at support@cyverse.org.