Our system configuration provides various options of archive tape redundancy to avoid lost files caused by a damaged tape; robustness in the face of disk failure; optimization according to file characteristics; and transparent operation with respect to users. Users concerned with optimized retrieval have the flexibility to customize their own migration model.
Standard Migration Configuration
Unless otherwise arranged, the data placed on the data store kept as two separate copies, each copy on a separate tape in the tape silo. This operation is transparent to the user and configured at the system level. The second copy provides a degree of redundancy, allowing the system to automatically recover from minor system errors and hardware failure. Note that in the event of an extraordinary catastrophic event (such as a fire in the machine room) both copies of the data could be lost.Projects that require a higher degree of redundancy should consider additional options, such as our off-site copy silo.
The SAM-FS configuration installed on store provides the following:
- write of 2 file copies onto separate archival tapes to forestall loss of data in the event of tape damage or other hardware failure;
- frequent scheduling of the archiver to write new or modified files to tape;
- frequent backup of on-line SAM-FS database thus allowing limited recovery of files lost due to user error (e.g. deletion or accidental modifications);
- We have the potential for users to tune the SAM-FS migration parameters for a project's specific requirements. See the section on Efficient Use for more information. Please contact us at email@example.com if you need assistance.
- flexibility of access. Users can access SAM-FS either:
- directly on the MDS host store.anu.edu.au, and consequently though standard unix services such as ftp/scp;
- via mdss commands on the APAC National Facility hosts, ac.apac.edu.au and lc.apac.edu.au;
- Through specially developed web service interfaces; or
- by accessing grid enabled data transfer services, such as gridftp and SRB.
In some special cases we all NFS mounts of a store user account on remote systems for restricted use. In general such access is not recommended. Where such access is granted the directory owner's UID must be identical on all systems to which the directory is exported.
Non-reproducable data and Off-site Redundancy
For projects that require a higher degree of data redundancy we have a smaller second silo which is located off-site in a separate location. In most cases the second silo is for irreproducable data needed for the long term term, such as data from a telescope. The second silo has fewer tape drives, and much less capacity than the main silo. In such cases the first copy is stored on both the large silo and the second copy is transparently stored on the smaller remote tape library. Projects that require such an option should discuss their need with the National Facility.