Documents in the NTIS Technical Reports collection are the results of federally funded research. They are directly submitted to or collected by NTIS from Federal agencies for permanent accessibility to industry, academia and the public.  Before purchasing from NTIS, you may want to check for free access from (1) the issuing organization's website; (2) the U.S. Government Printing Office's Federal Digital System website http://www.gpo.gov/fdsys; (3) the federal government Internet portal USA.gov; or (4) a web search conducted using a commercial search engine such as http://www.google.com.
Accession Number DE2012-1033537
Title Enhancing I/O Throughput via Efficient Routing and Placement for Large-scale Parallel File Systems.
Publication Date Jan 2011
Media Count 9p
Personal Author D. A. Dillow G. M. Shipman S. Oral Y. Kim Z. Zhang
Abstract As storage systems get larger to meet the demands of petascale systems, careful planning must be applied to avoid congestion points and extract the maximum performance. In addition, the large data sets generated by such systems makes it desirable for all compute resources to have common access to this data without needing to copy it to each machine. This paper describes a method of placing I/O close to the storage nodes to minimize contention on Crays SeaStar2+ network, and extends it to a routed Lustre configuration to gain the same benefits when running against a center-wide file system. Our experiments using half of the resources of Spider the center-wide file system at the Oak Ridge Leadership Computing Facility show that I/O write bandwidth can be improved by up to 45% (from 71.9 to 104 GB/s) for a direct-attached configuration and by 137% (47.6 GB/s to 115 GB/s) for a routed configuration. We demonstrated up to 20.7% reduction in run-time for production scientific applications. With the full Spider system, we demonstrated over 240 GB/s of aggregate bandwidth using our techniques.
Keywords Communications
Computer networks
Configuration
Data storage
Parallel processing
Performance
Planning
Production
Routing


 
Source Agency Technical Information Center Oak Ridge Tennessee
NTIS Subject Category 62B - Computer Software
45C - Common Carrier & Satellite
Corporate Author Oak Ridge National Lab., TN.
Document Type Technical report
Title Note N/A
NTIS Issue Number 1309
Contract Number DE-AC05-00OR22725

Science and Technology Highlights

See a sampling of the latest scientific, technical and engineering information from NTIS in the NTIS Technical Reports Newsletter

Acrobat Reader Mobile    Acrobat Reader