,ONFS:a hierarchical hybrid file system based on memory,SSD, and HDD for high performance computers

来源 :信息与电子工程前沿(英文版) | 被引量 : 0次 | 上传用户:lzxldf2003
下载到本地 , 更方便阅读
声明 : 本文档内容版权归属内容提供方 , 如果您对本文有版权争议 , 可与客服联系进行内容授权或下架
论文部分内容阅读
With supercomputers developing towards exascale, the number of compute cores increases dramatically, making more complex and larger-scale applications possible. The input/output (I/O) requirements of large-scale applications, workflow applications, and their checkpointing include substantial bandwidth and an extremely low latency, posing a serious challenge to high performance computing (HPC) storage systems. Current hard disk drive (HDD) based underlying storage systems are becoming more and more incompetent to meet the requirements of next-generation exascale supercomputers. To rise to the challenge, we propose a hierarchical hybrid storage system, on-line and near-line file system (ONFS). It leverages dynamic random access memory (DRAM) and solid state drive (SSD) in compute nodes, and HDD in storage servers to build a three-level storage system in a unified namespace. It supports portable operating system interface (POSIX) semantics, and provides high bandwidth, low latency, and huge storage capacity. In this paper, we present the technical details on distributed metadata management, the strategy of memory borrow and ret, data consistency, parallel access control, and mechanisms guiding downward and upward migration in ONFS. We implement an ONFS prototype on the TH-1A supercomputer, and conduct experiments to test its I/O performance and scalability. The results show that the bandwidths of single-thread and multi-thread ’read’/’write’ are 6-fold and 5-fold better than HDD-based Lustre, respectively. The I/O bandwidth of data-intensive applications in ONFS can be 6.35 times that in Lustre.
其他文献
细胞质雄性不育三种杂种的选育是油菜杂种优势利用主要途径.目前,生产上得到大面积应用的油菜细胞质雄性不育类型主要有Pol CMS、陕2A CMS和MI CMS.对于前两者的研究较为
Cascaded regression has been recently applied to reconstruct 3D faces from single 2D images directly in shape space, and has achieved state-of-the-art performan
该文以热带种(Badila)为材料,利用茎尖和心叶愈伤组织培养进行脱毒研究.探讨茎尖离体微培养和快速繁殖技术,分析脱毒效果,考察脱毒苗的田间表现和遗传稳定性.初步结果如下:1.
Industrial control systems (ICSs) are widely used in critical infrastructures, making them popular targets for attacks to cause catastrophic physical damage. As
A novel dual-edge implicit pulse-triggered flip-flop with an embedded clock-gating scheme(DIFF-CGS) is proposed, which employs a transmission-gate-logic(TGL) ba
本研究以福建道地药材泽泻为材料,采用同源克隆和快速cDNA末端扩增技术克隆泽泻三萜类化合物合成途径中的关键酶——法尼基焦磷酸合酶基因的保守序列以及全长,为法尼基焦磷酸合酶在泽泻植物体内的作用机理以及基因控制与表达提供了理论依据,并可以为利用基因工程改造中草药的品质奠定基础。对泽泻叶面喷施不同类型、不同浓度的诱导子,检测泽泻主要药用成分23-乙酰泽泻醇B含量的变化,可以揭示不同浓度的诱导子对泽泻中的
该课题首次有选择地对木立芦荟的组织和细胞培养进行了系统研究.通过单因素、多水平的重复试验确定了伤组织诱导的适合外植体部位与适宜条件;通过正交试验对愈伤组织继代培养
为了探讨蒽和UV-B辐射对底栖生物早期发育的影响,在实验生态学的条件下,选择海湾扇贝(Argopecten irradians)担轮幼虫和D型幼虫作为受试生物进行单一和联合急性毒性试验.结果
该文研究了粳糯高梁籽粒的酿酒品质、酿酒工艺参数及酿酒效果(该文所指的酿酒主要指酿造浓香型和酱香型白酒)差异,在此基础上,进一步研究酿酒高梁的品种改良,分析了糯高梁的
“两个办法”的出台有利于有效盘活试点地区农村资源,为农村中小金融机构创新农地金融相关产品、开拓农村市场提供了机遇日前,人民银行会同相关部门联合印发了《农村承包土地