A Task Allocation Method for Stream Processing with Recovery Latency Constraint

来源 :计算机科学技术学报(英文版) | 被引量 : 0次 | 上传用户:lanshi2008
下载到本地 , 更方便阅读
声明 : 本文档内容版权归属内容提供方 , 如果您对本文有版权争议 , 可与客服联系进行内容授权或下架
论文部分内容阅读
Stream processing applications continuously process large amounts of online streaming data in real time or near real time.They have strict latency constraints.However,the continuous processing makes them vulnerable to any failures,and the recoveries may slow down the entire processing pipeline and break latency constraints.The upstream backup scheme is one of the most widely applied fault-tolerant schemes for stream processing systems.It introduces complex backup dependencies to tasks,which increases the difficulty of controlling recovery latencies.Moreover,when dependent tasks are located on the same processor,they fail at the same time in processor-level failures,bringing extra recovery latencies that increase the impacts of failures.This paper studies the relationship between the task allocation and the recovery latency of a stream processing application.We present a correlated failure effect model to describe the recovery latency of a stream topology in processor-level failures under a task allocation plan.We introduce a recovery-latency aware task allocation problem (RTAP) that seeks task allocation plans for stream topologies that will achieve guaranteed recovery latencies.We discuss the difference between RTAP and classic task allocation problems and present a heuristic algorithm with a computational complexity of O(n log2 n) to solve the problem.Extensive experiments were conducted to verify the correctness and effectiveness of our approach.It improves the resource usage by 15% 20% on average.
其他文献
先天性髋脱位(congenital dislocation of hip,CDH)是婴儿出生时就患有的髋关节疾患.但是,并不是所有的病儿在刚出生时即可诊断本病[1~3],而是在出生后逐渐演变而成.因此,有作
患者 女 ,5 6岁 ,无明显诱因出现腰痛伴左下肢不适 1月余。查体 :L4~ 5椎间隙左侧压痛明显 ,左小腿外侧皮肤感觉较对侧减弱。X线摄片检查 :未见椎体破坏征象 ,椎间隙未见狭窄。
一、养殖池的准备 选择的池塘要求水源清新,纳水方便,未受污染,底质为砂底或泥砂底,单池面积一般为20~60亩,水深平均1.7米左右,最好为长方形东西走向,两端设进排水闸门,可顺
期刊
1.硕果累累,乡村旧貌换新颜rn长春市是一个农业大市,但各乡村普遍存在财力不足,基础设施欠账太多等问题.“一条街,两排房,后面臭水塘,到处垃圾场”是多数村屯环境的真实写照.
疑难性消化道出血是临床工作中的棘手问题 ,通过一般检查方法难以确诊 ,且明确出血部位及性质对指导治疗有重要意义。我们从 1989年开始应用选择性血管造影诊断疑难性消化道
长春市双阳区是一个以农业为主的新城区.几年来,用新农村建设统领“三农”工作,按照“干村示范,万村提升”新农村建设总要求,结合打造“宜居、宜游、宜兴业”区域发展战略,在
长春市朝阳区区委、区政府以农村基础设施建设为重点,采取高起点规划、全方位铺开、多方面参与、大幅度投入、长效机制保障等措施,在朝阳大地上描绘出一幅幅如诗如画的优美画