全部 标题 作者
关键词 摘要

OALib Journal期刊
ISSN: 2333-9721
费用:99美元

查看量下载量

相关文章

更多...

云计算环境下MapReduce的资源建模与性能预测

, PP. 115-119

Keywords: 云计算,MapReduce,资源建模,MapReduce作业耗时建模,性能预测

Full-Text   Cite this paper   Add to My Lib

Abstract:

为了预测云计算环境下的作业资源与时间消耗,根据MapReduce的资源消耗模式,量化了MapReduce作业的资源使用,提出了一种预估Hadoop的MapReduce作业的中央处理器(CPU)利用率和运行时间的模型.使用多项式回归的方法,可以在云计算环境下,对不同配置的MapReduce作业的CPU利用率和运行时间作出预判.使用不同配置条件下CPU密集型的Hadoop基准测试验证了该模型的有效性,最后使用误差平方和、平均绝对百分误差、标准差和确定系数4种评估方法计算了模型预测的精准度.

References

[1]  White T. Hadoop: the definitive guide[M].O'Reilly, 2012.
[2]  Dean J, Ghemawat S. MapReduce: simplified data processing on large clusters[J]. Communications of the ACM, 2008,51(1):107-113.
[3]  Wang G, Butt A R, Pandey P, et al. Using realistic simulation for performance analysis of mapreduce setups[C]// Proceedings of the 1st ACM Workshop on Large-Scale System and Application Performance. ACM,2009.
[4]  Yang Hailong, Luan Zhongzhi, Li Wenjun, et al. MapReduce workload modeling with statistical approach[J]. Journal of Grid Computing, 2012,10(2):279-310.
[5]  Rizvandi N B, Taheri J, Moraveji R, et al. On modelling and prediction of total CPU usage for applications in mapreduce environments[M]//Algorithms and Architectures for Parallel Processing. Springer, 2012:414-427.
[6]  Rizvandi N B, Taheri J, Moraveji R, et al. A study on using uncertain time series matching algorithms for MapReduce applications[J]. Concurrency and Computation: Practice and Experience, 2013,25(12):1699-1718.

Full-Text

Contact Us

service@oalib.com

QQ:3279437679

WhatsApp +8615387084133