|
|
- 2018
基于Spark的分布式空间数据存储结构设计与实现
|
Abstract:
Apache Spark分布式计算框架可用于空间大数据的管理与计算，为实现云GIS提供基础平台。针对Apache Spark的数据组织与计算模型，结合Apache HBase分布式数据库，从分布式GIS内核的理念出发，设计并实现了分布式空间数据存储结构与对象接口，并基于某国产GIS平台软件内核进行了实现。针对点、线、面数据的存储与查询，与传统空间数据库系统PostGIS进行了一系列对比实验，验证了提出的分布式空间数据存储架构的可行性与高效性
| [1] | Eldawy A, Mokbel M F. Spatial Hadoop:A Map-Reduce Framework for Spatial Data[C]. 31st IEEE International Conference on Data Engineering, Seoal, Korea, 2015 |
| [2] | Hall G B, Leahy M G. Open Source Approaches in Spatial Data Handling[M]. Berlin, Heidelberg:Springer, 2008:87-104 |
| [3] | You S, Zhang J, Le G. Large-Scale Spatial Join Query Processing in Cloud[C]. 31st IEEE International Conference on Data Engineering, Seoal, Korea, 2015 |
| [4] | Yu J, Wu J, Sarwat M. GeoSpark:A Cluter Computing Framework for Processing Large-Scale Spatial Data[C]. 23rd ACM Sigspatial International Conference on Advances in Geographic Information Systems, Seattle, Washington, 2015 |
| [5] | Zaharia M, Chowdhury M, Das T, et al. Resilient Distributed Datasets:A Fault-tolerant Abstraction for In-memory Cluster Computing[C]. 9th ACM Usenix Conference on Networked Systems Design and Implementation, San Jose, USA, 2012 |
| [6] | Yue P, Ramachandran R, Baumann P, et al. Recent Activities in Earth Data Science[J]. IEEE Geoscience & Remote Sensing Magazine, 2016, 4(4):84-89 |
| [7] | Stonebraker M. SQL Databases v. NoSQL Databases[J]. Communications of the ACM, 2010, 53(4):10-11 |
| [8] | Huang B. Comprehensive Geographic Information Systems[M]. US:Elsevier, 2018:50-79 |
| [9] | Aji A, Wang F, Vo H, et al. Hadoop-GIS:A High Performance Spatial Data Warehousing System over MapReduce[J]. Proceedings of the VLDB Endowment, 2013, 6(11):1009-1020 |
| [10] | Tang M, Yu Y, Malluhi Q M, et al. LocationSpark:A Distributed In-memory Data Management System for Big Spatial Data[J]. Proceedings of the VLDB Endowment, 2016, 9(13):1565-1568 |
| [11] | Zaharia M, Chowdhury M, Franklin M J, et al. Spark:Cluster Computing with Working Sets[C]. 2nd ACM Usenix Conference on Hot Topics in Cloud Computing, Boston, USA, 2010 |
| [12] | Li D, Gong J, Zhu Q, et al. GeoStar-A China Made GIS Software for Digital Earth[C]. International Symposium on Digital Earth, Beijing, China, 1999 |
| [13] | Zhu Xinyan, Gong Jianya, Huang Juntao, et al. Spatial Data Organization and Management in GeoStar[J]. Geomatics and Information Science of Wuhan University, 2000, 25(2):122-126(朱欣焰, 龚健雅, 黄俊韬, 等. GeoStar空间数据组织与管理[J]. 武汉大学学报·信息科学版, 2000, 25(2):122-126) |
| [14] | Li Deren. Towards Geo-spatial Information Science in Big Data Era[J]. Acta Geodaetica et Cartographica Sinica, 2016, 45(4):379-384(李德仁. 展望大数据时代的地球空间信息学[J]. 测绘学报, 2016, 45(4):379-384) |
| [15] | Yue P, Baumann P, Bugbee K, et al. Towards Intelligent GIServices[J]. Earth Science Informatics, 2015, 8(3):463-481 |
| [16] | Vora M N. Hadoop-HBase for Large-Scale Data[C]. 1st IEEE International Conference on Computer Science and Network Technology, Harbin, China, 2011 |
