A Novel Spatio-Temporal Data Storage and Index Method for ARM-Based Hadoop Server

Laipeng Han; Lan Huang; Xueyi Yang; Wei Pang; Kangping Wang

doi:10.1007/978-3-319-48671-0_19

A Novel Spatio-Temporal Data Storage and Index Method for ARM-Based Hadoop Server

Laipeng Han, Lan Huang, Xueyi Yang, Wei Pang, Kangping Wang

Computing Science

Jilin University

Research output: Chapter in Book/Report/Conference proceeding › Published conference contribution

4 Citations (Scopus)

8 Downloads (Pure)

Abstract

During the past decade, a vast number of GPS devices have produced massive amounts of data containing both time and spatial information. This poses a great challenge for traditional spatial databases. With the development of distributed cloud computing, many highperformance cloud platforms have been built, which can be used to process such spatio-temporal data. In this research, to store and process data in an effective and green way, we propose the following solutions: firstly, we build a Hadoop cloud computing platform using Cubieboards2, an ARM development board with A20 processors; secondly, we design two types of indexes for different types of spatio-temporal data at the HDFS level. We use a specific partitioning strategy to divide data in order to ensure load balancing and efficient range query. To improve the efficiency
of disk utilisation and network transmission, we also optimise the storage structure. The experimental results show that our cloud platform is highly scalable, and the two types of indexes are effective for spatio-temporal data storage optimisation and they can help achieve high retrieval efficiency.

Original language	English
Title of host publication	Cloud Computing and Security
Subtitle of host publication	Second International Conference, ICCCS 2016. Lecture Notes in Computer Science
Editors	X. Sun, A. Liu, H.-C. Chao, E. Bertino
Publisher	Springer
Pages	206-216
Number of pages	11
ISBN (Electronic)	978-3-319-48671-0
ISBN (Print)	978-3-319-48670-3
DOIs	https://doi.org/10.1007/978-3-319-48671-0_19
Publication status	Published - 2016
Event	Second International Conference, ICCCS 2016 - Nanjing, China Duration: 29 Jul 2016 → 31 Jul 2016

Publication series

Name	Information Systems and Applications, incl. Internet/Web, and HCI (Lecture Notes in Computer Science)
Publisher	Springer
Volume	10039

Conference

Conference	Second International Conference, ICCCS 2016
Country/Territory	China
City	Nanjing
Period	29/07/16 → 31/07/16

Bibliographical note

This project is supported by Science and Technology Development Plan of Jilin Province (20140204010SF) and Chinese National Natural Science Foundation (61472159). WP is supported by the PECE bursary from The Scottish Informatics and Computer Science Alliance (SICSA).

Access to Document

10.1007/978-3-319-48671-0_19Licence: Unspecified

Accepted Manuscript
The final publication is available at Springer via http://dx.doi.org/10.1007/978-3-319-48671-0_19
Accepted author manuscript, 312 KBLicence: Unspecified

Cite this

Han, L., Huang, L., Yang, X., Pang, W., & Wang, K. (2016). A Novel Spatio-Temporal Data Storage and Index Method for ARM-Based Hadoop Server. In X. Sun, A. Liu, H-C. Chao, & E. Bertino (Eds.), Cloud Computing and Security: Second International Conference, ICCCS 2016. Lecture Notes in Computer Science (pp. 206-216). (Information Systems and Applications, incl. Internet/Web, and HCI (Lecture Notes in Computer Science); Vol. 10039). Springer . https://doi.org/10.1007/978-3-319-48671-0_19

A Novel Spatio-Temporal Data Storage and Index Method for ARM-Based Hadoop Server. / Han, Laipeng; Huang, Lan; Yang, Xueyi et al.
Cloud Computing and Security: Second International Conference, ICCCS 2016. Lecture Notes in Computer Science. ed. / X. Sun; A. Liu; H.-C. Chao; E. Bertino. Springer , 2016. p. 206-216 (Information Systems and Applications, incl. Internet/Web, and HCI (Lecture Notes in Computer Science); Vol. 10039).

Research output: Chapter in Book/Report/Conference proceeding › Published conference contribution

Han, L, Huang, L, Yang, X, Pang, W & Wang, K 2016, A Novel Spatio-Temporal Data Storage and Index Method for ARM-Based Hadoop Server. in X Sun, A Liu, H-C Chao & E Bertino (eds), Cloud Computing and Security: Second International Conference, ICCCS 2016. Lecture Notes in Computer Science. Information Systems and Applications, incl. Internet/Web, and HCI (Lecture Notes in Computer Science), vol. 10039, Springer , pp. 206-216, Second International Conference, ICCCS 2016, Nanjing, China, 29/07/16. https://doi.org/10.1007/978-3-319-48671-0_19

Han L, Huang L, Yang X, Pang W, Wang K. A Novel Spatio-Temporal Data Storage and Index Method for ARM-Based Hadoop Server. In Sun X, Liu A, Chao H-C, Bertino E, editors, Cloud Computing and Security: Second International Conference, ICCCS 2016. Lecture Notes in Computer Science. Springer . 2016. p. 206-216. (Information Systems and Applications, incl. Internet/Web, and HCI (Lecture Notes in Computer Science)). Epub 2016 Nov 1. doi: 10.1007/978-3-319-48671-0_19

Han, Laipeng ; Huang, Lan ; Yang, Xueyi et al. / A Novel Spatio-Temporal Data Storage and Index Method for ARM-Based Hadoop Server. Cloud Computing and Security: Second International Conference, ICCCS 2016. Lecture Notes in Computer Science. editor / X. Sun ; A. Liu ; H.-C. Chao ; E. Bertino. Springer , 2016. pp. 206-216 (Information Systems and Applications, incl. Internet/Web, and HCI (Lecture Notes in Computer Science)).

@inproceedings{1bc9f85b2e474d2f9dc5d302e350755c,

title = "A Novel Spatio-Temporal Data Storage and Index Method for ARM-Based Hadoop Server",

abstract = "During the past decade, a vast number of GPS devices have produced massive amounts of data containing both time and spatial information. This poses a great challenge for traditional spatial databases. With the development of distributed cloud computing, many highperformance cloud platforms have been built, which can be used to process such spatio-temporal data. In this research, to store and process data in an effective and green way, we propose the following solutions: firstly, we build a Hadoop cloud computing platform using Cubieboards2, an ARM development board with A20 processors; secondly, we design two types of indexes for different types of spatio-temporal data at the HDFS level. We use a specific partitioning strategy to divide data in order to ensure load balancing and efficient range query. To improve the efficiencyof disk utilisation and network transmission, we also optimise the storage structure. The experimental results show that our cloud platform is highly scalable, and the two types of indexes are effective for spatio-temporal data storage optimisation and they can help achieve high retrieval efficiency.",

author = "Laipeng Han and Lan Huang and Xueyi Yang and Wei Pang and Kangping Wang",

note = "This project is supported by Science and Technology Development Plan of Jilin Province (20140204010SF) and Chinese National Natural Science Foundation (61472159). WP is supported by the PECE bursary from The Scottish Informatics and Computer Science Alliance (SICSA). ; Second International Conference, ICCCS 2016 ; Conference date: 29-07-2016 Through 31-07-2016",

year = "2016",

doi = "10.1007/978-3-319-48671-0_19",

language = "English",

isbn = "978-3-319-48670-3",

series = "Information Systems and Applications, incl. Internet/Web, and HCI (Lecture Notes in Computer Science)",

publisher = "Springer ",

pages = "206--216",

editor = "X. Sun and A. Liu and H.-C. Chao and E. Bertino",

booktitle = "Cloud Computing and Security",

}

TY - GEN

T1 - A Novel Spatio-Temporal Data Storage and Index Method for ARM-Based Hadoop Server

AU - Han, Laipeng

AU - Huang, Lan

AU - Yang, Xueyi

AU - Pang, Wei

AU - Wang, Kangping

N1 - This project is supported by Science and Technology Development Plan of Jilin Province (20140204010SF) and Chinese National Natural Science Foundation (61472159). WP is supported by the PECE bursary from The Scottish Informatics and Computer Science Alliance (SICSA).

PY - 2016

Y1 - 2016

N2 - During the past decade, a vast number of GPS devices have produced massive amounts of data containing both time and spatial information. This poses a great challenge for traditional spatial databases. With the development of distributed cloud computing, many highperformance cloud platforms have been built, which can be used to process such spatio-temporal data. In this research, to store and process data in an effective and green way, we propose the following solutions: firstly, we build a Hadoop cloud computing platform using Cubieboards2, an ARM development board with A20 processors; secondly, we design two types of indexes for different types of spatio-temporal data at the HDFS level. We use a specific partitioning strategy to divide data in order to ensure load balancing and efficient range query. To improve the efficiencyof disk utilisation and network transmission, we also optimise the storage structure. The experimental results show that our cloud platform is highly scalable, and the two types of indexes are effective for spatio-temporal data storage optimisation and they can help achieve high retrieval efficiency.

AB - During the past decade, a vast number of GPS devices have produced massive amounts of data containing both time and spatial information. This poses a great challenge for traditional spatial databases. With the development of distributed cloud computing, many highperformance cloud platforms have been built, which can be used to process such spatio-temporal data. In this research, to store and process data in an effective and green way, we propose the following solutions: firstly, we build a Hadoop cloud computing platform using Cubieboards2, an ARM development board with A20 processors; secondly, we design two types of indexes for different types of spatio-temporal data at the HDFS level. We use a specific partitioning strategy to divide data in order to ensure load balancing and efficient range query. To improve the efficiencyof disk utilisation and network transmission, we also optimise the storage structure. The experimental results show that our cloud platform is highly scalable, and the two types of indexes are effective for spatio-temporal data storage optimisation and they can help achieve high retrieval efficiency.

U2 - 10.1007/978-3-319-48671-0_19

DO - 10.1007/978-3-319-48671-0_19

M3 - Published conference contribution

SN - 978-3-319-48670-3

T3 - Information Systems and Applications, incl. Internet/Web, and HCI (Lecture Notes in Computer Science)

SP - 206

EP - 216

BT - Cloud Computing and Security

A2 - Sun, X.

A2 - Liu, A.

A2 - Chao, H.-C.

A2 - Bertino, E.

PB - Springer

T2 - Second International Conference, ICCCS 2016

Y2 - 29 July 2016 through 31 July 2016

ER -

A Novel Spatio-Temporal Data Storage and Index Method for ARM-Based Hadoop Server

Abstract

Publication series

Conference

Bibliographical note

Access to Document

Fingerprint

Cite this