|
|
摘 要
随着云时代和大数据时代的到来，跟随着5G时代的马上到来，大数据慢慢的让大家关注，大数据在企业中的各种应用也随着时间慢慢多了起来，企业在日常运行中产生的，和累计的数据，不计其数，这些庞大的数据的存储量可能已经达到了Tb，pb的级别，甚至有的已经达到eb的级别，那么这么大量的数据我们需要有效的利用起来是很困难的，所以我们需要大数据来出来这些大量而且种类多，有效数据稀少的数据，那么我们就需要一个系统来处理这些数据。Hadoop是一个开源的分布式文件系统并且还是并行的计算的编程模型，得到了很多使用他的人的喜爱，并且的到了广泛的应用[1]。本文将应用大数据中的hadoop系统来做我用户行为数据的存储系统，那么我的计算分析的计算所用到的模型就是hadoop中的mapreduce的一个计算模型，之后就是用hive的数据仓库做数据的查询和分析，然后完成整个用户行为分析系统，本文还有一个重点就是介绍了一个大数据中的前端展示框架echarts，这个框架是中国在大数据方向做的一个前端模块化展示的一个很好用的一个框架，还有hadoop是一个分布式的文件系统，那么他的稳定性，和适用性是很好的。当然在这个大数据的时代还有几个概率也是本文要介绍的：hadoop中的集群是什么，分布式又是什么,集群中的负载均衡是什么，集群的扩展能力又是什么意思，这些就是本文需要介绍的内容。
关键词： Hadoop；hive；mapreduce；用户行为分析；pb；eb；分布式；
ABSTRACT
As the cloud era and the advent of the era of big data, with 5 g era soon, big data more and more get the attention of public, big data in the enterprise application is already very common, enterprises generated in the daily operation, and accumulated data, countless, these huge data storage may have already reached the Tb, pb level, and even some have reached the level of eb, so such a large amount of data we need effective use of it is very difficult, so we need big data to come out of the large and variety, the effective data sparse data, then we will need a system to deal with these data. Hadoop, as an open source distributed file system and parallel computing programming model, has been widely loved by users and widely used. This paper USES hadoop in large data systems to do my user behavior data storage system, then my analysis of the calculation model is the model of the graphs for the hadoop, and then based on the data warehouse to do data hive query analysis system, and then complete the user behavior analysis system, there is also a key is introduced in this paper a big echarts front-end display in the data frame, the frame is made of China in the direction of big data a front-end modular show a very useful framework, and hadoop is a distributed file system, So his stability, his applicability is good. Of course, in this era of big data, there are several other probabilities that this paper will introduce: what is cluster in hadoop, what is distributed, what is load balancing in cluster, and what is the extension ability of cluster. These are the contents that this paper needs to introduce
Key words: Hadoop；hive；mapreduce；Distributed；pb；eb；
