Hive的安装部署

大数据平台建设系列

Posted by 王灿辉 on 2020-02-22

获取Hive

从清华镜像源下载Apache的hive二进制压缩包到家目录的download目录下

1
wget https://mirrors.tuna.tsinghua.edu.cn/apache/hive/hive-2.3.6/apache-hive-2.3.6-bin.tar.gz -P ~/download

解压缩

将hive解压到/usr/local目录下

1
sudo tar zxvf apache-hive-2.3.6-bin.tar.gz -C /usr/local/

添加环境变量

将hive的主目录添加到环境变量中,编辑家目录下的.bashrc文件

1
vi ~/.bashrc

输入下面的环境变量

1
2
export HIVE_HOME=/usr/local/apache-hive-2.3.6-bin
export PATH=$HIVE_HOME/bin:$PATH

最后更新环境变量

1
source ~/.bashrc

修改配置文件

切换到hive的配置文件目录下,拷贝配置文件的模板,以便接下来进行修改

1
2
3
4
5
6
cd $HIVE_HOME/conf
cp hive-env.sh{.template,}
cp hive-{default.xml.template,site.xml}
cp hive-log4j2.properties{.template,}
cp hive-exec-log4j2.properties{.template,}
cp beeline-log4j2.properties{.template,}

首先修改hive-env.sh,找到HADOOP_HOME,修改为相应路径,如

1
HADOOP_HOME=/usr/local/hadoop

然后是hive-site.xml,添加如下配置

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
<configuration>
  <property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:mysql://localhost:3306/metastore?createDatabaseIfNotExist=true</value>
    <description>the URL of Mysql database</description>
  </property>

  <property>
    <name>javax.jdo.option.ConnectionDriverName</name>
    <value>com.mysql.jdbc.Driver</value>
    <description>Driver class name for a JDBC metastore</description>
  </property>

  <property>
    <name>javax.jdo.option.ConnectionUserName</name>
    <value>hive</value>
  </property>

  <property>
    <name>javax.jdo.option.ConnectionPassword</name>
    <value>123456</value>
  </property>

  <property>
    <name>hive.metastore.warehouse.dir</name>
    <value>/hive/warehouse</value>
  </property>

  <property>
    <name>hive.exec.scratchdir</name>
    <value>/hive/tmp</value>
  </property>

  <property>
    <name>hive.querylog.location</name>
    <value>/hive/log</value>
  </property>
</configuration>

安装MySQL Driver

下载MySQL Driver

1
wget https://mirrors.tuna.tsinghua.edu.cn/mysql/downloads/Connector-J/mysql-connector-java-5.1.48.zip -P ~/download/

解压MySQL Driver,并复制到hive的库目录下

1
2
unzip mysql-connector-java-5.1.48.zip
cp mysql-connector-java-5.1.48/mysql-connector-java-5.1.48-bin.jar $HIVE_HOME/lib

创建hive目录

在HDFS上创建相关目录,并设置读写权限

1
2
hadoop fs -mkdir -p /hive/tmp /hive/log /hive/warehouse
hadoop fs -chmod 777 /hive/tmp

初始化元数据

执行如下指令初始化元数据相关信息

1
schematool --dbType mysql --initSchema

显示如下信息表示成功

1
2
3
4
5
6
7
8
9
10
11
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.   SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL:        jdbc:mysql://localhost:3306/metastore?createDatabaseIfNotExist=true
Metastore Connection Driver :    com.mysql.jdbc.Driver
Metastore connection User:       root
Starting metastore schema initialization to 2.3.0
Initialization script hive-schema-2.3.0.mysql.sql
Initialization script completed
schemaTool completed

启动服务并连接

启动hive的metastore service(元数据服务)

1
hive --service metastore &

如果看到如下信息,说明服务已经成功启动了

1
2
3
4
5
2020-02-22 20:06:42: Starting Hive Metastore Server
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.   SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]

我们可以尝试使用hive客户端进行连接

1
hive

启动hiveserver2,用于支持JDBC的查询服务

1
hiveserver2 &

看到如下内容,表示启动成功

1
2
3
4
5
6
2020-02-22 20:20:53: Starting HiveServer2
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]

通过beeline连接hiveserver2

1
beeline -u jdbc:hive2://localhost:10000 hadoop


赞赏支持
微信赞赏
微信赞赏
支付宝
支付宝