Hive的安装和配置（包含MySQL安装）_hive配置mysql

目录

搭建Hadoop伪分布式环境

MySQL安装

hive安装

---

 在搭建hive前要有一个伪分布式环境和MySQL

搭建Hadoop伪分布式环境

创建一个虚拟机并安装好centos7系统

准备工作

1.修改主机名 vim /etc/hostname
    更改为hadoop00
2.设置固定IP vim /etc/sysconfig/network-scripts/ifcfg-ens33
    将dhcp改为static
     ONBOOT=yes
     IPADDR=192.168.32.130
     PREFIX=24
3.关闭防火墙
    systemctl stop firewalld.service
    systemctl disable firewalld.service
4.新建安装目录
    mkdir /opt/packages
    mkdir /opt/programs
5.配置主机映射 vim /etc/hosts
    添加 192.168.32.130 hadoop00
6.配置ssh免密码登录本地hadoop00节点
    #ssh-keygen
7.安装配置jdk
    先检查centos7中预安装的openjdk (执行"rpm -qa | grep openjdk"命令）
    若有则运行# rpm -e --nodeps +出现的jdk
    将jdk安装包上传至/opt/packages中
    解压安装包至/opt/programs  #tar -zxvf jdk-8u261-linux-x64.tar.gz -C /opt/programs
    配置环境变量 #vim /etc/profile.d/hadoop.sh
        #JAVA_HOME
        export JAVA_HOME=/opt/programs/jdk1.8.0_261
        export PATH=$PATH:$JAVA_HOME/bin
    刷新文件  #source /etc/profile
    执行 #java -version 若输出jdk版本，则安装成功

接下来安装配置Hadoop

1.将hadoop-3.3.1.tar.gz上传至/opt/packages并解压

tar -xvf hadoop-3.3.1.tar.gz -C /opt/programs/

2.进入/opt/programs/hadoop-3.3.1/etc/hadoop/依次修改配置文件

vim core-site.xml 

    <property>
        <name>fs.defaultFS</name>
        <value>hdfs://hadoop00</value>
    </property>
    <property>
        <name>hadoop.tmp.dir</name>
        <value>/opt/programs/hadoop-3.3.1/tmp</value>
    </property>

 vim hdfs-site.xml

    <property>
        <name>dfs.replication</name>
        <value>1</value>
    </property>

首先将mapred-site.xml.template重命名为mapred-site.xml

mv mapred-site.xml.template mapred-site.xml

 vim mapred-site.xml

    <property>
        <name>mapreduce.framework.name</name>
        <value>yarn</value>
    </property>
    <property>
        <name>dfs.permissions</name>
        <value>false</value>
    </property>

 vim yarn-site.xml

    <property>
        <name>yarn.resourcemanager.hostname</name>
        <value>hadoop00</value>
    </property>
    <property>
        <name>yarn.nodemanager.aux-services</name>
        <value>mapreduce_shuffle</value>
    </property>

 vim workers

hadoop00

 vim hadoop-env.sh

修改一行为 export HADOOP_CONF_DIR=/opt/programs/hadoop-3.3.1/etc/hadoop/
 
在文件末尾加入 export JAVA_HOME=/opt/programs/jdk1.8.0_261

3.添加环境变量 #vim /etc/profile.d/hadoop.sh 

#HADOOP_HOME
export HADOOP_HOME=/opt/programs/hadoop-3.3.1
export PATH=$PATH:$HADOOP_HOME/bin
export PATH=$PATH:$HADOOP_HOME/sbin

 刷新文件  #source /etc/profile

4.格式化hdfs

hdfs namenoded -format

 5.启动hdfs，yarn

MySQL安装

先检查centos7中已安装的数据库

rpm -qa|grep mariadb

若存在则将其删除：

rpm -e --nodeps mariadb-libs-5.5.68-1.el7.x86_64

1.上传mysql-5.7.28-1.el7.x86_64.rpm-bundle.tar到/opt/packages中，同时解压文件

tar -xvf mysql-5.7.28-1.el7.x86_64.rpm-bundle.tar -C /opt/programs/

2. 切换到/opt/programs文件夹下，查看信息

3.安装插件（按顺序安装）

rpm -ivh mysql-community-common-5.7.28-1.el7.x86_64.rpm
 
rpm -ivh mysql-community-libs-5.7.28-1.el7.x86_64.rpm
 
rpm -ivh mysql-community-client-5.7.28-1.el7.x86_64.rpm
 
rpm -ivh mysql-community-server-5.7.28-1.el7.x86_64.rpm

4.初始化MySQL

mysqld --initialize --user=mysql

5.启动MySQL

systemctl start mysqld.service

查看MySQL状态

systemctl status mysqld.service

6.查看MySQL密码

cat /var/log/mysqld.log |grep password

7.进入MySQL改密码

mysql -uroot -p
 
mysql>mysql>set password = password("root");   #修改密码为root

10设置MySQL可远程登录

mysql> show databases;
 
mysql> use mysql;
 
mysql> show tables;   #查看是否有user表
 
mysql> select Host,User from user;
 
mysql> update mysql.user set host='%' where user='root';   #更新user表中的权限
 
mysql> FLUSH PRIVILEGES;   #刷新

hive安装

1.上传apache-hive-3.1.3-bin.tar.gz到/opt/packages中，解压文件

tar -zxvf apache-hive-3.1.3-bin.tar.gz -C /opt/programs/

2.切换到/opt/programs中，更改名称

mv apache-hive-3.1.3-bin/ hive-3.1.3

3.配置hive的环境变量

vim /etc/profile.d/hadoop.sh

#HIVE_HOME
export HIVE_HOME=/opt/programs/hive-3.1.3
export PATH=$PATH:$HIVE_HOME/bin

刷新文件

 source /etc/profile

4.切换目录

cd /opt/programs/hive-3.1.3/conf

5.复制文件并更改以下信息

cp hive-env.sh.template hive-env.sh
 
vim hive-env.sh

6.在/opt/programs/hive-3.1.3/lib中将jar包注释

mv log4j-slf4j-impl-2.17.1.jar log4j-slf4j-impl-2.17.1.jar.bak

7.将mysql的jar驱动包上传至/opt/programs/hive-3.1.3/lib中

8.格式化hive

schematool -dbType mysql -initSchema

 出现此标志，格式化完成

9.新建并编辑hive-site.xml文件，添加以下信息

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
 
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
 
<configuration>
 
       <property>
 
              <name>javax.jdo.option.ConnectionURL</name>
 
              <value>jdbc:mysql://hadoop00:3306/hive?createDatabaseIfNotExist=true</value>
 
       </property>
 
       <property>
 
              <name>javax.jdo.option.ConnectionDriverName</name>
 
              <value>com.mysql.jdbc.Driver</value>
 
       </property>
 
       <property>
 
              <name>javax.jdo.option.ConnectionUserName</name>
 
              <value>root</value>
 
       </property>
 
       <property>
 
              <name>javax.jdo.option.ConnectionPassword</name>
 
              <value>root</value>
 
       </property>
 
       <property>
 
              <name>hive.metastore.schema.verification</name>
 
              <value>false</value>
 
       </property>
 
       <property>
 
              <name>hive.metastore.event.db.notification.api.auth</name>
 
              <value>false</value>
 
       </property>
 
</configuration>

10.在启动hive前，要保证hdfs，yarn集群启动并正常运行，同时确保MySQL已经启动

验证hive

执行命令#hive
并测试hive是否正常

11.在hive中若无法执行命令，则初始化hive在MySQL中的元数据

 schematool -initSchema -dbType mysql  -verbose