关于hive启动的相关问题记录

问题：初始化hive元数据报错

[atguigu@hadoop102 software]$ schematool -initSchema -dbType mysql -verboseError: Table 'CTLGS' already exists (state=42S01,code=1050)
Closing: 0: jdbc:mysql://hadoop102:3306/metastore?useSSL=false
org.apache.hadoop.hive.metastore.HiveMetaException: Schema initialization FAILED! Metastore state would be inconsistent !!
Underlying cause: java.io.IOException : Schema script failed, errorcode 2
org.apache.hadoop.hive.metastore.HiveMetaException: Schema initialization FAILED! Metastore state would be inconsistent !!at org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:594)at org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:567)at org.apache.hive.beeline.HiveSchemaTool.main(HiveSchemaTool.java:1517)at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)at java.lang.reflect.Method.invoke(Method.java:498)at org.apache.hadoop.util.RunJar.run(RunJar.java:318)at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
Caused by: java.io.IOException: Schema script failed, errorcode 2at org.apache.hive.beeline.HiveSchemaTool.runBeeLine(HiveSchemaTool.java:1226)at org.apache.hive.beeline.HiveSchemaTool.runBeeLine(HiveSchemaTool.java:1204)at org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:590)... 8 more
*** schemaTool failed ***

首先想到的是删除hive在MySQL中创建的元数据库Metastore
和HDFS中创建的目录
结果还是报错
仔细一看是hive.metastore连接问题

org.apache.hadoop.hive.metastore.HiveMetaException: Failed to get schema version.at org.apache.hadoop.hive.metastore.tools.HiveSchemaHelper.getConnectionToMetastore(HiveSchemaHelper.java:94)at org.apache.hive.beeline.HiveSchemaTool.getConnectionToMetastore(HiveSchemaTool.java:169)at org.apache.hive.beeline.HiveSchemaTool.testConnectionToMetastore(HiveSchemaTool.java:475)at org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:581)at org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:567)at org.apache.hive.beeline.HiveSchemaTool.main(HiveSchemaTool.java:1517)at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)at java.lang.reflect.Method.invoke(Method.java:498)at org.apache.hadoop.util.RunJar.run(RunJar.java:318)at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Unknown database 'metastore'at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)at java.lang.reflect.Constructor.newInstance(Constructor.java:423)at com.mysql.jdbc.Util.handleNewInstance(Util.java:404)at com.mysql.jdbc.Util.getInstance(Util.java:387)at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:941)at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3870)at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3806)at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:871)at com.mysql.jdbc.MysqlIO.proceedHandshakeWithPluggableAuthentication(MysqlIO.java:1686)at com.mysql.jdbc.MysqlIO.doHandshake(MysqlIO.java:1207)at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2254)at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2285)at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2084)at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:795)at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:44)at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)at java.lang.reflect.Constructor.newInstance(Constructor.java:423)at com.mysql.jdbc.Util.handleNewInstance(Util.java:404)at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:400)at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:327)at java.sql.DriverManager.getConnection(DriverManager.java:664)at java.sql.DriverManager.getConnection(DriverManager.java:247)at org.apache.hadoop.hive.metastore.tools.HiveSchemaHelper.getConnectionToMetastore(HiveSchemaHelper.java:88)

查看MySQL中的用户权限

参考：https://blog.csdn.net/peterchan88/article/details/78341852

1.首先登陆用root用户登录mysql
mysql -u root -p2.查看root下
mysql> select user,host from mysql.user where user='root';grant all on *.* to 'root'@'hadoop102' identified by 'hadoop';4.测试是否能登录
mysql -u root -h 'hadoop102' -phadoop

使用初始化命令还是报错，继续往下查

org.apache.hadoop.hive.metastore.HiveMetaException: Failed to get schema version.
Underlying cause: java.sql.SQLException : Access denied for user 'root'@'hadoop102' (using password: YES)
SQL Error code: 1045
org.apache.hadoop.hive.metastore.HiveMetaException: Failed to get schema version.

检查 MySQL 服务的状态

显示 MySQL 服务的状态以及相关的日志信息，包括任何错误消息或警告
sudo systemctl status mysqld.servicestart 命令启动
sudo systemctl start mysqld.service

之前由于内存原因，删掉了MySQL解压后的相关文件，于是开始重新走一遍安装步骤

问题：登入MySQL报错

ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)

解决：使用“ln -s /storage/db/mysql/mysql.sock /var/lib/mysql/mysql.sock”命令，将正确的socket文件位置，软链接到提示错误的socket文件路径位置
试了还是不行，在此基础上重启mysql就好了，systemctl restart mysqld
参考文章：https://blog.csdn.net/Aria_Miazzy/article/details/92803246
新建 Hive 元数据库

初始化 Hive 元数据库，成功

schematool -initSchema -dbType mysql -verbose

在这里插入图片描述

hiveserver2 启动失败

metastore未正常启动

[atguigu@hadoop202 hive]$ nohup hive --service hiveserver2 2>&1 &

java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
...
Caused by: MetaException(message:Could not connect to meta store using any of the URIs provided. Most recent failure: org.apache.thri
ft.transport.TTransportException: java.net.ConnectException: 拒绝连接 (Connection refused)
...
Caused by: java.net.ConnectException: 拒绝连接 (Connection refused)
...
Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient

查看元数据库是否启动

ps aux | grep metastore

一开始查到的
启动 metastore

[atguigu@hadoop202 hive]$ hive --service metastore

在这里插入图片描述

HiveServer2 未正常启动

当时的原因是引擎问题，安装配置了tez引擎

<property><name>hive.execution.engine</name><value>tez</value>
</property>

bin/beeline连接不了

[atguigu@hadoop102 hive]$ bin/beeline -u jdbc:hive2://hadoop102:10000 -n 用户名 -p 密码Error: Could not open client transport with JDBC Uri: jdbc:hive2://hadoop102:10000: Failed to open new session: java.lang.RuntimeExce
ption: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: link999 is not allowed to impersonate anonymous (state=08S01,code=0)Beeline version 3.1.2 by Apache Hive

在hadoop的配置文件core-site.xml增加如下配置，重启hdfs，其中“xxx”是连接beeline的用户，将“xxx”替换成自己的用户名即可。最关键的是一定要重启hadoop

    <property><name>hadoop.proxyuser.xxx.hosts</name><value>*</value></property><property><name>hadoop.proxyuser.xxx.groups</name><value>*</value></property>

“*”表示可通过超级代理“xxx”操作hadoop的用户、用户组和主机
主要原因是hadoop引入了一个安全伪装机制，使得hadoop 不允许上层系统直接将实际用户传递到hadoop层，而是将实际用户传递给一个超级代理，由此代理在hadoop上执行操作，避免任意客户端随意操作hadoop
参考文章https://blog.csdn.net/qq_35746739/article/details/119032729
beeline 的常用参数

Option Description  
--autoCommit=[true/false] ---进入一个自动提交模式：beeline --autoCommit=true  
--autosave=[true/false]   ---进入一个自动保存模式：beeline --autosave=true  
--color=[true/false]    ---显示用到的颜色：beeline --color=true  
--delimiterForDSV= DELIMITER ---分隔值输出格式的分隔符。默认是“|”字符。  
--fastConnect=[true/false]  ---在连接时，跳过组建表等对象：beeline --fastConnect=false  
--force=[true/false]    ---是否强制运行脚本：beeline--force=true  
--headerInterval=ROWS   ---输出的表间隔格式，默认是100: beeline --headerInterval=50  
--help ---帮助  beeline --help  
--hiveconf property=value  ---设置属性值，以防被hive.conf.restricted.list重置：beeline --hiveconf prop1=value1   
--hivevar name=value   ---设置变量名：beeline --hivevar var1=value1  
--incremental=[true/false]  ---输出增量
--isolation=LEVEL  ---设置事务隔离级别：beeline --isolation=TRANSACTION_SERIALIZABLE  
--maxColumnWidth=MAXCOLWIDTH ---设置字符串列的最大宽度：beeline --maxColumnWidth=25  
--maxWidth=MAXWIDTH ---设置截断数据的最大宽度：beeline --maxWidth=150  
--nullemptystring=[true/false]  ---打印空字符串：beeline --nullemptystring=false  
--numberFormat=[pattern]     ---数字使用DecimalFormat：beeline --numberFormat="#,###,##0.00"  
--outputformat=[table/vertical/csv/tsv/dsv/csv2/tsv2] ---输出格式：beeline --outputformat=tsv   
--showHeader=[true/false]   ---显示查询结果的列名：beeline --showHeader=false  
--showNestedErrs=[true/false] ---显示嵌套错误：beeline --showNestedErrs=true  
--showWarnings=[true/false] ---显示警告：beeline --showWarnings=true  
--silent=[true/false]  ---静默方式执行，不显示执行过程信息：beeline --silent=true  
--truncateTable=[true/false] ---是否在客户端截断表的列     
--verbose=[true/false]  ---显示详细错误信息和调试信息：beeline --verbose=true  
-d <driver class>  ---使用一个驱动类：beeline -d driver_class  
-e <query>  ---使用一个查询语句：beeline -e "query_string"  
-f <file>  ---加载一个文件：beeline -f filepath  多个文件用-e file1 -e file2
-n <username>  ---加载一个用户名：beeline -n valid_user  
-p <password>  ---加载一个密码：beeline -p valid_password  
-u <database URL> ---加载一个JDBC连接字符串：beeline -u db_URL