Hive Abnormal Error Analysis
Introduction
In this article, we will delve into the anomalies that occur when integrating HBase with Hive. We will explore the errors that arise during the creation of a table in Hive and examine the reasons behind these exceptions.
The Anomalies
When creating a table in Hive that integrates with HBase, the following statement is used:
CREATE TABLE hbase_table_1 (
key int,
value string
)
STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
WITH SERDEPROPERTIES (
"hbase.columns.mapping" = ": key, cf1: val"
)
TBLPROPERTIES (
"hbase.table.name" = "xyz",
"hbase.mapred.output.outputtable" = "xyz"
);
However, an abnormal error appears, indicating a failure in the execution of the DDL task:
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
org.apache.hadoop.hbase.HTableDescriptor.addFamily (Lorg / apache / hadoop / hbase / HColumnDescriptor;)
Detailed Log Analysis
To further investigate the issue, we can use the following command to view the detailed log in Hive:
hive -hiveconf hive.root.logger = DEBUG, console
The log details reveal the following:
18/03/26 16:19:51 [main]: DEBUG metastore.ObjectStore: Open transaction: count = 1, isActive = true at: org.apache.hadoop.hive.metastore.ObjectStore.addRole (ObjectStore.java:3202)
18/03/26 16:19:51 [main]: DEBUG metastore.ObjectStore: Open transaction: count = 2, isActive = true at: org.apache.hadoop.hive.metastore.ObjectStore.getMRole (ObjectStore.java:3509)
18/03/26 16:19:51 [main]: DEBUG metastore.ObjectStore: Commit transaction: count = 1, isActive = true at: org.apache.hadoop.hive.metastore.ObjectStore.getMRole (ObjectStore.java:3515)
18/03/26 16:19:51 [main]: DEBUG metastore.ObjectStore: Rollback transaction, isActive: true at: org.apache.hadoop.hive.metastore.ObjectStore.addRole (ObjectStore.java:3215)
18/03/26 16:19:51 [main]: DEBUG metastore.HiveMetaStore: admin role already exists
InvalidObjectException (message: Role admin already exists.)
at org.apache.hadoop.hive.metastore.ObjectStore.addRole (ObjectStore.java:3205)
at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:606)
at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke (RawStoreProxy.java:98)
at com.sun.proxy. $ Proxy5.addRole (Unknown Source)
at org.apache.hadoop.hive.metastore.HiveMetaStore $ HMSHandler.createDefaultRoles_core (HiveMetaStore.java:642)
at org.apache.hadoop.hive.metastore.HiveMetaStore $ HMSHandler.createDefaultRoles (HiveMetaStore.java:631)
at org.apache.hadoop.hive.metastore.HiveMetaStore $ HMSHandler.init (HiveMetaStore.java:449)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler. <init> (RetryingHMSHandler.java:66)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy (RetryingHMSHandler.java:72)
at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler (HiveMetaStore.java:5576)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient. <init> (HiveMetaStoreClient.java:193)
at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient. <init> (SessionHiveMetaStoreClient.java:74)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0 (Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance (NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance (DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance (Constructor.java:526)
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance (MetaStoreUtils.java:1483)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient. <init> (RetryingMetaStoreClient.java:64)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy (RetryingMetaStoreClient.java:74)
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient (Hive.java:2841)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC (Hive.java:2860)
at org.apache.hadoop.hive.ql.session.SessionState.start (SessionState.java:453)
at org.apache.hadoop.hive.cli.CliDriver.run (CliDriver.java:671)
at org.apache.hadoop.hive.cli.CliDriver.main (CliDriver.java:615)
at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:606)
at org.apache.hadoop.util.RunJar.run (RunJar.java:221)
at org.apache.hadoop.util.RunJar.main (RunJar.java:136)
18/03/26 16:19:51 [main]: INFO metastore.HiveMetaStore: Added admin role in metastore
The Reason for the Exception
The log analysis reveals that the exception is caused by an attempt to add a role that already exists in the metastore. The InvalidObjectException is thrown when the addRole method is called, indicating that the role admin already exists.
Reference Materials
For further information on resolving this issue, please refer to the following resources:
- Hive create a table report [Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaExceptionMurder Caused]
- This paper involved Tencent cloud from media sharing plan, you’re reading also welcome to join and share together.
- Published on 2018-08-10
- Distributed database Hive Hadoop Report 20 share it Share the article to a circle of friends Share articles to QQ Share article on Twitter Copy the link to the article to clipboard Scan QR code Community concerns cloud scan code + Tencent cloud receive vouchers