Home / ubuntu / Questions / 1073035
Accepted
SriniShine
Asked: 2018-09-08 02:04:44 +0800 CST

FAILED: HiveException java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient

  • 772

I'm trying to run Hive 3.1 with Hadoop 3.0. Following is my system configuration:

Ubuntu 18.04.1 LTS
Hadoop  version 3.0.3
Hive 3.1.0
Derby 10.14.2

When I execute the show tables; query I get the following error.

FAILED: HiveException java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient

Following is the detail error in the hive log file.

2018-09-05T11:38:25,952  INFO [main] conf.HiveConf: Found configuration file file:/usr/local/apache-hive-3.1.0-bin/conf/hive-site.xml
2018-09-05T11:38:30,549  INFO [main] SessionState: Hive Session ID = 826ec55c-7fca-4fff-baa5-b5a010e5af89
2018-09-05T11:38:35,948  INFO [main] SessionState:
Logging initialized using configuration in jar:file:/usr/local/apache-hive-3.1.0-bin/lib/hive-common-3.1.0.jar!/hive-log4j2.properties Asy$
2018-09-05T11:38:47,015  INFO [main] session.SessionState: Created HDFS directory: /tmp/hive/hadoop
2018-09-05T11:38:47,069  INFO [main] session.SessionState: Created local directory: /tmp/mydir
2018-09-05T11:38:47,096  INFO [main] session.SessionState: Created HDFS directory: /tmp/hive/hadoop/826ec55c-7fca-4fff-baa5-b5a010e5af89
2018-09-05T11:38:47,104  INFO [main] session.SessionState: Created local directory: /tmp/mydir/826ec55c-7fca-4fff-baa5-b5a010e5af89
2018-09-05T11:38:47,122  INFO [main] session.SessionState: Created HDFS directory: /tmp/hive/hadoop/826ec55c-7fca-4fff-baa5-b5a010e5af89/_$
2018-09-05T11:38:47,125  INFO [main] conf.HiveConf: Using the default value passed in for log id: 826ec55c-7fca-4fff-baa5-b5a010e5af89
2018-09-05T11:38:47,126  INFO [main] session.SessionState: Updating thread name to 826ec55c-7fca-4fff-baa5-b5a010e5af89 main
2018-09-05T11:38:50,476  INFO [826ec55c-7fca-4fff-baa5-b5a010e5af89 main] metastore.HiveMetaStore: 0: Opening raw store with implementatio$
2018-09-05T11:38:50,695  WARN [826ec55c-7fca-4fff-baa5-b5a010e5af89 main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set$
2018-09-05T11:38:50,714  INFO [826ec55c-7fca-4fff-baa5-b5a010e5af89 main] metastore.ObjectStore: ObjectStore, initialize called
2018-09-05T11:38:50,717  INFO [826ec55c-7fca-4fff-baa5-b5a010e5af89 main] conf.MetastoreConf: Found configuration file file:/usr/local/apa$
2018-09-05T11:38:50,719  INFO [826ec55c-7fca-4fff-baa5-b5a010e5af89 main] conf.MetastoreConf: Unable to find config file hivemetastore-sit$
2018-09-05T11:38:50,720  INFO [826ec55c-7fca-4fff-baa5-b5a010e5af89 main] conf.MetastoreConf: Found configuration file null
2018-09-05T11:38:50,722  INFO [826ec55c-7fca-4fff-baa5-b5a010e5af89 main] conf.MetastoreConf: Unable to find config file metastore-site.xml
2018-09-05T11:38:50,722  INFO [826ec55c-7fca-4fff-baa5-b5a010e5af89 main] conf.MetastoreConf: Found configuration file null

hive-site.xml

<property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:derby:;databaseName=metastore_db;create=true</value>
    <description>
      JDBC connect string for a JDBC metastore.
      To use SSL to encrypt/authenticate the connection, provide database-specific SSL flag in the connection URL.
      For example, jdbc:postgresql://myhost/db?ssl=true for postgres database.
    </description>
  </property>

Environment variables in .profile (I am trying to configure an installation done by someone else so the environment variables are set up in .profile instead of .bashrc even though Hadoop is run manually)

#HIVE
export HIVE_HOME=/usr/local/apache-hive-3.1.0-bin
export HIVE_CONF_DIR=/usr/local/apache-hive-3.1.0-bin/conf
export PATH=$HIVE_HOME/bin:$PATH
export CLASSPATH=$CLASSPATH:/usr/local/hadoop/lib/*:.
export CLASSPATH=$CLASSPATH:/usr/local/apache-hive-3.1.0-bin/lib/*:.

#DERBY
DERBY_HOME=/usr/local/db-derby-10.14.2.0-bin
export PATH=$PATH:$DERBY_HOME/bin
export CLASSPATH=$CLASSPATH:$DERBY_HOME/lib/derby.jar:$DERBY_HOME/lib/derbytool$

The error messages indicates about a metastore-site.xml file which I did not come across the configuration of hive.

18.04 hadoop hive

1 Answers

  1. Best Answer

    I think Hive expect to find Derby database in current working directory. Hive will create the database, because create=true in javax.jdo.option.ConnectionURL property, but will not initialize schema in the database.

    My configuration:

    • Ubuntu 18.04.1 LTS
    • Hadoop 3.1.1
    • Hive 3.1.0

    • Apache Derby from Ubuntu repositories (Version: 10.14.1.0-1ubuntu1 currently)

      sudo apt install derby-tools libderby-java libderbyclient-java

    According Hive Documentation we need to run ShemaTool explicity.

    I decide to keep my Derby database into HIVE_HOME (/opt/hive-3.1.0-bin in my case) directory. You can setup Derby database location in hive-site.xml:

    <property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:derby:/opt/hive-3.1.0-bin/metastore_db;databaseName=metastore_db;create=true</value>
</property>

    Ensure that there is no Derby database yet:

    $ ls $HIVE_HOME/meta*
ls: cannot access '/opt/hive-3.1.0-bin/meta*': No such file or directory

    Start Hive:

    $ pwd
/home/hadoop
$ hive
hive> show tables;
FAILED: HiveException java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
hive> exit;

    Now we got Derby database, but it's not functional:

    $ ls -d $HIVE_HOME/meta*
/opt/hive-3.1.0-bin/metastore_db

    So, we need to create metada schema:

    $ rm -rf $HIVE_HOME/metastore_db
$ cd $HIVE_HOME
$ schematool -initSchema -dbType derby
...
Initialization script completed
schemaTool completed

    Now Hive is working as expected:

    $ cd
$ hive
hive> show tables;
OK
Time taken: 0.803 seconds
    • 1