Install Hive on Ubuntu: Step-by-Step Guide

This guide explains how to install Apache Hive on Ubuntu, covering prerequisites, installation steps, configuration, and common troubleshooting. You will learn to set up Hive with MySQL as the metastore and verify the installation with basic queries.

Prerequisites

Before installing Hive, your Ubuntu system must have Java (JDK 8 or 11), Hadoop (2.x or 3.x), and MySQL or MariaDB (for the metastore). Hive is a data warehouse tool that runs on top of Hadoop, so Hadoop must be installed and running. Check versions with java -version, hadoop version, and mysql --version. If any is missing, install them first. For Java, use sudo apt install openjdk-8-jdk or openjdk-11-jdk. For Hadoop, download a stable release from the Apache website and set HADOOP_HOME in ~/.bashrc.

Downloading and Extracting Hive

Download the latest stable Hive release (e.g., 3.1.3) from the Apache Hive website. Use the binary tarball, not the source. Extract it to a directory like /opt/hive:

1. wget https://downloads.apache.org/hive/hive-3.1.3/apache-hive-3.1.3-bin.tar.gz
2. sudo tar -xzf apache-hive-3.1.3-bin.tar.gz -C /opt
3. sudo mv /opt/apache-hive-3.1.3-bin /opt/hive

Set environment variables by adding these lines to ~/.bashrc:

export HIVE_HOME=/opt/hive
export PATH=$HIVE_HOME/bin:$PATH

Then run source ~/.bashrc.

Configuring the Metastore

Hive uses a metastore to store schema information. By default, it uses an embedded Derby database, which is not suitable for production. For a reliable setup, use MySQL. Create a database and user for Hive:

mysql -u root -p
CREATE DATABASE metastore;
CREATE USER 'hive'@'localhost' IDENTIFIED BY 'your_password';
GRANT ALL PRIVILEGES ON metastore.* TO 'hive'@'localhost';
FLUSH PRIVILEGES;
EXIT;

Edit $HIVE_HOME/conf/hive-site.xml (create it if missing) with the following properties:

<configuration>
  <property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:mysql://localhost/metastore?createDatabaseIfNotExist=true</value>
  </property>
  <property>
    <name>javax.jdo.option.ConnectionDriverName</name>
    <value>com.mysql.jdbc.Driver</value>
  </property>
  <property>
    <name>javax.jdo.option.ConnectionUserName</name>
    <value>hive</value>
  </property>
  <property>
    <name>javax.jdo.option.ConnectionPassword</name>
    <value>your_password</value>
  </property>
</configuration>

Download the MySQL JDBC driver (Connector/J) and place it in $HIVE_HOME/lib.

Initializing the Schema

Run the schema initialization command to create the required tables in the metastore:

$HIVE_HOME/bin/schematool -initSchema -dbType mysql

This command reads the configuration from hive-site.xml and creates tables. If it succeeds, you will see “Initialization script completed”. If you encounter errors, check the MySQL connection details and ensure the JDBC driver is present.

Starting Hive and Testing

Start Hadoop services if they are not already running (start-dfs.sh and start-yarn.sh). Then launch the Hive CLI:

hive

You should see the Hive prompt. Test with a simple query:

CREATE TABLE test (id INT, name STRING);
INSERT INTO test VALUES (1, 'Alice');
SELECT * FROM test;

If the query returns the row, Hive is working. You can also run HiveServer2 for remote connections using hive --service hiveserver2 and connect via Beeline.

Common Problems and Fixes

• Metastore connection failure: Verify MySQL is running and the JDBC driver is in $HIVE_HOME/lib. Also check that the password and username are correct.
• Java version issues: Hive 3.x requires Java 8 or 11. If you have a newer version, install an older JDK and set JAVA_HOME.
• Hadoop not in PATH: Hive needs HADOOP_HOME set. Add export HADOOP_HOME=/path/to/hadoop to ~/.bashrc.
• Permission denied on HDFS: Create the Hive warehouse directory and set permissions: hdfs dfs -mkdir -p /user/hive/warehouse and hdfs dfs -chmod g+w /user/hive/warehouse.
• Derby lock errors: If you previously used the embedded metastore, delete the derby.log and metastore_db directory in your home folder.

Frequently Asked Questions

Can I install Hive without Hadoop?

No. Hive is built on Hadoop and uses HDFS for storage and YARN for execution. You must have Hadoop installed and running.

What is the difference between Hive and MySQL?

Hive is a data warehouse for querying large datasets stored in HDFS using SQL-like language (HiveQL). MySQL is a relational database for transactional workloads. Hive is not suitable for real-time queries or updates.

How do I uninstall Hive?

Remove the Hive directory (sudo rm -rf /opt/hive), delete the metastore database in MySQL (DROP DATABASE metastore;), and remove the environment variables from ~/.bashrc.

Why is Hive slow?

Hive is designed for batch processing. It converts queries to MapReduce or Tez jobs, which have high latency. For faster queries, use Tez or Spark as the execution engine, and partition or bucket tables.

Can I use PostgreSQL instead of MySQL for the metastore?

Yes, Hive supports PostgreSQL. You need to change the connection URL and driver in hive-site.xml and run schematool -initSchema -dbType postgres.

Related

• iTunes Error Code 50 on Windows: Temporary Problem?
• No SIM Card Detected Android: How to Fix It
• Change Touchpad Sensitivity in Windows