Sqoop2: Installation

(Last Updated On: )

We are going to install Sqoop. Ensure you have Hadoop installed already.

This assumes your hostname is “hadoop”

Install Java JDK

apt-get update
apt-get upgrade
apt-get install default-jdk

Download Sqoop:

wget https://archive.apache.org/dist/sqoop/1.99.7/sqoop-1.99.7-bin-hadoop200.tar.gz
tar -zxvf sqoop-1.99.7-bin-hadoop200.tar.gz
sudo mv sqoop-1.99.7-bin-hadoop200 /usr/local/sqoop/
sudo chown -R root:hadoopuser /usr/local/sqoop/

Setup .bashrc:

 sudo nano ~/.bashrc

Add the following to the end of the file.

#SQOOP VARIABLES START
export SQOOP_HOME=/usr/local/sqoop
export PATH=$PATH:$SQOOP_HOME/bin
export SQOOP_CONF_DIR=$SQOOP_HOME/conf
export SQOOP_CLASS_PATH=$SQOOP_CONF_DIR
#SQOOP VARIABLES STOP

 source ~/.bashrc

Initialise Repository

./bin/sqoop2-tool upgrade

Modify sqoop2-server

If you are running Hadoop on the same server as Sqoop Server you will need to modify this file. The reason is because Sqoop needs you to point to the lib directory for common, hdfs, mapreduce and yarn.

nano /usr/loca/sqoop/bin/sqoop.sh

#Modify these lines
  HADOOP_COMMON_HOME=${HADOOP_COMMON_HOME:-${HADOOP_HOME}/share/hadoop/common}
  HADOOP_HDFS_HOME=${HADOOP_HDFS_HOME:-${HADOOP_HOME}/share/hadoop/hdfs}
  HADOOP_MAPRED_HOME=${HADOOP_MAPRED_HOME:-${HADOOP_HOME}/share/hadoop/mapreduce}
  HADOOP_YARN_HOME=${HADOOP_YARN_HOME:-${HADOOP_HOME}/share/hadoop/yarn}

#TO

  HADOOP_COMMON_HOME=${HADOOP_HOME}/share/hadoop/common
  HADOOP_HDFS_HOME=${HADOOP_HOME}/share/hadoop/hdfs
  HADOOP_MAPRED_HOME=${HADOOP_HOME}/share/hadoop/mapreduce
  HADOOP_YARN_HOME=${HADOOP_HOME}/share/hadoop/yarn

Configuration

nano /usr/local/sqoop/conf/sqoop.properties
#Update the following line
org.apache.sqoop.submission.engine.mapreduce.configuration.directory=/usr/local/hadoop/etc/hadoop/

Start Sqoop Server

./bin/sqoop2-server start

References

https://linoxide.com/tools/install-apache-sqoop-ubuntu-16-04/