Sqoop2: Installation

(Last Updated On: )

We are going to install Sqoop. Ensure you have Hadoop installed already.

This assumes your hostname is “hadoop”

Install Java JDK

  1. apt-get update
  2. apt-get upgrade
  3. apt-get install default-jdk

Download Sqoop:

  1. wget https://archive.apache.org/dist/sqoop/1.99.7/sqoop-1.99.7-bin-hadoop200.tar.gz
  2. tar -zxvf sqoop-1.99.7-bin-hadoop200.tar.gz
  3. sudo mv sqoop-1.99.7-bin-hadoop200 /usr/local/sqoop/
  4. sudo chown -R root:hadoopuser /usr/local/sqoop/

Setup .bashrc:

  1.  sudo nano ~/.bashrc

Add the following to the end of the file.

#SQOOP VARIABLES START
export SQOOP_HOME=/usr/local/sqoop
export PATH=$PATH:$SQOOP_HOME/bin
export SQOOP_CONF_DIR=$SQOOP_HOME/conf
export SQOOP_CLASS_PATH=$SQOOP_CONF_DIR
#SQOOP VARIABLES STOP

  1.  source ~/.bashrc

Initialise Repository

  1. ./bin/sqoop2-tool upgrade

Modify sqoop2-server

If you are running Hadoop on the same server as Sqoop Server you will need to modify this file. The reason is because Sqoop needs you to point to the lib directory for common, hdfs, mapreduce and yarn.

  1. nano /usr/loca/sqoop/bin/sqoop.sh
  2.  
  3. #Modify these lines
  4.   HADOOP_COMMON_HOME=${HADOOP_COMMON_HOME:-${HADOOP_HOME}/share/hadoop/common}
  5.   HADOOP_HDFS_HOME=${HADOOP_HDFS_HOME:-${HADOOP_HOME}/share/hadoop/hdfs}
  6.   HADOOP_MAPRED_HOME=${HADOOP_MAPRED_HOME:-${HADOOP_HOME}/share/hadoop/mapreduce}
  7.   HADOOP_YARN_HOME=${HADOOP_YARN_HOME:-${HADOOP_HOME}/share/hadoop/yarn}
  8.  
  9. #TO
  10.  
  11.   HADOOP_COMMON_HOME=${HADOOP_HOME}/share/hadoop/common
  12.   HADOOP_HDFS_HOME=${HADOOP_HOME}/share/hadoop/hdfs
  13.   HADOOP_MAPRED_HOME=${HADOOP_HOME}/share/hadoop/mapreduce
  14.   HADOOP_YARN_HOME=${HADOOP_HOME}/share/hadoop/yarn

Configuration

  1. nano /usr/local/sqoop/conf/sqoop.properties
  2. #Update the following line
  3. org.apache.sqoop.submission.engine.mapreduce.configuration.directory=/usr/local/hadoop/etc/hadoop/

Start Sqoop Server

  1. ./bin/sqoop2-server start

References

https://linoxide.com/tools/install-apache-sqoop-ubuntu-16-04/