4
HADOOP INSTALLATION Download JDK and install. (jdk-6u35-linux- i586.bin) Download Hadoop and install.(hadoop-1.2.1.tar) Note: Download JDK for Linux with .bin extension. Download hadoop with .tar extension. 1.Create a directory Mkdir “directory name” 2.Change mode Chmod 777 *.* 3. ./jdk-6u35-linux-i586.bin 4. gunzip hadoop-1.2.1.tar.gz 5.tar –xvf hadoop-1.2.1.tar 6.sudo apt-get install openssh-client 7.sudo apt-get install openssh-server 8.ssh-keygen -t rsa -P “ “ 9.cat $HOME/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys 10.ssh localhost Vim install 11.sudo vi /etc/sysctl.conf net.ipv6.conf.all.disable_ipv6 =1 net.ipv6.conf.default.disable_ipv6 =1 net.ipv6.conf.io.disable_ipv6 =1 REBOOT YOUR MACHINE(changes will take place only when you reboot your linux) 12.To cross check ipv6 is disabled

HadoopInstallation.docx

  • Upload
    vkthik

  • View
    4

  • Download
    2

Embed Size (px)

DESCRIPTION

Hadoop setup guide

Citation preview

Page 1: HadoopInstallation.docx

HADOOP INSTALLATION

Download JDK and install. (jdk-6u35-linux-i586.bin)Download Hadoop and install.(hadoop-1.2.1.tar)Note:Download JDK for Linux with .bin extension.Download hadoop with .tar extension.

1.Create a directoryMkdir “directory name”2.Change modeChmod 777 *.*3. ./jdk-6u35-linux-i586.bin4. gunzip hadoop-1.2.1.tar.gz5.tar –xvf hadoop-1.2.1.tar6.sudo apt-get install openssh-client7.sudo apt-get install openssh-server8.ssh-keygen -t rsa -P “ “9.cat $HOME/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys10.ssh localhost Vim install11.sudo vi /etc/sysctl.conf

net.ipv6.conf.all.disable_ipv6 =1net.ipv6.conf.default.disable_ipv6 =1net.ipv6.conf.io.disable_ipv6 =1

REBOOT YOUR MACHINE(changes will take place only when you reboot your linux)12.To cross check ipv6 is disabledSudo cat /proc/sys/net/ipv6/conf/all/disable_ipv6Note:If 0 not disabledIf 1 disabled13.cd~

Page 2: HadoopInstallation.docx

14.vi .bashrc#setting javapathexport JAVA_HOME=/home/user/dirname/jdk1.6.0-35export Path=$JAVA_HOME/bin:$PATH#setting Hadoop pathexport HADOOP_PREFIX=/home/user/dirname/hadoop-1.2.1export PATH=$HADOOP_PREFIX/bin:$PATH15.Edit these configuration files hadoop-env.sh,mapred-site.xml,hdfs-site.xml,core-site.xml(your directory->hadoop1.2.1->conf)Hadoop-env.sh:# The java implementation to use. Required. export JAVA_HOME=/home/user/yoga/jdk1.6.0_35# Hadoop_Prefixexport HADOOP_HOME=/home/user/yoga/hadoop-1.2.1

NOTE: Replace yoga with your directory name.

mapred-site.xml:<configuration><property><name>mapred.job.tracker</name><value>hdfs://localhost:50001</value></property></configuration>

hdfs-site.xml:<configuration><property><name>dfs.replication</name><value>1</value></property>

<property><name>dfs.webhdfs.enable</name>

Page 3: HadoopInstallation.docx

<value>true</value></property>

<property><name>dfs.name.dir</name><value>/home/ubuntu/hadoop-dir/name-dir</value></property>

<property><name>dfs.data.dir</name><value>/home/ubuntu/hadoop-dir/data-dir</value></property></configuration>

core-site.xml:<configuration><!---global properties--><property><name>fs.default.name</name><value>hdfs://localhost:50000</value></property></configuration>

Cd dirname/hadoop-1.2.1/binHadoop namenode -format./start-all.shJps