Hadoop2.x下安装HBase
时间:2017-02-05 23:39 来源:linux.it.net.cn 作者:IT
环境:CentOS6.5 Hadoop2.5.2 HBase1.0.0
1.安装好 hadoop 集群,并启动
[grid@hadoop4 ~]$ sh hadoop-2.5.2/sbin/start-dfs.sh
[grid@hadoop4 ~]$ sh hadoop-2.5.2/sbin/start-yarn.sh
查看 hadoop 版本:
[grid@hadoop4 ~]$ hadoop-2.5.2/bin/hadoop version
Hadoop 2.5.2
Subversion Unknown -r Unknown
Compiled by grid on 2014-12-31T01:40Z
Compiled with protoc 2.5.0
From source with checksum df7537a4faa4658983d397abf4514320
This command was run using /home/grid/hadoop-2.5.2/share/hadoop/common/hadoop-common-2.5.2.jar
2.查看 hbase 官方文档(http://hbase.apache.org/book.html#basic.prerequisites),找到与 hadoop 版本对应的 hbase 并下载
[grid@hadoop4 ~]$ wget http://mirrors.cnnic.cn/apache/hbase/hbase-1.0.0/hbase-1.0.0-bin.tar.gz
3.解压
[grid@hadoop4 ~]$ tar -zxf hbase-1.0.0-bin.tar.gz
4.进入 hbase 的 lib 目录,查看 hadoop jar 包的版本
[grid@hadoop4 ~]$ cd hbase-1.0.0/lib/
[grid@hadoop4 lib]$ find -name 'hadoop*jar'
./hadoop-common-2.5.1.jar
./hadoop-mapreduce-client-common-2.5.1.jar
./hadoop-annotations-2.5.1.jar
./hadoop-yarn-server-common-2.5.1.jar
./hadoop-hdfs-2.5.1.jar
./hadoop-client-2.5.1.jar
./hadoop-mapreduce-client-shuffle-2.5.1.jar
./hadoop-yarn-common-2.5.1.jar
./hadoop-yarn-server-nodemanager-2.5.1.jar
./hadoop-yarn-client-2.5.1.jar
./hadoop-mapreduce-client-core-2.5.1.jar
./hadoop-auth-2.5.1.jar
./hadoop-mapreduce-client-app-2.5.1.jar
./hadoop-yarn-api-2.5.1.jar
./hadoop-mapreduce-client-jobclient-2.5.1.jar
发现与 hadoop 集群的版本号不一致,需要用 hadoop 目录下的 jar 替换 hbase/lib 目录下的 jar 文件。
编写脚本来完成替换,如下所示:
[grid@hadoop4 lib]$ pwd
/home/grid/hbase-1.0.0/lib
[grid@hadoop4 lib]$ vim f.sh
find -name "hadoop*jar" | sed 's/2.5.1/2.5.2/g' | sed 's/\.\///g' > f.log
rm ./hadoop*jar
cat ./f.log | while read Line
do
find /home/grid/hadoop-2.5.2 -name "$Line" | xargs -i cp {} ./
done
rm ./f.log
[grid@hadoop4 lib]$ chmod u+x f.sh
[grid@hadoop4 lib]$ ./f.sh
[grid@hadoop4 lib]$ find -name 'hadoop*jar'
./hadoop-yarn-api-2.5.2.jar
./hadoop-mapreduce-client-app-2.5.2.jar
./hadoop-common-2.5.2.jar
./hadoop-mapreduce-client-jobclient-2.5.2.jar
./hadoop-mapreduce-client-core-2.5.2.jar
./hadoop-yarn-server-nodemanager-2.5.2.jar
./hadoop-hdfs-2.5.2.jar
./hadoop-yarn-common-2.5.2.jar
./hadoop-mapreduce-client-shuffle-2.5.2.jar
./hadoop-auth-2.5.2.jar
./hadoop-mapreduce-client-common-2.5.2.jar
./hadoop-yarn-client-2.5.2.jar
./hadoop-annotations-2.5.2.jar
./hadoop-yarn-server-common-2.5.2.jar
OK,jar 包替换成功;hbase/lib 目录下还有个 slf4j-log4j12-XXX.jar,在机器有装hadoop时,由于classpath中会有hadoop中的这个jar包,会有冲突,直接删除掉
[grid@hadoop4 lib]$ rm `find -name 'slf4j-log4j12-*jar'`
5.修改配置文件
5.1.
[grid@hadoop4 hbase-1.0.0]$ vi conf/hbase-env.sh
export JAVA_HOME=/usr/java/jdk1.7.0_72
export HBASE_CLASSPATH=/home/grid/hadoop-2.5.2/etc/hadoop
export HBASE_MANAGES_ZK=true
第一个参数指定了JDK路径;第二个参数指定了 hadoop 的配置文件路径;第三个参数设置使用 hbase 默认自带的 Zookeeper
5.2.
[grid@hadoop4 hbase-1.0.0]$ vim conf/hbase-site.xml
<property>
<name>hbase.rootdir</name>
<value>hdfs://hadoop4:9000/hbase</value>
</property>
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
</property>
<property>
<name>hbase.tmp.dir</name>
<value>/home/grid/hbase-1.0.0/tmp</value>
</property>
<property>
<name>hbase.zookeeper.quorum</name>
<value>hadoop4,hadoop5,hadoop6</value>
</property>
<property>
<name>hbase.zookeeper.property.dataDir</name>
<value>/home/grid/hbase-1.0.0/zookeeper</value>
</property>
创建目录
[grid@hadoop4 hbase-1.0.0]$ mkdir tmp
[grid@hadoop4 hbase-1.0.0]$ mkdir zookeeper
5.3.
[grid@hadoop4 hbase-1.0.0]$ vim conf/regionservers
hadoop4
hadoop5
hadoop6
6.设置环境变量
[grid@hadoop4 ~]$ vi .bash_profile
export HBASE_HOME=/home/grid/hbase-1.0.0
export PATH=$PATH:$HBASE_HOME/bin
[grid@hadoop4 ~]$ source .bash_profile
7.分发 hbase 到其它机器,并在其上设置环境变量
[grid@hadoop4 ~]$ scp -r hbase-1.0.0 grid@hadoop5:~
[grid@hadoop4 ~]$ scp -r hbase-1.0.0 grid@hadoop6:~
8.启动 hbase
[grid@hadoop4 ~]$ sh start-hbase.sh
[grid@hadoop4 ~]$ jps
2388 ResourceManager
3692 Jps
2055 NameNode
3375 HQuorumPeer
2210 SecondaryNameNode
3431 HMaster
[grid@hadoop5 ~]$ jps
2795 Jps
2580 HQuorumPeer
2656 HRegionServer
2100 NodeManager
1983 DataNode
[grid@hadoop6 ~]$ jps
2566 HQuorumPeer
1984 DataNode
2101 NodeManager
2803 Jps
2639 HRegionServer
jps查看发现 Master 机上 HRegionServer 服务未启动,查看日志显示因16020端口被占用导致 HRegionServer 启动失败,查证发现占用16020端口的是 HMaster 进程,查看官方文档后解决:[grid@hadoop4 ~]$ sh local-regionservers.sh start 2
官方文档截图:
9.shell
10.Web管理界面
(责任编辑:IT)
环境:CentOS6.5 Hadoop2.5.2 HBase1.0.0 1.安装好 hadoop 集群,并启动 [grid@hadoop4 ~]$ sh hadoop-2.5.2/sbin/start-dfs.sh [grid@hadoop4 ~]$ sh hadoop-2.5.2/sbin/start-yarn.sh 查看 hadoop 版本: [grid@hadoop4 ~]$ hadoop-2.5.2/bin/hadoop version Hadoop 2.5.2 Subversion Unknown -r Unknown Compiled by grid on 2014-12-31T01:40Z Compiled with protoc 2.5.0 From source with checksum df7537a4faa4658983d397abf4514320 This command was run using /home/grid/hadoop-2.5.2/share/hadoop/common/hadoop-common-2.5.2.jar 2.查看 hbase 官方文档(http://hbase.apache.org/book.html#basic.prerequisites),找到与 hadoop 版本对应的 hbase 并下载 [grid@hadoop4 ~]$ wget http://mirrors.cnnic.cn/apache/hbase/hbase-1.0.0/hbase-1.0.0-bin.tar.gz 3.解压 [grid@hadoop4 ~]$ tar -zxf hbase-1.0.0-bin.tar.gz 4.进入 hbase 的 lib 目录,查看 hadoop jar 包的版本 [grid@hadoop4 ~]$ cd hbase-1.0.0/lib/ [grid@hadoop4 lib]$ find -name 'hadoop*jar' ./hadoop-common-2.5.1.jar ./hadoop-mapreduce-client-common-2.5.1.jar ./hadoop-annotations-2.5.1.jar ./hadoop-yarn-server-common-2.5.1.jar ./hadoop-hdfs-2.5.1.jar ./hadoop-client-2.5.1.jar ./hadoop-mapreduce-client-shuffle-2.5.1.jar ./hadoop-yarn-common-2.5.1.jar ./hadoop-yarn-server-nodemanager-2.5.1.jar ./hadoop-yarn-client-2.5.1.jar ./hadoop-mapreduce-client-core-2.5.1.jar ./hadoop-auth-2.5.1.jar ./hadoop-mapreduce-client-app-2.5.1.jar ./hadoop-yarn-api-2.5.1.jar ./hadoop-mapreduce-client-jobclient-2.5.1.jar 发现与 hadoop 集群的版本号不一致,需要用 hadoop 目录下的 jar 替换 hbase/lib 目录下的 jar 文件。 编写脚本来完成替换,如下所示: [grid@hadoop4 lib]$ pwd /home/grid/hbase-1.0.0/lib [grid@hadoop4 lib]$ vim f.sh find -name "hadoop*jar" | sed 's/2.5.1/2.5.2/g' | sed 's/\.\///g' > f.log rm ./hadoop*jar cat ./f.log | while read Line do find /home/grid/hadoop-2.5.2 -name "$Line" | xargs -i cp {} ./ done rm ./f.log [grid@hadoop4 lib]$ chmod u+x f.sh [grid@hadoop4 lib]$ ./f.sh [grid@hadoop4 lib]$ find -name 'hadoop*jar' ./hadoop-yarn-api-2.5.2.jar ./hadoop-mapreduce-client-app-2.5.2.jar ./hadoop-common-2.5.2.jar ./hadoop-mapreduce-client-jobclient-2.5.2.jar ./hadoop-mapreduce-client-core-2.5.2.jar ./hadoop-yarn-server-nodemanager-2.5.2.jar ./hadoop-hdfs-2.5.2.jar ./hadoop-yarn-common-2.5.2.jar ./hadoop-mapreduce-client-shuffle-2.5.2.jar ./hadoop-auth-2.5.2.jar ./hadoop-mapreduce-client-common-2.5.2.jar ./hadoop-yarn-client-2.5.2.jar ./hadoop-annotations-2.5.2.jar ./hadoop-yarn-server-common-2.5.2.jar OK,jar 包替换成功;hbase/lib 目录下还有个 slf4j-log4j12-XXX.jar,在机器有装hadoop时,由于classpath中会有hadoop中的这个jar包,会有冲突,直接删除掉 [grid@hadoop4 lib]$ rm `find -name 'slf4j-log4j12-*jar'`
[grid@hadoop4 hbase-1.0.0]$ vi conf/hbase-env.sh export JAVA_HOME=/usr/java/jdk1.7.0_72 export HBASE_CLASSPATH=/home/grid/hadoop-2.5.2/etc/hadoop export HBASE_MANAGES_ZK=true 第一个参数指定了JDK路径;第二个参数指定了 hadoop 的配置文件路径;第三个参数设置使用 hbase 默认自带的 Zookeeper 5.2. [grid@hadoop4 hbase-1.0.0]$ vim conf/hbase-site.xml <property> <name>hbase.rootdir</name> <value>hdfs://hadoop4:9000/hbase</value> </property> <property> <name>hbase.cluster.distributed</name> <value>true</value> </property> <property> <name>hbase.tmp.dir</name> <value>/home/grid/hbase-1.0.0/tmp</value> </property> <property> <name>hbase.zookeeper.quorum</name> <value>hadoop4,hadoop5,hadoop6</value> </property> <property> <name>hbase.zookeeper.property.dataDir</name> <value>/home/grid/hbase-1.0.0/zookeeper</value> </property> 创建目录 [grid@hadoop4 hbase-1.0.0]$ mkdir tmp[grid@hadoop4 hbase-1.0.0]$ mkdir zookeeper 5.3. [grid@hadoop4 hbase-1.0.0]$ vim conf/regionservers hadoop4 hadoop5 hadoop6 6.设置环境变量 [grid@hadoop4 ~]$ vi .bash_profile export HBASE_HOME=/home/grid/hbase-1.0.0 export PATH=$PATH:$HBASE_HOME/bin [grid@hadoop4 ~]$ source .bash_profile 7.分发 hbase 到其它机器,并在其上设置环境变量 [grid@hadoop4 ~]$ scp -r hbase-1.0.0 grid@hadoop5:~ [grid@hadoop4 ~]$ scp -r hbase-1.0.0 grid@hadoop6:~ 8.启动 hbase [grid@hadoop4 ~]$ sh start-hbase.sh [grid@hadoop4 ~]$ jps 2388 ResourceManager 3692 Jps 2055 NameNode 3375 HQuorumPeer 2210 SecondaryNameNode 3431 HMaster [grid@hadoop5 ~]$ jps 2795 Jps 2580 HQuorumPeer 2656 HRegionServer 2100 NodeManager 1983 DataNode [grid@hadoop6 ~]$ jps 2566 HQuorumPeer 1984 DataNode 2101 NodeManager 2803 Jps 2639 HRegionServer jps查看发现 Master 机上 HRegionServer 服务未启动,查看日志显示因16020端口被占用导致 HRegionServer 启动失败,查证发现占用16020端口的是 HMaster 进程,查看官方文档后解决:[grid@hadoop4 ~]$ sh local-regionservers.sh start 2 官方文档截图: 9.shell 10.Web管理界面 (责任编辑:IT) |