> Linux集群 > Hadoop >

Hadoop中的SSH设置

首先hadoop使用SSH来实现cluster中各个node间的登录认证。作为一个开源框架,hadoop当然也使用开源的OpenSSH啦~

SSH utilizes standard public key cryptography to create a pair of keys for user verification—one public, one private.
The public key is stored locally on every node in the cluster, and the master node sends the private key when attempting to access a remote machine.

With both pieces of information, the target machine can validate the login attempt.

因此我们需要做的就是在每个需要通信的nodes上设置好public key。

我们可以在所有的nodes上创建一个公共的account,这个账户仅仅用于管理hadoop cluster。一旦cluster设置完毕并工作,我们可以使用其他的账户来实际运行jobs。

下面开始了~

 

1,首先需要确定,所有的node上都安装了OpenSSH。没有安装可以去OpenSSH网站上下载后安装。

[hadoop-user@master]$ which ssh
/usr/bin/ssh
[hadoop-user@master]$ which sshd
/usr/bin/sshd
[hadoop-user@master]$ which ssh-keygen
/usr/bin/ssh-keygen

2,然后创建SSH密钥对。

we use ssh keygen on the master node to generate an RSA key pair .

[hadoop-user@master]$ ssh-keygen -t rsa

会提示输入用于存储密钥的文件和passphrase。但是不要输入passphrase,直接回车,否则每次master node需要访问其他node的时候,都需要手工输入passphrase。

完成后会提示:

Your identification has been saved in /home/hadoop-user/.ssh/id_rsa.
Your public key has been saved in /home/hadoop-user/.ssh/id_rsa.pub.

 

3,将public key拷贝到每一个slave node和master node的指定目录下。

下面的命令从master node上将前面生成的id_rsa.pub文件copy到远程slave node的用户home目录下的master_key目录:

[hadoop-user@master]$ scp ~/.ssh/id_rsa.pub hadoop-user@target:~/master_key
手工login到每一个slave 目录下,将master key设置为an authorized key (or append to the list of authorized keys if you have others defined).
[hadoop-user@target]$ mkdir ~/.ssh
[hadoop-user@target]$ chmod 700 ~/.ssh
[hadoop-user@target]$ mv ~/master_key ~/.ssh/authorized_keys
[hadoop-user@target]$ chmod 600 ~/.ssh/authorized_keys

 

4,试着在master node上登陆试试:

[hadoop-user@master]$ ssh target

The authenticity of host 'target (xxx.xxx.xxx.xxx)' can’t be established.
RSA key fingerprint is 72:31:d8:1b:11:36:43:52:56:11:77:a4:ec:82:03:1d.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'target' (RSA) to the list of known hosts.
Last login: Sun Jan 4 15:32:22 2009 from master

当slave node到master node的认证通过并建立连接后,就不再需要加密解密了。直到下次再次需要建立连接。



(责任编辑:IT)