[one-users] SSH key exchange failing for InM

Jaime Melis jmelis at opennebula.org
Mon Aug 5 06:46:23 PDT 2013 

Hi Pierre,

it looks like it could be an environment issue. Can you stop OpenNebula and
start it with the 'one start' command in the oneadmin account, instead of
using the init script?

Maybe there's something there that is not inheriting properly.

cheers,
Jaime


On Mon, Aug 5, 2013 at 6:59 AM, Olivier Sallou <olivier.sallou at irisa.fr>wrote:

>
> On 08/05/2013 12:40 PM, Pierre Naude wrote:
>
>   Hi Olivier,
>
>  No - as per the docs the key is not password protected.
>
>  Also neither of the systems are configured to use ssh-agent
> (ForwardAgent is set to no and SSH_AUTH_SOCK never gets set).
>
>  From the command line it works whether I force the key or not:
>
> [oneadmin at rtfwops1 ~]$ ssh -v -i /var/lib/one/.ssh/id_dsa
> oneadmin at rtfwops2
> OpenSSH_5.3p1, OpenSSL 1.0.0-fips 29 Mar 2010
> debug1: Reading configuration data /etc/ssh/ssh_config
> debug1: Applying options for *
> debug1: Connecting to rtfwops2 [xxx.xxx.xxx.138] port 22.
> debug1: Connection established.
> debug1: identity file /var/lib/one/.ssh/id_dsa type 2
> debug1: Remote protocol version 2.0, remote software version OpenSSH_5.3
> debug1: match: OpenSSH_5.3 pat OpenSSH*
> debug1: Enabling compatibility mode for protocol 2.0
> debug1: Local version string SSH-2.0-OpenSSH_5.3
> debug1: SSH2_MSG_KEXINIT sent
> debug1: SSH2_MSG_KEXINIT received
> debug1: kex: server->client aes128-ctr hmac-md5 none
> debug1: kex: client->server aes128-ctr hmac-md5 none
> debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent
> debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP
> debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
> debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY
> debug1: Host 'rtfwops2' is known and matches the RSA host key.
> debug1: Found key in /var/lib/one/.ssh/known_hosts:1
> debug1: ssh_rsa_verify: signature correct
> debug1: SSH2_MSG_NEWKEYS sent
> debug1: expecting SSH2_MSG_NEWKEYS
> debug1: SSH2_MSG_NEWKEYS received
> debug1: SSH2_MSG_SERVICE_REQUEST sent
> debug1: SSH2_MSG_SERVICE_ACCEPT received
> debug1: Authentications that can continue:
> publickey,gssapi-keyex,gssapi-with-mic,password
> debug1: Next authentication method: publickey
> debug1: Offering public key: /var/lib/one/.ssh/id_dsa
> debug1: Server accepts key: pkalg ssh-dss blen 434
> debug1: read PEM private key done: type DSA
> debug1: Authentication succeeded (publickey).
> debug1: channel 0: new [client-session]
> debug1: Requesting no-more-sessions at openssh.com
> debug1: Entering interactive session.
> debug1: Sending environment.
> debug1: Sending env LANG = en_US.UTF-8
> Last login: Mon Aug  5 11:37:43 2013 from xxx.xxx.xxx.138
> [oneadmin at rtfwops2 ~]$ debug1: client_input_channel_req: channel 0 rtype
> exit-status reply 0
> debug1: client_input_channel_req: channel 0 rtype eow at openssh.com reply 0
> debug1: channel 0: forcing write
> logout
> debug1: channel 0: free: client-session, nchannels 1
> Connection to rtfwops2 closed.
> Transferred: sent 2992, received 3064 bytes, in 25.6 seconds
> Bytes per second: sent 116.7, received 119.5
> debug1: Exit status 0
>
> [oneadmin at rtfwops1 ~]$ ssh -v oneadmin at rtfwops2
> OpenSSH_5.3p1, OpenSSL 1.0.0-fips 29 Mar 2010
> debug1: Reading configuration data /etc/ssh/ssh_config
> debug1: Applying options for *
> debug1: Connecting to rtfwops2 [xxx.xxx.xxx.138] port 22.
> debug1: Connection established.
> debug1: identity file /var/lib/one/.ssh/identity type -1
> debug1: identity file /var/lib/one/.ssh/id_rsa type -1
> debug1: identity file /var/lib/one/.ssh/id_dsa type 2
> debug1: Remote protocol version 2.0, remote software version OpenSSH_5.3
> debug1: match: OpenSSH_5.3 pat OpenSSH*
> debug1: Enabling compatibility mode for protocol 2.0
> debug1: Local version string SSH-2.0-OpenSSH_5.3
> debug1: SSH2_MSG_KEXINIT sent
> debug1: SSH2_MSG_KEXINIT received
> debug1: kex: server->client aes128-ctr hmac-md5 none
> debug1: kex: client->server aes128-ctr hmac-md5 none
> debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent
> debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP
> debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
> debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY
> debug1: Host 'rtfwops2' is known and matches the RSA host key.
>
>
> This is gine for host match here, but not in your previous log:
> "Mon Aug  5 11:48:10 2013 [InM][I]: Host key verification failed."
> Could be known_host issue but it should fail via command line too.
>
>  debug1: Found key in /var/lib/one/.ssh/known_hosts:1
> debug1: ssh_rsa_verify: signature correct
> debug1: SSH2_MSG_NEWKEYS sent
> debug1: expecting SSH2_MSG_NEWKEYS
> debug1: SSH2_MSG_NEWKEYS received
> debug1: SSH2_MSG_SERVICE_REQUEST sent
> debug1: SSH2_MSG_SERVICE_ACCEPT received
> debug1: Authentications that can continue:
> publickey,gssapi-keyex,gssapi-with-mic,password
> debug1: Next authentication method: publickey
> debug1: Trying private key: /var/lib/one/.ssh/identity
> debug1: Trying private key: /var/lib/one/.ssh/id_rsa
> debug1: Offering public key: /var/lib/one/.ssh/id_dsa
> debug1: Server accepts key: pkalg ssh-dss blen 434
> debug1: read PEM private key done: type DSA
> debug1: Authentication succeeded (publickey).
> debug1: channel 0: new [client-session]
> debug1: Requesting no-more-sessions at openssh.com
> debug1: Entering interactive session.
> debug1: Sending environment.
> debug1: Sending env LANG = en_US.UTF-8
> Last login: Mon Aug  5 12:21:57 2013 from xxx.xxx.xxx.137
> [oneadmin at rtfwops2 ~]$
>
>  HTH
>
>  Pierre
>
>
>
>
> On 5 August 2013 12:19, Olivier Sallou <olivier.sallou at irisa.fr> wrote:
>
>>
>> On 08/05/2013 11:59 AM, Pierre Naude wrote:
>>
>>  Good Morning,
>>
>> I'm busy setting up a proof-of-concept using ONE and have run into a
>> problem adding hosts to the server.
>>
>>  My ONE server is a Centos 6.4 installation, and so is the host I'm
>> adding to the server.
>>
>>  I am able to ssh successfully without password from the server to the
>> host as root and oneadmin and vice versa (I have also made sure the servers
>> can connect to themselves without password).
>>
>>  The problem is that the one server monitoring process is failing to ssh
>> passwordlessly from the server to the host:
>>
>>  Debug from the server:
>>
>> Mon Aug  5 11:48:10 2013 [InM][I]: Monitoring host rtfwops2.rorotika (7)
>> Mon Aug  5 11:48:10 2013 [InM][I]: Command execution fail: 'if [ -x
>> "/var/tmp/one/im/run_probes" ]; then
>>  /var/tmp/one/im/run_probes kvm 7 rtfwops2.rorotika;
>> else                              exit 42; fi'
>> Mon Aug  5 11:48:10 2013 [InM][I]: debug1: Connecting to
>> rtfwops2.rorotika [xxx.xxx.xxx.138] port 22.
>> Mon Aug  5 11:48:10 2013 [InM][I]: debug1: Connection established.
>> Mon Aug  5 11:48:10 2013 [InM][I]: debug1: identity file
>> /var/lib/one/.ssh/identity type -1
>> Mon Aug  5 11:48:10 2013 [InM][I]: debug1: identity file
>> /var/lib/one/.ssh/id_rsa type -1
>> Mon Aug  5 11:48:10 2013 [InM][I]: debug1: identity file
>> /var/lib/one/.ssh/id_dsa type 2
>> Mon Aug  5 11:48:10 2013 [InM][I]: debug1: Remote protocol version 2.0,
>> remote software version OpenSSH_5.3
>> Mon Aug  5 11:48:10 2013 [InM][I]: debug1: match: OpenSSH_5.3 pat OpenSSH*
>> Mon Aug  5 11:48:10 2013 [InM][I]: debug1: Enabling compatibility mode
>> for protocol 2.0
>> Mon Aug  5 11:48:10 2013 [InM][I]: debug1: Local version string
>> SSH-2.0-OpenSSH_5.3
>> Mon Aug  5 11:48:10 2013 [InM][I]: debug1: SSH2_MSG_KEXINIT sent
>> Mon Aug  5 11:48:10 2013 [InM][I]: debug1: SSH2_MSG_KEXINIT received
>> Mon Aug  5 11:48:10 2013 [InM][I]: debug1: kex: server->client aes128-ctr
>> hmac-md5 none
>> Mon Aug  5 11:48:10 2013 [InM][I]: debug1: kex: client->server aes128-ctr
>> hmac-md5 none
>> Mon Aug  5 11:48:10 2013 [InM][I]: debug1:
>> SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent
>> Mon Aug  5 11:48:10 2013 [InM][I]: debug1: expecting
>> SSH2_MSG_KEX_DH_GEX_GROUP
>> Mon Aug  5 11:48:10 2013 [InM][I]: debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
>> Mon Aug  5 11:48:10 2013 [InM][I]: debug1: expecting
>> SSH2_MSG_KEX_DH_GEX_REPLY
>> Mon Aug  5 11:48:10 2013 [InM][I]: debug1: read_passphrase: can't open
>> /dev/tty: No such device or address
>>
>>  It seems it expects to get your passphrase here. I think your key is
>> password protected (and this is fine).
>> When you made your connection tests, are you sure you used the oneadmin
>> user key (and not one loaded via ssh-agent or something like that) ?
>>
>> Olivier
>>
>>    Mon Aug  5 11:48:10 2013 [InM][I]: Host key verification failed.
>> Mon Aug  5 11:48:10 2013 [InM][I]: ExitCode: 255
>> Mon Aug  5 11:48:10 2013 [ONE][E]: Error monitoring Host
>> rtfwops2.rorotika (7): -
>>
>>  Debug from the host:
>>
>> Aug  5 11:48:10 rtfwops2 sshd[2301]: debug1: Forked child 11777.
>> Aug  5 11:48:10 rtfwops2 sshd[11777]: Set /proc/self/oom_score_adj to 0
>> Aug  5 11:48:10 rtfwops2 sshd[11777]: debug1: rexec start in 5 out 5
>> newsock 5 pipe 7 sock 8
>> Aug  5 11:48:10 rtfwops2 sshd[11777]: debug1: inetd sockets after
>> dupping: 3, 3
>> Aug  5 11:48:10 rtfwops2 sshd[11777]: Connection from 172.28.200.137 port
>> 52989
>> Aug  5 11:48:10 rtfwops2 sshd[11777]: debug1: Client protocol version
>> 2.0; client software version Open
>> SSH_5.3
>> Aug  5 11:48:10 rtfwops2 sshd[11777]: debug1: match: OpenSSH_5.3 pat
>> OpenSSH*
>> Aug  5 11:48:10 rtfwops2 sshd[11777]: debug1: Enabling compatibility mode
>> for protocol 2.0
>> Aug  5 11:48:10 rtfwops2 sshd[11777]: debug1: Local version string
>> SSH-2.0-OpenSSH_5.3
>> Aug  5 11:48:10 rtfwops2 sshd[11778]: debug1: permanently_set_uid: 74/74
>> Aug  5 11:48:10 rtfwops2 sshd[11778]: debug1: list_hostkey_types:
>> ssh-rsa,ssh-dss
>> Aug  5 11:48:10 rtfwops2 sshd[11778]: debug1: SSH2_MSG_KEXINIT sent
>> Aug  5 11:48:10 rtfwops2 sshd[11778]: debug1: SSH2_MSG_KEXINIT received
>> Aug  5 11:48:10 rtfwops2 sshd[11778]: debug1: kex: client->server
>> aes128-ctr hmac-md5 none
>> Aug  5 11:48:10 rtfwops2 sshd[11778]: debug1: kex: server->client
>> aes128-ctr hmac-md5 none
>> Aug  5 11:48:10 rtfwops2 sshd[11778]: debug1: SSH2_MSG_KEX_DH_GEX_REQUEST
>> received
>> Aug  5 11:48:10 rtfwops2 sshd[11778]: debug1: SSH2_MSG_KEX_DH_GEX_GROUP
>> sent
>> Aug  5 11:48:10 rtfwops2 sshd[11778]: debug1: expecting
>> SSH2_MSG_KEX_DH_GEX_INIT
>> Aug  5 11:48:10 rtfwops2 sshd[11778]: debug1: SSH2_MSG_KEX_DH_GEX_REPLY
>> sent
>> Aug  5 11:48:10 rtfwops2 sshd[11778]: debug1: SSH2_MSG_NEWKEYS sent
>> Aug  5 11:48:10 rtfwops2 sshd[11778]: debug1: expecting SSH2_MSG_NEWKEYS
>> Aug  5 11:48:10 rtfwops2 sshd[11778]: Connection closed by xxx.xxx.xxx.137
>> Aug  5 11:48:10 rtfwops2 sshd[11778]: debug1: do_cleanup
>> Aug  5 11:48:10 rtfwops2 sshd[11777]: debug1: do_cleanup
>>
>>   When I run a script from onadmin's cron on the server it can also ssh
>> successfully without password - I don't think this is a key issue.
>>
>>  Any suggestions?
>>
>>  Thanks
>>
>> Pierre
>>
>>  --
>> Pierre Naude
>> Rorotika Technologies
>>
>> e-mail: pierre.naude at rorotika.com
>> Tel.: +27-11-568-0805
>> Cell.:  +27-82-901-9609
>> Skype: pierre_naude
>> Google Hangouts: pierre.naude at rorotika.com
>>
>>
>>  _______________________________________________
>> Users mailing listUsers at lists.opennebula.orghttp://lists.opennebula.org/listinfo.cgi/users-opennebula.org
>>
>>
>> --
>> Olivier Sallou
>> IRISA / University of Rennes 1
>> Campus de Beaulieu, 35000 RENNES - FRANCE
>> Tel: 02.99.84.71.95
>>
>> gpg key id: 4096R/326D8438  (keyring.debian.org)
>> Key fingerprint = 5FB4 6F83 D3B9 5204 6335  D26D 78DC 68DB 326D 8438
>>
>>
>>
>
>
> --
> Pierre Naude
> Rorotika Technologies
>
> e-mail: pierre.naude at rorotika.com
> Tel.: +27-11-568-0805
> Cell.:  +27-82-901-9609
> Skype: pierre_naude
> Google Hangouts: pierre.naude at rorotika.com
>
>
> --
> Olivier Sallou
> IRISA / University of Rennes 1
> Campus de Beaulieu, 35000 RENNES - FRANCE
> Tel: 02.99.84.71.95
>
> gpg key id: 4096R/326D8438  (keyring.debian.org)
> Key fingerprint = 5FB4 6F83 D3B9 5204 6335  D26D 78DC 68DB 326D 8438
>
>
>
> _______________________________________________
> Users mailing list
> Users at lists.opennebula.org
> http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
>
>


-- 
Join us at OpenNebulaConf2013 <http://opennebulaconf.com/> in Berlin, 24-26
September, 2013
--
Jaime Melis
Project Engineer
OpenNebula - The Open Source Toolkit for Cloud Computing
www.OpenNebula.org | jmelis at opennebula.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opennebula.org/pipermail/users-opennebula.org/attachments/20130805/e0ee5762/attachment-0002.htm>

More information about the Users mailing list