[one-users] Problem monitoring ESX host with the CentOS 6.2 appliance listed in marketplace

Thu Aug 23 08:07:59 PDT 2012

Hi Tino,

Here is the full command, hostnames and passwords changed.  The command executes just fine when I copy it out of the log into the terminal. I am very confused. Both the daemon and the shell session appear to run as oneadmin.

Wed Aug 22 18:15:49 2012 [InM][I]: Monitoring host labhost (12)
Wed Aug 22 18:15:51 2012 [InM][I]: Command execution fail: /var/lib/one/remotes/im/run_probes vmware 12 labhost
Wed Aug 22 18:15:51 2012 [InM][E]: vmware.rb: Error executing: unset http_proxy && /usr/bin/tty_expect -u oneadmin -p MYPASSWORD virsh -c 'esx://labhost/?no_verify=1&auto_answer=1' --readonly nodeinfo err: ExitCode: 1
Wed Aug 22 18:15:51 2012 [InM][I]: out:
Wed Aug 22 18:15:51 2012 [InM][I]: error: internal error curl_easy_perform() returned an error: Couldn't connect to server (7) : Failed to connect to 1.2.3.4: Permission denied
Wed Aug 22 18:15:51 2012 [InM][I]: error: failed to connect to the hypervisor
Wed Aug 22 18:15:51 2012 [InM][E]: Error executing vmware.rb
Wed Aug 22 18:15:51 2012 [InM][I]: ExitCode: 1
Wed Aug 22 18:15:51 2012 [InM][E]: Error monitoring host 12 : MONITOR FAILURE 12 Error executing vmware.rb

Kind Regards
Stephen

From: Tino Vazquez [mailto:cvazquez at opennebula.org]
Sent: 23 August 2012 15:17
To: Howell, Stephen R
Cc: users at lists.opennebula.org
Subject: Re: [one-users] Problem monitoring ESX host with the CentOS 6.2 appliance listed in marketplace

Dear Stephan,

Thanks a lot for the email, this way we can benefit from other people's insights. I couldn't see anything wrong in the env variables, so let's try the http_proxy env var possibility.

Please replace the vmware.rb script with the one attached, and retry the monitorization. It basically does an unset of the http_proxy variable and then logs the complete command used. With the logged output we may be able to figure out what is going on.

Best regards,

-Tino

--
Constantino Vázquez Blanco, MSc
Project Engineer
OpenNebula - The Open-Source Solution for Data Center Virtualization
www.OpenNebula.org<http://www.OpenNebula.org> | @tinova79 | @OpenNebula

On Thu, Aug 23, 2012 at 3:42 PM, Howell, Stephen R <stephen.r.howell at jpmchase.com<mailto:stephen.r.howell at jpmchase.com>> wrote:
I have opened a bug but it was closed.... I have no idea why this is not working.  Somehow the host info collection works when the script is run by oneadmin but not when invoked via the "onehost create" command.  Any help is welcome:

Cannot add ESX host vai "onehost create" but /var/lib/one/remotes/im/run_probes vmware runs correctly under oneadmin account

Added by stephen howell<http://dev.opennebula.org/users/1197> about 23 hours<http://dev.opennebula.org/projects/opennebula/activity?from=2012-08-22> ago. Updated about 2 hours<http://dev.opennebula.org/projects/opennebula/activity?from=2012-08-23> ago.
Status:

Closed

Start date:

08/22/2012

Priority:

Normal

Due date:

Assignee:

-

% Done:

0%

Category:

-

Target version:

-

Resolution:

Affected Version:

OpenNebula 3.4

Story points

-

Velocity based estimate

-

________________________________
Quote<http://dev.opennebula.org/issues/1425>

Description

When adding an ESX host to OpenNebula 3.4.1 using the CentOS appliance the host produces log similar to:

Monitoring host labhost (8)
Tue Aug 21 14:42:48 2012 [InM][I]: Command execution fail: /var/lib/one/remotes/im/run_probes vmware 8 labhost
Tue Aug 21 14:42:48 2012 [InM][E]: vmware.rb: Error executing: virsh -c 'esx://labhost/?no_verify=1&auto_answer=1' --readonly nodeinfo err: ExitCode: 1

Tue Aug 21 14:42:48 2012 [InM][I]: out:
Tue Aug 21 14:42:48 2012 [InM][I]: error: internal error curl_easy_perform() returned an error: Couldn't connect to server (7) : Failed to connect to X.X.X.X: Permission denied

Tue Aug 21 14:42:48 2012 [InM][I]: error: failed to connect to the hypervisor
Tue Aug 21 14:42:48 2012 [InM][E]: Error executing vmware.rb
Tue Aug 21 14:42:48 2012 [InM][I]: ExitCode: 1

Tue Aug 21 14:42:48 2012 [InM][E]: Error monitoring host 8 : MONITOR FAILURE 8 Error executing vmware.rb

HOWEVER, when running the command from the shell it completes without error:
[oneadmin at ONE341 ~]$ /var/lib/one/remotes/im/run_probes vmware 8 labhost
HYPERVISOR=vmware TOTALCPU=800 FREECPU=800 CPUSPEED=2612 TOTALMEMORY=33550332 FREEMEMORY=26840265

History
#1<http://dev.opennebula.org/issues/1425#note-1>
Updated by stephen howell<http://dev.opennebula.org/users/1197> about 23 hours<http://dev.opennebula.org/projects/opennebula/activity?from=2012-08-22> ago
<http://dev.opennebula.org/issues/1425>

ESX host is ESX 4.1 update 1, SSH access for oneadmin is enabled, with passwords correctly added and SSH keys imported into the host.<http://dev.opennebula.org/issues/1425>

#2<http://dev.opennebula.org/issues/1425>
Updated by Tino Vázquez about 22 hours ago<http://dev.opennebula.org/issues/1425>
<http://dev.opennebula.org/issues/1425>

Since the command executes correctly under the shell, I can only think of environmental variables. Could you compare the variables accesible from the vmware.rb script when ran directly on the shell compared with the ones available when ONE runs the script?

Also, are you running any sort of HTTP proxy?

#3
Updated by stephen howell about 22 hours ago

Not familiar with ruby, could you provide a code snippet to show how I can show those environment variables?

No proxy here, direct connection although the system did have http_proxy set I have removed that and rebooted and symptoms remain unchanged.

Thanks

#4
Updated by Tino Vázquez about 22 hours ago

Add the following in line 46 of vmware.rb

`env > /tmp/vmware_env_vars`

Run it once from the shell, backup the file, make ONE monitor the host, and compare both files.

#5
Updated by stephen howell about 21 hours ago

Thanks for the assistance, this is the difference between the version called via "onehost create" and the one run from the shell:

[oneadmin at ONE341 ~]$ diff /tmp/vmware_env_vars /tmp/vmware_env_vars_shell
1a2,3

SELINUX_ROLE_REQUESTED=
TERM=xterm

3d4
< TERM=linux
5c6,7
< PRIORITY=19
---

SSH_CLIENT=169.66.210.159 58446 22
SELINUX_USE_CURRENT_RANGE=

7a10

SSH_TTY=/dev/pts/0

8a12

LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=01;05;37;41:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arj=01;31:*.taz=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lz=01;31:*.xz=01;31:*.bz2=01;31:*.tbz=01;31:*.tbz2=01;31:*.bz=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.rar=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=01;36:*.au=01;36:*.flac=01;36:*.mid=01;36:*.midi=01;36:*.mka=01;36:*.mp3=01;36:*.mpc=01;36:*.ogg=01;36:*.ra=01;36:*.wav=01;36:*.axa=01;36:*.oga=01;36:*.spx=01;36:*.xspf=01;36:

13a18

SELINUX_LEVEL_REQUESTED=

15c20
< SHLVL=5
---

SHLVL=3

19a25

SSH_CONNECTION=169.66.210.159 58446 169.82.112.252 22

22d27
< ONE_MAD_DEBUG=

#6
Updated by stephen howell about 4 hours ago

Please let me know if you need further information or if this bug is known to you and could be fixed by upgrading to a newer release. I have also noticed that I am unable to login to Sunstone, log error is " [E]: Error initializing authentication system" Perhaps the two are linked somehow.

#7
Updated by Tino Vázquez about 2 hours ago

  *   Status changed from New to Closed

I would suggest an email to the support list describing the monitorization problem, and a second one for the sunstone problem. This way other members of the community can help. I am closing this bug as it is not clear wether this is a bug or a problem with configuration. We will reopen it if we find out in the mailing list that there is indeed a bug.

This communication is for informational purposes only. It is not intended as an offer or solicitation for the purchase or sale of any financial instrument or as an official confirmation of any transaction. All market prices, data and other information are not warranted as to completeness or accuracy and are subject to change without notice. Any comments or statements made herein do not necessarily reflect those of JPMorgan Chase & Co., its subsidiaries and affiliates. This transmission may contain information that is privileged, confidential, legally privileged, and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is STRICTLY PROHIBITED. Although this transmission and any attachments are believed to be free of any virus or other defect that might affect any computer system into which it is received and opened, it is the responsibility of the recipient to ensure that it is virus free and no responsibility is accepted by JPMorgan Chase & Co., its subsidiaries and affiliates, as applicable, for any loss or damage arising in any way from its use. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format. Thank you. Please refer to http://www.jpmorgan.com/pages/disclosures for disclosures relating to European legal entities.

_______________________________________________
Users mailing list
Users at lists.opennebula.org
http://lists.opennebula.org/listinfo.cgi/users-opennebula.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opennebula.org/pipermail/users-opennebula.org/attachments/20120823/5fe46d0a/attachment-0003.htm>