Hi there,<div><br></div><div>Thank you for the feedback.</div><div><br></div><div>I have a comment though. Nokogiri should be installed by the install_gems script [1], so maybe you skipped that step.</div><div>Nokogiri drastically improves the CLI speed, but since you use python it should be unrelated to your response time... </div>
<div><br></div><div>Regards</div><div><br></div><div>[1] <a href="http://opennebula.org/documentation:rel3.6:ignc#ruby_libraries_requirements_front-end">http://opennebula.org/documentation:rel3.6:ignc#ruby_libraries_requirements_front-end</a><br clear="all">
--<br>Carlos Martín, MSc<br>Project Engineer<br>OpenNebula - The Open-source Solution for Data Center Virtualization<div><span style="border-collapse:collapse;color:rgb(136,136,136);font-family:arial,sans-serif;font-size:13px"><a href="http://www.OpenNebula.org" target="_blank">www.OpenNebula.org</a> | <a href="mailto:cmartin@opennebula.org" target="_blank">cmartin@opennebula.org</a> | <a href="http://twitter.com/opennebula" target="_blank">@OpenNebula</a></span><span style="border-collapse:collapse;color:rgb(136,136,136);font-family:arial,sans-serif;font-size:13px"><a href="mailto:cmartin@opennebula.org" style="color:rgb(42,93,176)" target="_blank"></a></span></div>
<br>
<br><br><div class="gmail_quote">On Tue, Sep 25, 2012 at 11:21 AM, Christoph Robbert <span dir="ltr"><<a href="mailto:chrobbert@googlemail.com" target="_blank">chrobbert@googlemail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<div>Hello Carlos, <br>
<br>
actually i'm running Ubuntu 11.04 on the x86_64 architecture.
Opennebula 3.6 was installed from the sourcefiles.<br>
<br>
The xmlrpc-c is installed in version 1.16.32-0ubuntu3.1 .<br>
Ruby is installed in Version 1.8.7 (2010-08-16 patchlevel 302).<br>
<br>
I didn't find any installation of the nokogiri ruby gem. So i
can't provide you a version.<br>
<br>
Regards, <br>
<br>
Christoph<br>
<br>
<br>
<br>
<br>
<br>
Am <a href="tel:21.09.2012%2012" value="+12109201212" target="_blank">21.09.2012 12</a>:28, schrieb Carlos Martín Sánchez:<br>
</div><div><div class="h5">
<blockquote type="cite">Hi Christoph,
<div><br>
</div>
<div>I'm sorry but I ran out of ideas. The only thing I can try is
to replicate your complete setup and give it a try with gdb when
I find the time.</div>
<div><br>
</div>
<div>Could you please share your distribution, opennebula
installation mode (from packages or source), and the versions of</div>
<div>- xmlrpc-c library</div>
<div>- ruby</div>
<div>- nokogiri ruby gem</div>
<div><br>
</div>
<div>Regards</div>
<div>--<br>
Carlos Martín, MSc<br>
Project Engineer<br>
OpenNebula - The Open-source Solution for Data Center
Virtualization
<div>
<span style="border-collapse:collapse;color:rgb(136,136,136);font-family:arial,sans-serif;font-size:13px"><a href="http://www.OpenNebula.org" target="_blank">www.OpenNebula.org</a> | <a href="mailto:cmartin@opennebula.org" target="_blank">cmartin@opennebula.org</a>
| <a href="http://twitter.com/opennebula" target="_blank">@OpenNebula</a></span><span style="border-collapse:collapse;color:rgb(136,136,136);font-family:arial,sans-serif;font-size:13px"></span></div>
<br>
<br>
<br>
<div class="gmail_quote">On Wed, Sep 5, 2012 at 2:53 PM,
Christoph Robbert <span dir="ltr"><<a href="mailto:chrobbert@googlemail.com" target="_blank">chrobbert@googlemail.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<div>Hey, <br>
<br>
my oned.conf is very close to the default oned.conf. I
only changed the DB Backend. So the values around
MANAGER_TIMER looks like this:<br>
<br>
#MANAGER_TIMER = 30<br>
<br>
HOST_MONITORING_INTERVAL = 600<br>
#HOST_PER_INTERVAL = 15<br>
#HOST_MONITORING_EXPIRATION_TIME = 86400<br>
<br>
VM_POLLING_INTERVAL = 600<br>
#VM_PER_INTERVAL = 5<br>
#VM_MONITORING_EXPIRATION_TIME = 86400<br>
<br>
I assume, that the default values are used because of
the comments. <br>
<br>
I didn't activate any external authentication or
authorization drivers by hand and even could see any but
the default in the oned.conf. <br>
My user is the oneadmin.<br>
<br>
Before using MySQL I used SQLite as Database. My first
step to tackle the response time increasing was
replacing SQLite with MySQL.<br>
<br>
I append the output of the MySQL command to this email.
I executed after during a 400 seconds stuck. <br>
<br>
Thanks for your help.<br>
<br>
Regards, <br>
<br>
Christoph Robbert<br>
<br>
<br>
Am 05.09.2012 14:17, schrieb Carlos Martín Sánchez:<br>
</div>
<div>
<div>
<blockquote type="cite">Hi,
<div><br>
</div>
<div>Let's try to rule out one thing at a time.</div>
<div><br>
</div>
<div>Did you set any timer values in oned.conf that
may overload opennebula? If the values
of MANAGER_TIMER, HOST and VM MONITORING_INTERVAL
are too low, opennebula could choke.</div>
<div><br>
</div>
<div>Do you have any external authentication or
authorization drivers enabled in oned.conf? Are
you using oneadmin to do the requests, or a
regular user? Doing a call to external drivers for
each request may be a possible reason...</div>
<div><br>
</div>
<div>Is the communication with MySQL the problem?
Next time you see OpenNebula slowing down, you
could try to execute, from the front-end machine,
the following:</div>
<div><br>
</div>
<div>$ mysql -u oneadmin -poneadmin -h localhost -P
0 opennebula -e "SELECT body FROM vm_pool WHERE
state<>6;"<br>
<br>
</div>
<div><br>
</div>
<div>Thanks for your feedback</div>
<div><br clear="all">
--<br>
Carlos Martín, MSc<br>
Project Engineer<br>
OpenNebula - The Open-source Solution for Data
Center Virtualization
<div><span style="border-collapse:collapse;color:rgb(136,136,136);font-family:arial,sans-serif;font-size:13px"><a href="http://www.OpenNebula.org" target="_blank">www.OpenNebula.org</a> | <a href="mailto:cmartin@opennebula.org" target="_blank">cmartin@opennebula.org</a> |
<a href="http://twitter.com/opennebula" target="_blank">@OpenNebula</a></span><span style="border-collapse:collapse;color:rgb(136,136,136);font-family:arial,sans-serif;font-size:13px"></span></div>
<br>
<br>
<br>
<div class="gmail_quote">On Tue, Sep 4, 2012 at
1:25 PM, Christoph Robbert <span dir="ltr"><<a href="mailto:chrobbert@googlemail.com" target="_blank">chrobbert@googlemail.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<div>Hello, <br>
<br>
i use MySQL as Database. I query
Opennebula from python via pyoca[1]. But i
registered the same effect using the
command "onevm list". <br>
<br>
The effect depend also on the number of
running VMs, but i run only at maximum 30
VMs. The effect starts at round about 6
VMs. Usually the time increases to round
about one or two seconds. But suddenly it
response very slow >>>60 seconds
or didn't answer. <br>
<br>
Also the creation of a new VM getting
stucked (response time also increases to
over 60 seconds). <br>
<br>
Sometimes the time increases to round
about 240 seconds for one call. Then the
next call takes about one or two seconds.
<br>
<br>
I couldn't see an xml-rpc request in the
oned.log because my gui wait until the
last xml-rpc request is finished.<br>
<br>
I profile every part of my code with time
measurements and traced it down to the
xml-rpc requests to opennebula.<br>
<br>
Hope this help.<br>
<br>
Regards, <br>
<br>
Christoph Robbert<br>
<br>
<br>
<br>
[1] <a href="https://github.com/lukaszo/python-oca" target="_blank">
https://github.com/lukaszo/python-oca</a><br>
<br>
<br>
<br>
Am 04.09.2012 12:59, schrieb Carlos Martín
Sánchez:<br>
</div>
<div>
<div>
<blockquote type="cite">Hi,
<div><br>
</div>
<div>Can you share some more
information about your scenario? Are
you using sqlite, or mysql? MySQL
can drastically improve the
performance over sqlite.</div>
<div><br>
</div>
<div>How are you querying OpenNebula,
are you using the CLI, our ruby/java
OCA? The response time can be
affected by the xml processing that
the OCA has to do. If you are using
Ruby, it is crucial that you have
the nokogiri gem installed</div>
<div><br>
</div>
<div>Does the response time increase
always over time, or is it related
to the number of existing VMs? If
so, how many VMs does it take to
make it irresponsive?</div>
<div><br>
</div>
<div>Can you still see the xml-rpc
requests in oned.log each second?</div>
<div><br>
</div>
<div>I'm trying to reproduce the
problem, having over a 1000 running
VMs. I'm doing a onevm create &
shutdown every 5 seconds while
checking the time it takes to do a
onevm list each second, but can't
see any response taking more than
one or two seconds.</div>
<div><br>
</div>
<div>Regards<br clear="all">
--<br>
Carlos Martín, MSc<br>
Project Engineer<br>
OpenNebula - The Open-source
Solution for Data Center
Virtualization
<div><span style="border-collapse:collapse;color:rgb(136,136,136);font-family:arial,sans-serif;font-size:13px"><a href="http://www.OpenNebula.org" target="_blank">www.OpenNebula.org</a>
| <a href="mailto:cmartin@opennebula.org" target="_blank">cmartin@opennebula.org</a>
| <a href="http://twitter.com/opennebula" target="_blank">@OpenNebula</a></span><span style="border-collapse:collapse;color:rgb(136,136,136);font-family:arial,sans-serif;font-size:13px"></span></div>
<br>
<br>
<br>
<div class="gmail_quote">On Tue, Aug
28, 2012 at 1:40 PM, Christoph
Robbert <span dir="ltr"><<a href="mailto:chrobbert@googlemail.com" target="_blank">chrobbert@googlemail.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> Hello,<br>
<br>
I'm working on project with
Opennebula 3.6 as
cloudcontroller. We start and
stop VMs via xml-rpc nearly
every 15 seconds. To monitor the
actions in realtime, i
implemented a gui, which calls
Opennebula every second via
xml-rpc. Now i notice a real big
increase of the response time
after 10 minutes. The response
time increases from nearly 1
second to 5 Minutes. Some time i
have to restart Opennebula
because the response time
increase to infinity.<br>
Could you give me a hind where i
should start to trace the
bottleneck in Opennebula?<br>
<br>
<br>
Best Regards,<br>
<br>
Christoph Robbert<br>
_______________________________________________<br>
Users mailing list<br>
<a href="mailto:Users@lists.opennebula.org" target="_blank">Users@lists.opennebula.org</a><br>
<a href="http://lists.opennebula.org/listinfo.cgi/users-opennebula.org" target="_blank">http://lists.opennebula.org/listinfo.cgi/users-opennebula.org</a><br>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
</div></div></div>
<br>_______________________________________________<br>
Users mailing list<br>
<a href="mailto:Users@lists.opennebula.org">Users@lists.opennebula.org</a><br>
<a href="http://lists.opennebula.org/listinfo.cgi/users-opennebula.org" target="_blank">http://lists.opennebula.org/listinfo.cgi/users-opennebula.org</a><br>
<br></blockquote></div><br></div>