[one-users] VM monitoring information do not get deleted

[Stefan Kooman]

Quoting Wilma Hermann (wilma.hermann at gmail.com):
> Hi,
> 
> I observed a problem with two OpenNebula setups, that I set up with version
> 4.4 and which I upgraded to 4.6 some weeks ago: The VM monitoring
> information does not seem to be deleted from the database (MySQL) after
> VM_MONITORING_EXPIRATION_TIME has expired.
> 
> I have a sandbox for testing issues: A single machine (both frontend and
> host) with a single virtual machine, that runs 24/7. When I upgraded
> OpenNebula 4.4 to 4.6, the SQL-Dump created by "onedb upgrade" was 3.6 MB
> big (perfectly okay for such a small setup). Today, when I dumped the DB,
> the backup file is 176 MB in size. Wondering about the size, I inspected
> the database and found ~77k rows in the "vm_monitoring" table. Obviously,
> OpenNebula writes rows into this table every few seconds without ever
> deleting anything.
> 
> I didn't change VM_MONITORING_EXPIRATION_TIME in oned.conf (it was
> commented out), so it should delete old values after 4h. I manually set
> VM_MONITORING_EXPIRATION_TIME to 14400 as well as other values: No effect,
> the DB continues to inflate.
> 
> Meanwhile, Sunstone begins to become unresponsible when I open the details
> of a VM. I believe this is due to generating the CPU and memory graphs
> which has to process several ten thousands of rows.
> 
> Did I miss some setting or is this a bug?

After reading the above things start to make sense. We're using a
MySQL master-master replication setup, with one oned server as primary
master. The amount of network traffic, InnoDB activity, disk throughput,
etc have gone up tremendously. See attached images to get an impression.
For newly created vm's opening "capacity" or "network" tab this isn't a problem, yet.
But for vm's that are already running for month's this is a problem. I
see the sunstone instance that is serving me dropping out of the
load-balancer for not replying to health-checks in time. Just by
clicking the "network" tab of a long running vm.

If this is a bug I need a workaround soon before running out of disk
space ;).

Gr. Stefan

P.s Thanks for Wilma for spotting this, haven't had time to look into
this issue: too busy with reverting back from trusty -> saucy on
hypervisors, more on that later.


-- 
| BIT BV  http://www.bit.nl/        Kamer van Koophandel 09090351
| GPG: 0xD14839C6                   +31 318 648 688 / info at bit.nl