[one-users] Xen problem - Dom0 using up all memory on Scientific Linux

[Tres Wong-Godfrey]

Hi Maciej,

I believe the issue is Xen's default configuration. By default it will reserve all memory to dom0 & dynamically allocate it to domU as they get created. To fix, you just need to statically define the maximum memory available to dom0.

To fix this, just pass a kernel parameter on boot. Edit the Xen kernel definition in /boot/grub/grub.conf and add dom0_mem=XXXMB. So for example on my CentOS hardware nodes I have:

       kernel /xen.gz-2.6.18-128.4.1.el5 dom0_mem=256MB

Regards,
Tres

Tres Wong-Godfrey

On Jan 21, 2010, at 12:11 PM, makhanmail wrote:

> Hi again,
> 
> I got another problem with configuration of my new cloud. Although there is no big process running in the system, xentop states the Domain-0 is using up 95% of the RAM. The same thing is happening on two completely different servers, so I think it may have something to do with OS, which is Scientific Linux 5.4.
> 
> Do you know what may be the cause and how to fix this?
> 
> Regards,
> Maciej
> 
> 
> 
> 
> [root at phrixus ~]# xentop -bi1
> xentop - 20:53:57   Xen 3.1.2-164.2.1.el5
> 1 domains: 1 running, 0 blocked, 0 paused, 0 crashed, 0 dying, 0 shutdown
> Mem: 16776548k total, 16383780k used, 392768k free    CPUs: 8 @ 2666MHz
>      NAME  STATE   CPU(sec) CPU(%)     MEM(k) MEM(%)  MAXMEM(k) MAXMEM(%) VCPUS NETS NETTX(k) NETRX(k) VBDS   VBD_OO   VBD_RD   VBD_WR SSID
>  Domain-0 -----r        650    0.0   16038912   95.6   no limit       n/a     8    5    16244    94733    0        0        0        0    0
> 
> [root at phrixus ~]# free
>             total       used       free     shared    buffers     cached
> Mem:      16038912    1451612   14587300          0     196588     669004
> -/+ buffers/cache:     586020   15452892
> Swap:     18481144          0   18481144
> 
> [root at phrixus ~]# xm info
> host                   : phrixus
> release                : 2.6.18-164.2.1.el5xen
> version                : #1 SMP Tue Sep 29 19:06:31 EDT 2009
> machine                : x86_64
> nr_cpus                : 8
> nr_nodes               : 1
> sockets_per_node       : 2
> cores_per_socket       : 4
> threads_per_core       : 1
> cpu_mhz                : 2666
> hw_caps                : bfebfbff:20000800:00000000:00000140:000ce3bd:00000000:00000001
> total_memory           : 16383
> free_memory            : 383
> node_to_cpu            : node0:0-7
> xen_major              : 3
> xen_minor              : 1
> xen_extra              : .2-164.2.1.el5
> xen_caps               : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64
> xen_pagesize           : 4096
> platform_params        : virt_start=0xffff800000000000
> xen_changeset          : unavailable
> cc_compiler            : gcc version 4.1.2 20080704 (Red Hat 4.1.2-46)
> cc_compile_by          : brewbuilder
> cc_compile_domain      : fnal.gov
> cc_compile_date        : Tue Sep 29 18:42:12 EDT 2009
> xend_config_format     : 2
> 
> 
> [root at phrixus ~]# lsb_release -a
> LSB Version:    :core-3.1-amd64:core-3.1-ia32:core-3.1-noarch:graphics-3.1-amd64:graphics-3.1-ia32:graphics-3.1-noarch
> Distributor ID: ScientificSL
> Description:    Scientific Linux SL release 5.4 (Boron)
> Release:        5.4
> Codename:       Boron
> 
> [root at phrixus ~]# uname -a
> Linux phrixus 2.6.18-164.2.1.el5xen #1 SMP Tue Sep 29 19:06:31 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux
> 
> 
> [root at nephele ~]# xentop -bi1
> xentop - 21:01:49   Xen 3.1.2-164.2.1.el5
> 1 domains: 1 running, 0 blocked, 0 paused, 0 crashed, 0 dying, 0 shutdown
> Mem: 4192600k total, 3799740k used, 392860k free    CPUs: 4 @ 2394MHz
>      NAME  STATE   CPU(sec) CPU(%)     MEM(k) MEM(%)  MAXMEM(k) MAXMEM(%) VCPUS NETS NETTX(k) NETRX(k) VBDS   VBD_OO   VBD_RD   VBD_WR SSID
>  Domain-0 -----r       2297    0.0    3665920   87.4   no limit       n/a     4    6     8843   434339    0        0        0        0    0
> 
> [root at nephele ~]# free
>             total       used       free     shared    buffers     cached
> Mem:       3665920    1336244    2329676          0     243360     706732
> -/+ buffers/cache:     386152    3279768
> Swap:      6062072          0    6062072
> 
> [root at nephele ~]# xm info
> host                   : nephele.ifj.edu.pl
> release                : 2.6.18-164.2.1.el5xen
> version                : #1 SMP Tue Sep 29 19:06:31 EDT 2009
> machine                : x86_64
> nr_cpus                : 4
> nr_nodes               : 1
> sockets_per_node       : 1
> cores_per_socket       : 4
> threads_per_core       : 1
> cpu_mhz                : 2394
> hw_caps                : bfebfbff:20100800:00000000:00000140:0000e3bd:00000000:00000001
> total_memory           : 4094
> free_memory            : 383
> node_to_cpu            : node0:0-3
> xen_major              : 3
> xen_minor              : 1
> xen_extra              : .2-164.2.1.el5
> xen_caps               : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64
> xen_pagesize           : 4096
> platform_params        : virt_start=0xffff800000000000
> xen_changeset          : unavailable
> cc_compiler            : gcc version 4.1.2 20080704 (Red Hat 4.1.2-46)
> cc_compile_by          : brewbuilder
> cc_compile_domain      : fnal.gov
> cc_compile_date        : Tue Sep 29 18:42:12 EDT 2009
> xend_config_format     : 2
> 
> 
> 
> _______________________________________________
> Users mailing list
> Users at lists.opennebula.org
> http://lists.opennebula.org/listinfo.cgi/users-opennebula.org