Hi<div><br></div><div>It seems that you have already created the VLANs, so you do not need OpenNebula to do this for you. In your case you can just add the host and use --net dummy. The 802.1Q drivers would create eth0.160 and a bridge for it.</div>
<div><br></div><div>Now your problem, I think that OpenNebula is not able to detect your hypervisor, probably because in yout system lsmod is not in "/sbin/lsmod". Could you check that, and set it to the proper path in OpenNebulaNetwork.rb that should be in /var/lib/one/remotes/vnm. (In multi-node installations you would need onehost sync to propagate the changes). But as I said if you want OpenNebula to do the bridge work for you you need to:</div>
<div><br></div><div>1.- Add to the Network templates</div><div><br></div><div>PHYDEV = eth0</div><div>VLAN = YES</div><div>VLAN_ID = 160 # This is optional OPenNebula would pick one for you</div><div>BRIDGE = server160 #This is optional OpenNebula would pick one for you</div>
<div><br></div><div><br></div><div>2.- Add a host with --net 802.1Q</div><div><br></div><div><br></div><div>I'd suggest to first keep trying with your current setup (manual VLAN), remove the host and readd it with --net dummy; and then try the automatic VLAN mode (as described above). For the automatic VLAN we need to solve the issue you've reported that I believe could be the lsmod issue....</div>
<div><br></div><div>Cheers</div><div><br></div><div>Ruben</div><div><br></div><div><br></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Tue, Nov 27, 2012 at 8:42 AM, Stuart Longland <span dir="ltr"><<a href="mailto:stuartl@vrt.com.au" target="_blank">stuartl@vrt.com.au</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi all,<br>
<br>
Firstly I'll start by pointing out that I'm very new to OpenNebula,<br>
having grown used to just using plain shell scripts to fire up QEMU<br>
instances.<br>
<br>
My workplace is looking to use OpenNebula on a multi-node cluster for<br>
their server infrastructure. The machines are Intel Core i3 systems<br>
with Intel DQ77KB mainboards. The systems run Ubuntu 12.04 AMD64.<br>
<br>
The plan is to use the mainboard's two onboard network cards in LACP to<br>
facilitate high-speed data transfer between VMs for live migration and<br>
for distributed storage using Ceph, and a third PCI express card on a<br>
VLAN trunk port facing the network.<br>
<br>
Right now I'm testing on a single node, not one of the ones we'll<br>
eventually use but rather an older, former VMWare ESX box.<br>
<br>
The status:<br>
<br>
- Ubuntu 12.04 is installed. OpenNebula 3.8.1 packages installed from<br>
the OpenNebula website.<br>
- I have the switch port connected to eth0 set up as a VLAN trunk<br>
- I have /etc/network/interfaces set up to create the eth0.XXX devices<br>
for each VLAN, and sensibly named bridge devices, `brctl show` looks<br>
like this:<br>
> root@bneprdvm0:~# brctl show<br>
> bridge name bridge id STP enabled interfaces<br>
> client130 8000.00101814cb1b no eth0.130<br>
> project144 8000.00101814cb1b no eth0.144<br>
> project159 8000.00101814cb1b no eth0.159<br>
> server160 8000.00101814cb1b no eth0.160<br>
<br>
This works fine with plain QEMU.<br>
<br>
Now, I've tried to create a virtual network to place a VM on the<br>
'server160' bridge (i.e. I'd expect kvm to start; then it do a `brctl<br>
addif server160 tapX`). The following is my configuration for that<br>
network interface:<br>
<br>
> oneadmin@bneprdvm0:~$ onevnet show 4<br>
> VIRTUAL NETWORK 4 INFORMATION<br>
> ID : 4<br>
> NAME : Server 160 LAN<br>
> USER : oneadmin<br>
> GROUP : oneadmin<br>
> CLUSTER : -<br>
> TYPE : FIXED<br>
> BRIDGE : server160<br>
> VLAN : No<br>
> PHYSICAL DEVICE:<br>
> VLAN ID :<br>
> USED LEASES : 0<br>
><br>
> PERMISSIONS<br>
> OWNER : um-<br>
> GROUP : ---<br>
> OTHER : ---<br>
><br>
> VIRTUAL NETWORK TEMPLATE<br>
><br>
><br>
> FREE LEASES<br>
> LEASE=[ IP="10.87.160.13", MAC="52:54:00:12:34:58", USED="0", VID="-1" ]<br>
<br>
The VM template looks like this:<br>
> CPU="1.0"<br>
> DISK=[<br>
> CLONE="NO",<br>
> CLUSTER_ID="100",<br>
> DATASTORE="default",<br>
> DATASTORE_ID="1",<br>
> DEV_PREFIX="hd",<br>
> DISK_ID="0",<br>
> IMAGE="bneprdsrv5-hda",<br>
> IMAGE_ID="3",<br>
> IMAGE_UNAME="vrtadmin",<br>
> PERSISTENT="YES",<br>
> READONLY="NO",<br>
> SAVE="YES",<br>
> SOURCE="/var/lib/one/datastores/1/301aac89561c0fbeaf140cb09124f941",<br>
> TARGET="hda",<br>
> TM_MAD="shared",<br>
> TYPE="FILE" ]<br>
> MEMORY="256"<br>
> NAME="bneprdsrv5"<br>
> NIC=[<br>
> BRIDGE="project160",<br>
> IP="10.87.160.13",<br>
> MAC="52:54:00:12:34:58",<br>
> NETWORK="server160",<br>
> NETWORK_ID="2",<br>
> NETWORK_UNAME="vrtadmin",<br>
> PHYDEV="eth0.160",<br>
> VLAN="YES" ]<br>
> REQUIREMENTS="CLUSTER_ID = 100"<br>
> VMID="3"<br>
(Anyone know how to edit that by the way? So far all I seem to be able<br>
to do is blow away the VM configuration and rebuild from scratch!)<br>
<br>
I get the following error (apologies for the wall of text in advance):<br>
> Tue Nov 27 17:09:35 2012 [LCM][I]: New VM state is BOOT<br>
> Tue Nov 27 17:09:35 2012 [VMM][I]: Generating deployment file: /var/lib/one/3/deployment.0<br>
> Tue Nov 27 17:09:36 2012 [VMM][I]: Command execution fail: /var/tmp/one/vnm/802.1Q/pre PFZNPjxJRD4zPC9JRD48VUlEPj<br>
> A8L1VJRD48R0lEPjA8L0dJRD48VU5BTUU+b25lYWRtaW48L1VOQU1FPjxHTkFNRT5vbmVhZG1pbjwvR05BTUU+PE5BTUU+Ym5lcHJkc3J2NTwvTkF<br>
> NRT48UEVSTUlTU0lPTlM+PE9XTkVSX1U+MTwvT1dORVJfVT48T1dORVJfTT4xPC9PV05FUl9NPjxPV05FUl9BPjA8L09XTkVSX0E+PEdST1VQX1U+<br>
> MDwvR1JPVVBfVT48R1JPVVBfTT4wPC9HUk9VUF9NPjxHUk9VUF9BPjA8L0dST1VQX0E+PE9USEVSX1U+MDwvT1RIRVJfVT48T1RIRVJfTT4wPC9PV<br>
> EhFUl9NPjxPVEhFUl9BPjA8L09USEVSX0E+PC9QRVJNSVNTSU9OUz48TEFTVF9QT0xMPjA8L0xBU1RfUE9MTD48U1RBVEU+MzwvU1RBVEU+PExDTV<br>
> 9TVEFURT4yPC9MQ01fU1RBVEU+PFJFU0NIRUQ+MDwvUkVTQ0hFRD48U1RJTUU+MTM1NDAwMDE1NjwvU1RJTUU+PEVUSU1FPjA8L0VUSU1FPjxERVB<br>
> MT1lfSUQvPjxNRU1PUlk+MDwvTUVNT1JZPjxDUFU+MDwvQ1BVPjxORVRfVFg+MDwvTkVUX1RYPjxORVRfUlg+MDwvTkVUX1JYPjxURU1QTEFURT48<br>
> Q1BVPjwhW0NEQVRBWzEuMF1dPjwvQ1BVPjxESVNLPjxDTE9ORT48IVtDREFUQVtOT11dPjwvQ0xPTkU+PENMVVNURVJfSUQ+PCFbQ0RBVEFbMTAwX<br>
> V0+PC9DTFVTVEVSX0lEPjxEQVRBU1RPUkU+PCFbQ0RBVEFbZGVmYXVsdF1dPjwvREFUQVNUT1JFPjxEQVRBU1RPUkVfSUQ+PCFbQ0RBVEFbMV1dPj<br>
> wvREFUQVNUT1JFX0lEPjxERVZfUFJFRklYPjwhW0NEQVRBW2hkXV0+PC9ERVZfUFJFRklYPjxESVNLX0lEPjwhW0NEQVRBWzBdXT48L0RJU0tfSUQ<br>
> +PElNQUdFPjwhW0NEQVRBW2JuZXByZHNydjUtaGRhXV0+PC9JTUFHRT48SU1BR0VfSUQ+PCFbQ0RBVEFbM11dPjwvSU1BR0VfSUQ+PElNQUdFX1VO<br>
> QU1FPjwhW0NEQVRBW3ZydGFkbWluXV0+PC9JTUFHRV9VTkFNRT48UEVSU0lTVEVOVD48IVtDREFUQVtZRVNdXT48L1BFUlNJU1RFTlQ+PFJFQURPT<br>
> kxZPjwhW0NEQVRBW05PXV0+PC9SRUFET05MWT48U0FWRT48IVtDREFUQVtZRVNdXT48L1NBVkU+PFNPVVJDRT48IVtDREFUQVsvdmFyL2xpYi9vbm<br>
> UvZGF0YXN0b3Jlcy8xLzMwMWFhYzg5NTYxYzBmYmVhZjE0MGNiMDkxMjRmOTQxXV0+PC9TT1VSQ0U+PFRBUkdFVD48IVtDREFUQVtoZGFdXT48L1R<br>
> BUkdFVD48VE1fTUFEPjwhW0NEQVRBW3NoYXJlZF1dPjwvVE1fTUFEPjxUWVBFPjwhW0NEQVRBW0ZJTEVdXT48L1RZUEU+PC9ESVNLPjxNRU1PUlk+<br>
> PCFbQ0RBVEFbMjU2XV0+PC9NRU1PUlk+PE5BTUU+PCFbQ0RBVEFbYm5lcHJkc3J2NV1dPjwvTkFNRT48TklDPjxCUklER0U+PCFbQ0RBVEFbcHJva<br>
> mVjdDE2MF1dPjwvQlJJREdFPjxJUD48IVtDREFUQVsxMC44Ny4xNjAuMTNdXT48L0lQPjxNQUM+PCFbQ0RBVEFbNTI6NTQ6MDA6MTI6MzQ6NThdXT<br>
> 48L01BQz48TkVUV09SSz48IVtDREFUQVtzZXJ2ZXIxNjBdXT48L05FVFdPUks+PE5FVFdPUktfSUQ+PCFbQ0RBVEFbMl1dPjwvTkVUV09SS19JRD4<br>
> 8TkVUV09SS19VTkFNRT48IVtDREFUQVt2cnRhZG1pbl1dPjwvTkVUV09SS19VTkFNRT48UEhZREVWPjwhW0NEQVRBW2V0aDAuMTYwXV0+PC9QSFlE<br>
> RVY+PFZMQU4+PCFbQ0RBVEFbWUVTXV0+PC9WTEFOPjwvTklDPjxSRVFVSVJFTUVOVFM+PCFbQ0RBVEFbQ0xVU1RFUl9JRCA9IDEwMF1dPjwvUkVRV<br>
> UlSRU1FTlRTPjxWTUlEPjwhW0NEQVRBWzNdXT48L1ZNSUQ+PC9URU1QTEFURT48SElTVE9SWV9SRUNPUkRTPjxISVNUT1JZPjxPSUQ+MzwvT0lEPj<br>
> xTRVE+MDwvU0VRPjxIT1NUTkFNRT5ibmVwcmR2bTAudnJ0LmludDwvSE9TVE5BTUU+PEhJRD4yPC9ISUQ+PFNUSU1FPjEzNTQwMDAxNzU8L1NUSU1<br>
> FPjxFVElNRT4wPC9FVElNRT48Vk1NTUFEPnZtbV9rdm08L1ZNTU1BRD48Vk5NTUFEPjgwMi4xUTwvVk5NTUFEPjxUTU1BRD5zaGFyZWQ8L1RNTUFE<br>
> PjxEU19MT0NBVElPTj4vdmFyL2xpYi9vbmUvL2RhdGFzdG9yZXM8L0RTX0xPQ0FUSU9OPjxEU19JRD4wPC9EU19JRD48UFNUSU1FPjEzNTQwMDAxNzU8L1BTVElNRT48UEVUSU1FPjEzNTQwMDAxNzU8L1BFVElNRT48UlNUSU1FPjEzNTQwMDAxNzU8L1JTVElNRT48UkVUSU1FPjA8L1JFVElNRT48RVNUSU1FPjA8L0VTVElNRT48RUVUSU1FPjA8L0VFVElNRT48UkVBU09OPjA8L1JFQVNPTj48L0hJU1RPUlk+PC9ISVNUT1JZX1JFQ09SRFM+PC9WTT4=<br>
> Tue Nov 27 17:09:36 2012 [VMM][I]: /var/tmp/one/vnm/802.1Q/../OpenNebulaNic.rb:31:in `new_nic': undefined method `new' for nil:NilClass (NoMethodError)<br>
> Tue Nov 27 17:09:36 2012 [VMM][I]: from /var/tmp/one/vnm/802.1Q/../OpenNebulaNetwork.rb:60:in `initialize'<br>
> Tue Nov 27 17:09:36 2012 [VMM][I]: from /usr/lib/ruby/1.8/rexml/element.rb:892:in `each'<br>
> Tue Nov 27 17:09:36 2012 [VMM][I]: from /usr/lib/ruby/1.8/rexml/xpath.rb:53:in `each'<br>
> Tue Nov 27 17:09:36 2012 [VMM][I]: from /usr/lib/ruby/1.8/rexml/element.rb:892:in `each'<br>
> Tue Nov 27 17:09:36 2012 [VMM][I]: from /var/tmp/one/vnm/802.1Q/../OpenNebulaNetwork.rb:59:in `initialize'<br>
> Tue Nov 27 17:09:36 2012 [VMM][I]: from /var/tmp/one/vnm/802.1Q/../OpenNebulaNetwork.rb:110:in `new'<br>
> Tue Nov 27 17:09:36 2012 [VMM][I]: from /var/tmp/one/vnm/802.1Q/../OpenNebulaNetwork.rb:110:in `initialize'<br>
> Tue Nov 27 17:09:36 2012 [VMM][I]: from /var/tmp/one/vnm/802.1Q/HostManaged.rb:23:in `initialize'<br>
> Tue Nov 27 17:09:36 2012 [VMM][I]: from /var/tmp/one/vnm/802.1Q/../OpenNebulaNetwork.rb:100:in `new'<br>
> Tue Nov 27 17:09:36 2012 [VMM][I]: from /var/tmp/one/vnm/802.1Q/../OpenNebulaNetwork.rb:100:in `from_base64'<br>
> Tue Nov 27 17:09:36 2012 [VMM][I]: from /var/tmp/one/vnm/802.1Q/pre:24<br>
> Tue Nov 27 17:09:36 2012 [VMM][I]: ExitCode: 1<br>
> Tue Nov 27 17:09:36 2012 [VMM][I]: Failed to execute network driver operation: pre.<br>
> Tue Nov 27 17:09:36 2012 [VMM][E]: Error deploying virtual machine<br>
> Tue Nov 27 17:09:36 2012 [DiM][I]: New VM state is FAILED<br>
<br>
The error message was suspiciously like the following report…<br>
<a href="http://comments.gmane.org/gmane.comp.distributed.opennebula.user/8527" target="_blank">http://comments.gmane.org/gmane.comp.distributed.opennebula.user/8527</a><br>
… where the user miraculously fixed it by installing a missing package.<br>
Lord knows what that package was, as I seem to be missing it too and am<br>
having a hard time figuring out which one.<br>
<br>
Could someone on this list enlighten me as to what I might be doing<br>
wrong? Apologies if this has been asked before, but I can only find the<br>
above thread which gives me no further information.<br>
<br>
Regards,<br>
<span><font color="#888888">--<br>
## -,-''''-. ###### Stuart Longland, Software Engineer<br>
##. : ## : ## 38b Douglas Street -+*WE HAVE*+-<br>
## # ## -'` .#' Milton, QLD, 4034 --+*MOVED*+--<br>
'#' *' '-. *' <a href="http://www.vrt.com.au" target="_blank">http://www.vrt.com.au</a><br>
S Y S T E M S T: 07 3535 9619 F: 07 3535 9699<br>
_______________________________________________<br>
Users mailing list<br>
<a href="mailto:Users@lists.opennebula.org" target="_blank">Users@lists.opennebula.org</a><br>
<a href="http://lists.opennebula.org/listinfo.cgi/users-opennebula.org" target="_blank">http://lists.opennebula.org/listinfo.cgi/users-opennebula.org</a><br>
</font></span></blockquote></div><br><br clear="all"><div><br></div>-- <br>Ruben S. Montero, PhD<br>Project co-Lead and Chief Architect<br>OpenNebula - The Open Source Solution for Data Center Virtualization<br><a href="http://www.OpenNebula.org" target="_blank">www.OpenNebula.org</a> | <a href="mailto:rsmontero@opennebula.org" target="_blank">rsmontero@opennebula.org</a> | @OpenNebula<br>
</div>