[one-dev] New repository for context packages

[Javier Fontan]

Hi Boris,

It makes total sense and I would love to just rely in cloud-init. The
problem is that I had no luck making it work in all distributions, for
example Ubuntu 12.04.

cloud-init was thought to be used alongside dhcp configuration for
network and cloud-init for all the other things. Network configuration
was added to it but it still has some troubles in some distros.
Sometimes requiring a restart after configuration or unconfiguring the
network (so it does not try dhcp) in the cloud config commands.

Another problem is adding new features to it as it may take time to
put them in the upstream repo. This may be fixed having a special
build or maybe moving all the non basic configuration to cloud config
data and some scripts pushed to the context.

If there's a way to make it work reliabily without dhcp and there's a
nice way to add new features without changing cloud-init I will push
for it and deprecate the context packages.

Cheers

On Thu, Jul 10, 2014 at 10:21 AM, Boris Parak <256254 at mail.muni.cz> wrote:
> Hi Javier,
>
> this might be unacceptable for you ... but ...
>
> What about dropping custom contextualization packages and switching
> completely to cloud-init? Since v0.7.5 it includes basic support for
> OpenNebula's contextualization disks, it supports full network
> configuration (ETH* variables), ssh key injection (SSH* variables),
> etc.
>
> It already has a mechanism for supporting different distros and
> platforms, it makes appliances compatible with multiple cloud
> management frameworks (and users like that very much), it has a
> community and won't be discontinued anytime soon.
>
> It might also help with future integration of advanced features in
> OpenNebula, it supports stuff like orchestration with chef/puppet and
> call-home hooks etc.
>
> Just an idea :-) We have been using it in production for quite some
> time and it works. What we would definitely appreciate (and are
> willing to help with) is deeper integration with OpenNebula.
>
> Cheers, Boris
>
> On Tue, Jul 8, 2014 at 6:19 PM, Javier Fontan <jfontan at opennebula.org> wrote:
>> Hello,
>>
>> TL;DR: Some ideas to improve the context packages. We need your
>> feedback and help with them!
>>
>> I have recently moved the context packages source files from the main
>> repository to a new one [1]. I think this will make easier to
>> contribute to it and have out of band releases that addresses problems
>> and supports more distributions.
>>
>> There are some changes that we have been thinking about for some time
>> and I think now is a good time to let you know them and maybe work on
>> them before 4.8.
>>
>>
>> Interface names and ifconfig
>> ============================
>>
>> Right now the network configuration scripts rely on ifconfig and the
>> assumption that the interfaces are called eth<n>. With recent
>> distributions it no longer works (for example CentOS 7 or Fedora 20).
>> ifconfig is deprecated (or has a different output than older versions)
>> and there is also the Predictable Network Interface Names [2]. This
>> means that a new system is needed to get information from the
>> interfaces and cope with the new naming scheme.
>>
>> To get information about the interfaces we can start using "ip"
>> command or use some other method you my know. Whatever we use we have
>> to move from "ifconfig".
>>
>> For naming I would like to continue to use the context generated so it
>> is backwards compatible, that is ETH0_IP, etc. My idea is to associate
>> the variables with the correct interface by looking up its MAC
>> address. The context generated has a ETH0_MAC with the MAC address
>> that OpenNebula gave to the interface, this way we can check which
>> interface has it. It may give problems with multiple interfaces with
>> the same MAC but I don't think this is a common case in OpenNebula
>> VMs.
>>
>>
>> Interface IP from MAC address
>> =============================
>>
>> You may know that by default OpenNebula generates the MAC address
>> using a prefix and the IP address assigned to it (PREFIX+IP). This is
>> nice as we could use the MAC address to set ebtables filters and such.
>> It was also used to configure the network when the contextualization
>> file was not present.
>>
>> I believe we should drop this system from the context packages:
>>
>> * context configures more things than just network
>> * the method adds complexity to the network configuration scripts
>> * it gives a false working state belief (gateway is not the one I'd configured)
>>
>> I'm not sure if someone is still using this system. Maybe a better way
>> is to just disable it by default but I think it is better to take out
>> the code [3].
>>
>>
>> Init system
>> ===========
>>
>> The context packages use the old SysV system to start. It worked more
>> or less fine but we are already having problems with some
>> distributions that just don't run it properly or at the right time and
>> it seems that everyone is moving to systemd. Maybe a new set of start
>> scripts should be tailored for each distro (and sometimes version).
>>
>>
>> Separate distribution scripts
>> =============================
>>
>> The source for the context packages was made with the idea of reusing
>> the scripts that go inside it as much as possible. There is a a "base"
>> directory with the common files for all the packages and "base_rpm"
>> and "base_deb" to put the specific files for RedHat and Debian based
>> packages. I still think we should share as much as possible but maybe
>> some more separation is needed to support more distros.
>>
>>
>> Post-install actions and dependencies
>> =====================================
>>
>> In the current version the post-install scripts do these actions:
>>
>> * delete udev rules for network and cdrom
>> * delete network configuration
>> * add the vmcontext init script to startup
>>
>> There may be other actions that could be interesting to do
>> automatically to prepare the image, for example disabling
>> NetworkManager or deleting the root password.
>>
>> Also some dependencies can be added to the context package. For
>> example it could be nice to install acpid daemons so the VM shutsdown
>> and reboots.
>>
>>
>> If you reached this far I'm sure you are interested in the context
>> packages :). We would like to tell us what do you think about these
>> ideas and give us some others that could be nice to do.
>>
>> This is also a call for help with the packages. If you think you can
>> give ideas, help with some developments or test the packages in new
>> distributions you are more than welcome.
>>
>> I think that for 4.8 we really need to change the use of "ifconfig" to
>> "ip" or some other system and at least I am going to open an issue in
>> the repository about it. If someone has some ideas or working code for
>> this it would be great. Maybe we can have a broad discussion here and
>> move the technical things to the repository tickets but we wanted to
>> hear what do you think about all of this.
>>
>> Cheers
>>
>>
>> [1] https://github.com/OpenNebula/addon-context-linux
>> [2] http://www.freedesktop.org/wiki/Software/systemd/PredictableNetworkInterfaceNames/
>> [3] https://github.com/OpenNebula/addon-context-linux/blob/master/base_rpm/etc/one-context.d/00-network#L37
>>
>>
>>
>> --
>> Javier Fontán Muiños
>> Developer
>> OpenNebula - Flexible Enterprise Cloud Made Simple
>> www.OpenNebula.org | @OpenNebula | github.com/jfontan
>> _______________________________________________
>> Dev mailing list
>> Dev at lists.opennebula.org
>> http://lists.opennebula.org/listinfo.cgi/dev-opennebula.org



-- 
Javier Fontán Muiños
Developer
OpenNebula - Flexible Enterprise Cloud Made Simple
www.OpenNebula.org | @OpenNebula | github.com/jfontan