[one-users] GRID-like functionalities with OpenNebula as a base? [Hadoop, or other possible solutions]

[Poul Kristensen]

Hi !

Does your succesfully setup mean that you did not face any problems at all
working with
all components of OpenNebula including cluster architecture, snapshots
rollback without
any io errors, sunstone working etc?

Concerning gridengine I know that medical companies like Novo Nordisk is
working
with a huge amount of data in PostgreSQL using gridengine (Sun's) which are
in the repository packages.

/Poul

IT-consultant / Linux-specialist


2012/3/20 Olivier Sallou <olivier.sallou at irisa.fr>

>
>
> Le 3/20/12 5:24 PM, biro lehel a écrit :
> > Dear all,
> >
> > After having successfully set up a small-scale (but up-and-running)
> > experimental Private Cloud (using OpenSUSE 12.1 64bit machines -also
> > VM's-, KVM as hypervisor, and ONE 3.2.0), my next task is to find a way
> > in which GRID-like services (see below) could be implemented inside an
> > OpenNebula-based Cloud. Basically, and architecturally-speaking, I think
> > I would need a PaaS layer above my existing IaaS layer.
> >
> > I am interested in the following tasks:
> >
> > *job distribution among a set of dynamically created (or, as a start,
> > fixed) VM's, initiated by a master-VM
> > * job execution on the worker-VM's and, more importantly, execution
> > monitoring of these jobs by the master-VM
> > * results gathering on the master-VM
> >
> > I have briefly looked though Hadoop, which uses MapReduce, but, having
> > no experience with it, I am not sure whether if it is the best tool to
> > use for such a use-case, and also, I don't know if it could be at all
> > used in my environment. So, my fundamental question is the following:
> > which is the best way to start on, a way that would lead to satisfying
> > my above-mentioned 3 necessities?
> > Is Hadoop perfectly compatible with OpenNebula, and could be used for
> > such purposes, or are there other and more fitting solutions or tools to
> > use? Basically, all I'm asking is, what is the most appropriate way for
> > obtaining these basic GRID-computing functionalities on a Platform
> > constructed above the Infrastructure?
> Hadoop will fit, but Hadoop is mainly focused on map/reduce jobs, and
> use of HDFS storage.
>
> There is no issue at setting up Hadoop on "top of" Open Nebula, I did
> it. IT is up to you to configure your images to setup Hadoop
> automatically (with ssh password-less communication).
>
> What you refer too would be more likely a really grid system like Torque
> ou Sun Grid Engine. Those can also be put above Open Nebula (or other
> cloud system). But, same thing, you need to create images that will
> automatically set them up (I did too with Torque)
> >
> > Any suggestions, thoughts, and hints are extremely welcome, since this
> > is a rather important and urgent matter for me.
> >
> > Thank you,
> > Lehel.
> >
> >
> >
> > _______________________________________________
> > Users mailing list
> > Users at lists.opennebula.org
> > http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
>
> --
> Olivier Sallou
> IRISA / University of Rennes 1
> Campus de Beaulieu, 35000 RENNES - FRANCE
> Tel: 02.99.84.71.95
>
> gpg key id: 4096R/326D8438  (keyring.debian.org)
> Key fingerprint = 5FB4 6F83 D3B9 5204 6335  D26D 78DC 68DB 326D 8438
>
> _______________________________________________
> Users mailing list
> Users at lists.opennebula.org
> http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opennebula.org/pipermail/users-opennebula.org/attachments/20120321/9d1ce1d7/attachment-0002.htm>