[one-users] MPI Cluster on OpenNebula

[Quynh Le]

Hello Shankhadeep,

Thank you for your information. I am able to setup such a virtual cluster using another cloud middleware like OpenNebula so I can understand the situation. What I wanna make clear is:
- This is a kind of Master/Slave cluster: 1 head node and N worker nodes.
- We can launch a group of VMs to make N+1 VMs for the cluster. 
- Then, do you have to setup hadoop master node and worker nodes manually, OR are they (VMs) automatically configured to be "1 master + N workers".
- In this case, how many VM images you use? 1 VM image for master node, 1 for worker nodes, or 1 for all?

I'm looking forward to your sharing.

Cheers,
Quynh


________________________________
 From: Shankhadeep Shome <shank15217 at gmail.com>
To: Quynh Le <lhnquynh at yahoo.com> 
Cc: "users at lists.opennebula.org" <users at lists.opennebula.org> 
Sent: Friday, July 13, 2012 12:22 PM
Subject: Re: [one-users] MPI Cluster on OpenNebula
 

Yes, clusters are particularly easy on this environment, I work with our analytics group on hadoop clusters. The most important thing to do is install good parallel computing utilities like pdsh and have ssh equivalency setup, then creating a cluster of machine sis realyl easy because you get N machines configured the same with full access to each other. Then you use ssh based scripts to distribute your software. In our environment we use Cloudera manager to distribute and start and manage our hadoop cluster, there should be something similar for mpi clusters. The nice thing about open nebula is that you can distribute your resource allocation over generic cpu, memory and disk resources and hardware optimization is easy because you can customize one template and redeploy the entire cluster.


On Thu, Jul 12, 2012 at 3:51 AM, Quynh Le <lhnquynh at yahoo.com> wrote:

Hello,
>
>
>I intend to setup a ready-to-go MPI Cluster on OpenNebula from a VM image. Is it possible? Can each VM know its role (master or slave), as well as other VM, to fill in the mpdhost file? I don't know much about context yet. 
>
>
>
>Thanks for any explanation.
>
>
>Best regards,
>Quynh
>_______________________________________________
>Users mailing list
>Users at lists.opennebula.org
>http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opennebula.org/pipermail/users-opennebula.org/attachments/20120712/99553ece/attachment-0002.htm>