[one-users] Image upload within the core

[Csom Gyula]

Hi!

Thanks Todd for your response! It was really helpful... maybe not directly, but it gave me a useful tipp to look for cloud-oriented data transfer solutions.

I Googled the topic but so far haven't found much. After all here's the list:

* There's GridFTP you proposed.
* CDMI [1] which is a rather complex standard even donno whether it supports file uploads or not:))
* UDT [2] which is a brand new technology meanwhile seems to be the fastest solution among data transfer methods (Supercomputing Bandwidth Challenge Winner at 2006, 2008 and 2009).
* And of course one can always choose well known protocols like scp, sftp, pure http, etc. [3]

Cheers
Gyula

---

[1] CDMI: http://cloud-standards.org/wiki/index.php?title=SNIA_Cloud_Data_Management_Interface_%28CDMI%29
[2] UDT: http://en.wikipedia.org/wiki/UDP-based_Data_Transfer_Protocol, http://udt.sourceforge.net/
________________________________________
Feladó: Todd Deshane [deshantm at gmail.com]
Küldve: 2010. június 23. 23:53
Címzett: Csom Gyula
Másolatot kap: users at lists.opennebula.org
Tárgy: Re: [one-users] Image upload within the core

Hi Gyula,

I can't answer the roadmap question, but just to let you know about a
couple related projects in case you were unaware of them.

For sending large amounts of data (such as disk images), Nimbus [1]
uses GridFTP [2].

Another really promising project for creating base images and
filesystem stacks is a project called Stacklet [3,4].

Hope that helps.

Thanks,
Todd

[1] http://www.nimbusproject.org/
[2] http://www.globus.org/toolkit/data/gridftp/
[3] http://stacklet.com/
[4] http://bitbucket.org/stacklet/stacklet/

On Wed, Jun 23, 2010 at 3:09 PM, Csom Gyula <csom at interface.hu> wrote:
> Hi!
>
>
> Do you have plans to support image uploads? Is it on your 1.6 roadmap? I ask this
> in order to coordinate our (extension) development with your roadmap. Some
> background:
>
>
> Currently we are in the procces to specify our golden image management service, eg:
>
> what features to provide exactly ? how to implement it? in a maintanable manner (eg.
>
> starting with ONE v.1.4 then smoothly migrate to v.1.6)?
>
>
> We've found that one of the biggest challenges is the image upload functionality.
>
> A clean solution would implement the upload service within the core. Meanwhile
> XML-RPC used by the RequestManager is inappropriate for large file uploads.
> XML-RPC provides nothing but base64 types for binary data, meanwhile base64
> coding-decoding of large files is a pain... Also xmlrpc-c used by ONE seems to
> represent binary data in byte arrays in memory which is unacceptable for large,
> multi-GB files...
>
>
> So if we decide to implement the upload service we will likely choose an alternative
> approach, eg. a separate channel for text data (XML-RPC) and another one for large
> files (scp, http whatever seems most reasonable). Meanwhile we'd like to adhere to
> the ONE mainline as much as possible.
>
>
> BTW: We are aware of the upcomming image repository feature [1]. To our
>
> understanding (after a quick code walk through) it provides image metadata-only
>
> service.
> .
>
> Cheers,
> Gyula
>
>
> [1] http://dev.opennebula.org/issues/200
> _______________________________________________
> Users mailing list
> Users at lists.opennebula.org
> http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
>



--
Todd Deshane
http://todddeshane.net
http://runningxen.com