Last month cloud computing and systems management expert John Willis published his best of Cloud Computing for 2009 list he calls the Cloudies. I am not an expert on the latest developments in cloud computing so it was nice to get a list of the best (in his expert opinion) cloud computing tools. I was especially interested in the latest open source software and I did a little research on each of these projects to see if they had active development mailing lists, regular releases and a real community behind them. At first glance my final list read like a cast of manga characters with names like Bitnami, CollectD, Enomaly, OpenNebula, RabbitMQ and Zenoss. However they all seem to benefit from a strong development ethic, a growing community of users and the ability to address challenges associated with cloud computing.
[Update 1/19/2011: After joining Cloud.com I would add Cloudstack as one of the open source cloud computing projects to watch.]
- Chef – Chef is a relative newcomer with their project being only one year old but they are aggressively developing their project as can be seen by the frequency of their code check-ins. is an integration framework for configuration management for all types of IT infrastructure but especially popular among cloud deployments. You write source code to describe how you want each part of your infrastructure to be built, then apply those descriptions to your servers. The result is a fully automated infrastructure. Chef is professionally supported and sponsored by Opscode.
- collectd – collectd is a daemon which collects system performance statistics store the values in a variety of ways, one implementation is to store them in RRD files. What makes collectd particularly useful is it’s plugin architecture which allow the daemon to collect information from a variety of services and servers like Apache, memcache and Linux Vserver. collectd on its own is really interesting but what I really see is an opportunity for collectd to feed other systems management tools.
- Eucalyptus - Eculyptus is an acronym for Elastic Utility Computing Architecture Linking Your Programs To Useful Systems, an open-source software infrastructure for implementing “cloud computing” on clusters. The current interface to Eucalyptus is compatible with Amazon’s EC2, S3, and EBS interfaces, but the infrastructure is designed to support multiple client-side interfaces. Eucalyptus is implemented using commonly available Linux tools and basic Web-service technologies making it easy to install and maintain. Eucalyptus Systems provides consulting, training and support services.
- OpenNebula – This is perhaps the most interesting and most relevant project of the list for cloud computing, billing itself as the open source tool kit for cloud computing. OpenNebula is a tool that can be used to build any type of Cloud deployment and manage virtual infrastructure in a data-center or cluster or to combine local infrastructure with public cloud-based infrastructure, for highly scalable hosting environments. OpenNebula also supports public clouds by providing cloud interfaces to expose its functionality for virtual machine, storage and network management.
- OpenQRM – I have been a fan of OpenQRM for a long-time or at least well before the hoopla around Cloud Computing and have had the opportunity to work with technology when it was still part of a now defunct venture-backed company, Qlusters, and then spun out as a stand-alone open source project. While OpenQRM is not what I would consider a tool for public clouds I think it’s geared towards people delivering private virtual clusters or cloud capabilities. openQRM is a single-management console for the complete IT infrastructure and provides a well defined API which can be used to integrate third-party tools as additional plugins. What’s really interesting about OpenQRM is that it can suck up physical installations, create an image, write that image to a SAN and then run the virtual instances on demand. I think the opportunity for OpenQRM is to be able to suck those images up and then spit them out to cloud computing resources like EC2, RightScale or Rackspace Cloud. UPDATED:
- Puppet – The Ruby on Rails project is a great configuration management tool, very similar to Cfengine (which pre-dates Puppet) and Chef (which was started after Puppet). Puppet is a model-driven open source framework designed to automate the building and configuration of servers. Puppet lets you perform normal administrative tasks (such as adding users, installing packages, and updating server configurations) on any number of systems, using essentially the same code, even if those systems are running completely different operating systems.
- RabbitMQ – RabbitMQ is an enterprise messaging system released under the Mozilla public license that adhere’s to the AMQP standard (AMQP is a standard that allows middleware products to send messages to one another). What really strikes me about RabbitMQ is the activity in their community especially their mailing lists and IRC channels. This was astounding to me as I feel like I am pretty up-to-date on active open source projects but before John’s post I was in the dark about RabbitMQ. Of the ones listed so far it’s really one that I feel like there’s really something substantial there. Commercial support is available from RabbitMQ LTD.
- Zenoss - Zenoss made John’s list because of it’s Amazon Web Services monitoring capabilities and its ability to monitor all sorts of other cloud and virtual infrastructure. I am also excited about Zenoss making the list because the Zenoss Community is working to extend Zenoss Core to monitor all infrastructure you already have. I am very excited about the future of Zenoss as the community continues to expand its virtualization and cloud computing capabilities.
[Disclosure: This caught my eye since I am the community guy for Zenoss.]
I also think there are a couple of other projects that weren’t on John’s list but have both an open source and cloud computing component:
- Bitnami – BitNami.org simplifies the process of deploying web applications virtually and in the cloud. Each BitNami Stack contains an application that is fully integrated with all of the software it requires to run. BitNami Stacks are available free of charge as native installers, virtual machine images and cloud templates. Examples of BitNami-packaged applications include Drupal, Joomla!, WordPress, SugarCRM, Alfresco, Redmine, Subversion and many more. Bitnami is sponsored by the cross-platform installer vendors, Bitrock.
- Enomaly’s Elastic Computing Platform (ECP) - ECP is worth watching is as much for its firebrand leader Reuven Cohen and his expertise as it for the merits of the project . ECP is a programmable virtual cloud infrastructure for small, medium and large businesses. ECP can help you design, deploy and manage virtual applications in the cloud and in the process significantly reduce administrative and systems workload. Our browser-based dashboard enables IT personnel to simply and efficiently plan deployments, automate VM scaling and load-balancing, and analyze, configure and optimize cloud capacity in an easy to use utility. We’ve designed the ECP platform to work alongside your current virtual data center, providing additional value and cost savings.
- Ubuntu Enterprise Cloud – Ubuntu Enterprise Cloud (UEC) is included with Ubuntu Server Edition and integrates a number of open source projects (including Eucalyptus). Given the rabid Ubuntu fan base I think the UEC will immediately benefit. This gives users a turnkey package to deploy a private cloud. I also liked the he code name for Ubuntu 9.10 the latest release was Karmic Koala and as we all know Koala’s eat Eucalyptus (a clever little joke from the Canonical folks who develop Ubuntu).
I am sure there are other notable open source cloud computing solutions and tools that I haven’t mentioned and hope to add to this list going forward.
My colleague Simon Jakesch (who’s a ton smarter than me) pointed out this omission from my list (which makes me feel sheepish since I have been working with Damon Edwards whose part of the ControlTier project for the last few weeks):
- ControlTier - ControlTier is an open source, cross-platform build and deployment automation framework. ControlTier can help you to coordinate and scale service management and administration activities across multiple nodes and application tiers. Where ControlTier is shines is as an orchestrator for administrative tasks like starting and stopping services and starting other programs. Many of the contributions and commercial support come from DTO Solutions.
- Open Source Meets Cloud Computing (Information Week Government Blog)
- Lowering the barrier to open source (NASA Nebula Blog)
- Forecast for 2010: The Rise of Hybrid Clouds (GigaOm)
- Application packaging for cloud computing: A proposal (c|net’s Wisdom of Clouds blog)
- Cross-Platform Clouds Coming to Storage (MSP Mentor)