Skip to content

Open Source

OpenNebula 5.4 ‚ÄėMedusa‚Äô Beta is Out!

OpenNebula Open-source Cloud Toolkit - Tue, 06/06/2017 - 15:19

The OpenNebula project is proud to announce the availability of the first beta¬†release of OpenNebula 5.4 ‚ÄėMedusa‚Äô. This version is the third release of the OpenNebula 5 series. A significant effort has been applied in this release to stabilize features introduced in 5.2 Excession, while keeping an eye in implementing those features more demanded by the community.

As usual almost every component of OpenNebula has been reviewed to target usability and functional improvements, trying to keep API changes to a minimum to avoid disrupting ecosystem components. An important focus has been on the vCenter integration, with an enhanced network and storage management. Also, new components have been added to improve the OpenNebula experience.

A major overhaul has been applied to the vCenter integration. The team decided to go all the way and level the vCenter integration with the KVM support. This means:

  • Full storage management. Non-Persistent images are now supported as well as volatile disks. OpenNebula is now aware of all VM disks and storage quotas can be enforced. Support for linked clones and Marketplace.
  • Full network management. It is now possible to create vCenter standard and distributed port groups and even vSwitches directly from within OpenNebula. You can assign a VLAN ID to a port group created by OpenNebula.
  • Improved monitoring. Up to two orders of magnitude of speedup.
  • An enhanced import process where naming limitations in imported resources has been removed and the ability to enable VNC automatically for Wild VMs.
  • Disk resizing, VM and Templates folder selection when a VM is deployed… and many more changes!

vcenter_network_create

A new resource to implement affinity/anti affinity VM-to-VM and Host-to-Host has been added to OpenNebula, the VM Groups. A VM group is a set of related virtual machines that may impose placement constraints based on affinity and anti-affinity rules. A VM group is defined as a set of Roles. A Role defines a VM type or class, and expressions to the VM Group can be added to define affinity between VM roles, or between VM and hosts. This ensures a dynamic approach to affinity/anti affinity since new VMs can be enrolled to a particular Role at boot time, after the VM Group has been defined and other VMs added to it.

vmgroups_ilustration

To top it all, OpenNebula 5.4 brings to the table a native implementation of a consensus algorithm, which enables the High Availability deployment of the OpenNebula front-end without relying to third party components. This distributed consensus protocol provides fault-tolerance and state consistency across OpenNebula services. A consensus algorithm is built around two concepts, System State -the data stored in the database tables- and Log -a sequence of SQL statements that are consistently applied to the OpenNebula DB in all servers-. To preserve a consistent view of the system across servers, modifications to system state are performed through a special node, the leader. The servers in the OpenNebula cluster elects a single node to be the leader. The leader periodically sends heartbeats to the other servers (follower*) to keep its leadership. If a leader fails to send the heartbeat, followers promote to candidates and start a new election. This feature, with support from floating IPs and a proper Sunstone configuration, gives robustness to OpenNebula clouds. This new functionality of distributed system state is also used to implement OpenNebula federation. In both cases (Federation and HA) no support is needed from MySQL to create a clustered DB, so admins can forget about MySQL replication.

There are many other improvements in 5.4, like improved VM lifecycle, flexible resource permissions, life disk resizing, improved Ceph support, enhanced disk I/O feedback, showback cost estimate in Sunstone, flexible IPv6 definition, http proxy support for marketplace, purge tools for the OpenNebula database, resource group isolation, multiple Sunstone improvements (VNC, password dialogs, confirmation dialogs, better vCenter support, persistent labels, usability enhacenents), networking improvements, user inputs in OneFlow and many many more features to enrich your cloud experience. As with previous releases, and in order to achieve a reliable cloud management platform, the team has gone great lengths to fix reported bugs and improve general usability.

This OpenNebula release is named after the Medula Nebula, a large planetary nebula in the constellation of Gemini on the Canis Minor border. It also known as Abell 21 and Sharpless 2-274. It was originally discovered in 1955 by UCLA astronomer George O. Abell, who classified it as an old planetary nebula. The braided serpentine filaments of glowing gas suggests the serpent hair of Medusa found in ancient Greek mythology.

The OpenNebula team is now set to bug-fixing mode. Note that this is a beta release aimed at testers and developers to try the new features, and send a more than welcomed feedback for the final release. Note that being a beta there is no migration path from the previous stable version (5.2.1) nor migration path to the final stable version (5.4.0).

The OpenNebula project would like to thank the community members and users who have contributed to this software release by being active with the discussions, answering user questions, or providing patches for bugfixes, features and documentation.

The VM Groups functionality, the configurable semantics of the VM operations permissions (ADMIN, MANAGE and USE) and the improved VM history functionality were funded by BlackBerry in the context of the Fund a Feature Program. The configurable image persistency setting and the new Non-SLAAC IPv6 Address Range were funded by University of Louvain.

Relevant Links

Categories: Open Source

OpenNebula Newsletter ‚Äď May 2017

OpenNebula Open-source Cloud Toolkit - Fri, 06/02/2017 - 17:30

If you want to keep up to date with the latest news from the OpenNebula project this past month, check out this monthly edition of the most innovative open source cloud management platform.

The OpenNebulaConf US 2017 will take place this month! This 19th an 20th part of the OpenNebula team will be in Cambridge, Massachusetts. You can already check the agenda with keynotes by Akamai, Telefonica and Harvard. Also, the keynote speakers for the OpenNebulaConf EU 2017, in Madrid this October, have already been announced, including BlackBerry, Trivago, Runtastic and the Ministry of the Flemish community.

Technology

OpenNebula 5.4 Beta is just around the corner! Codenamed Medusa, OpenNebula 5.4 is the third release of the OpenNebula 5 series. A significant effort has been applied in this release to stabilize features introduced in 5.2 Excession, while keeping an eye in implementing those features more demanded by the community.

As usual almost every component of OpenNebula has been reviewed to target usability and functional improvements, trying to keep API changes to a minimum to avoid disrupting ecosystem components. An important focus has been on the vCenter integration, with an enhanced network and storage management.
pasted-image-at-2017_06_01-08_46-am

A new resource to implement affinity/anti affinity VM-to-VM and Host-to-Host has been added to OpenNebula, the VM Groups. A VM group is a set of related virtual machines that may impose placement constraints based on affinity and anti-affinity rules.

To top it all, OpenNebula 5.4 brings to the table a native implementation of a consensus algorithm, which enables the High Availability deployment of the OpenNebula front-end without relying to third party components. This new functionality of distributed system state is also used to implement OpenNebula federation.

There are many other improvements in 5.4:

  • like improved VM lifecycle
  • flexible resource permissions
  • live disk resizing
  • improved Ceph support
  • enhanced disk I/O feedback
  • showback cost estimate in Sunstone
  • flexible IPv6 definition
  • http proxy support for marketplace
  • purge tools for the OpenNebula database
  • resource group isolation
  • user inputs in OneFlow

and many many more features to enrich your cloud experience! We’ll appreciate any feedback about the 5.4 Beta to iron out possible issues prior to the upcoming stable release.

Community

Maintaining an OpenNebula addon is not an easy task, but it is a crucial one for the success and widespread adoption of a third party component. Keeping up to date, and tested, against the latest OpenNebula version is of paramount importance to ensure its use by OpenNebula users. We want to bring you attention over the expertly crafted LXDoNe, and addon to enable OpenNebula on the management of LXD Containers. LXD is a daemon which provides a REST API to drive LXC containers. Containers are lightweight OS-level Virtualization instances, they behave like Virtual Machines but don’t suffer from hardware emulation processing penalties by sharing the kernel with the host.

Our friends at Comsysto have published a very interesting article on how to build a Private Cloud infrastructure with OpenNebula. This efforts are very important for the community since they spread the word and help to present what OpenNebula is about under other perspectives, enriching the ecosystem.

With OpenNebula we can build a simple and robust infrastructure that is focused on users needs and not on vendor specifics. Upon basic setup, user’s can already benefit from the installation because OpenNebula enables the user to be self-sustained and not to depend on System administrator or vendor to create and manage their VM’s. It is easily scalable and it can be installed on top of most hardware.

It is also very satisfying to learn how fellow developers chose OpenNebula to build their Continuous Integration infrastructures, like this fine article by the Qt folks.

Two thumbs up to all the OpenNebula advocates that support new OpenNebula users (or more seasoned users) in our support forum and in the IRC channel at freenode (#opennebula). You are a great asset to this project!

We do our best to review and incorporate those PRs aligned with the OpenNebula roadmap. We are grateful for the numerous contributions that help stabilize and enhance the OpenNebula software.

Outreach

The first US edition of the OpenNebulaConf is due in two weeks! Specifically, this June 19-20 in Cambridge, Massachusetts. There are a few places left if you are willing to attend, but you should register fast! The european edition will be held in Madrid, Spain, on October 23-24.

We had three very successful OpenNebula TechDays this past month of May. The first one was based in Madrid and hosted by Telefonica, featuring interesting talks about the use of OpenNebula in large enterprises. The TechDay in Sofia hosted by StorPool was oriented to integrations and interesting use cases on top of OpenNebula. The Barcelona TechDay hosted by CSUC, with a great turnout, was focused to the usage of OpenNebula in academic fields to foster research. Thanks a lot to those who came to the TechDays and for the organizers for hosting them!

techday-bcn-2017

The OpenNebula team is going to feature a booth in both VMworlds this year. If you are going to attend, do not forget to come by the OpenNebula booth to see a live demo of the latest stable version of your favourite CMP:

  • VMworld 2017 US, August 27-31, Las Vegas, Mandalay Bay Hotel & Convention Center
  • VMworld 2017 Europe, September 11-14, Barcelona, Fira Gran Via

Also, check out the list of official training from OpenNebula Systems for this year. If you are new to OpenNebula, or want to improve you knowledge with an in-depth OpenNebula admin course, those are the dates and locations you need to keep in mind.

Remember that you can see slides and resources from past events in our Events page. We have also created a Slideshare account where you can see the slides from some of our recent presentations.

Categories: Open Source

Performance study of a signal-extraction algorithm using different parallelisation strategies for the Cherenkov Telescope Array’s real-time-analysis software

Concurrency and Computation: Practice and Experience just published our latest work on parallelisation strategies in the context of the the Cherenkov Telescope Array project. This is a result of an ongoing collaboration with CIEMAT (Spain) and INAF (Italy) and it can be accessed here.

In this work, a signal-extraction algorithm pertaining to the Cherenkov Telescope Array’s real-time-analysis pipeline has been parallelised using SSE, POSIX Threads and CUDA. Because of the observatory’s constraints, the online analysis has to be conducted on site, on hardware located at the telescopes, and compels a search for efficient computing solutions to handle the huge amount of measured data. This work is framed in a series of studies which benchmark several algorithms of the real-time-analysis pipeline on different architectures to gain an insight into the suitability and performance of each platform.

J.L. V√°zquez-Poletti

Categories: Open Source

Talks of the TechDay Barcelona, 23 May 2017, hosted by CSUC

OpenNebula Open-source Cloud Toolkit - Wed, 05/31/2017 - 11:31

This was a really great event. Big thanks to all the attendees for being so engaged. There was a great discussion about vCenter, High Availability coming up in 5.4 and Containers!

More info of the event here: https://opennebula.org/community/techdays/techday-barcelona-2017/

We would like to thank also our host CSUC and the sponsor of the event FlyTech.

logo-flytech-ss-vectorizado
csuc_logotip

Enjoy!

techday-bcn-2017

Categories: Open Source

Talks of the TechDay Sofia, 11 May 2017, hosted by Storpool

OpenNebula Open-source Cloud Toolkit - Tue, 05/23/2017 - 08:36

Thanks to the attendees, speakers and Storpool for the successful event! Here you have the links to the presentations.

  OpenNebula at virtion, by Michael Kutzner, Owner and CEO of virtion GmbH   Disaster Recovery solution with OpenNebula and StorPool, by Boyan Krosnov,Chief of Product of StorPool  opennebula_cloud_logo_white_bg OpenNebula 5.4 Enhancements vCenter Integration, by OpenNebula Systems

Our friends at Storpool recorded the afternoon sessions and are available as video:

 

 

Enjoy!

Categories: Open Source

Talks of the TechDay Madrid, 9 May 2017, hosted by Telefónica

OpenNebula Open-source Cloud Toolkit - Thu, 05/11/2017 - 09:26

Big thank you to all the attendees and all the speakers for such a great event! The talks were really top-notch, so in case you didn’t have the chance to join us for this event, we would like share the slides with you!

 logo_telefonica_azul OpenNebula a key component of OnLife Networks by Alfonso Aurelio Carrillo Aspiazu, Chief Architect OnLife Networks, Telefónica S.A.  iecisa Automation strategy for the services provisioning (spanish), by Ruben Saavedra & Roman Martin, IECISA  csic The Science in the clouds. IAAS CSIC: present and future (spanish), by Jacinto Vera & Angel L. Rodríguez, CSIC  opennebula_cloud_logo_white_bg OpenNebula 5.4 Enhancements vCenter Integration, by OpenNebula Systems

 

Enjoy!
workshop

Categories: Open Source

2017 summer student projects are out!

Nimbus - Wed, 03/29/2017 - 20:53

We are looking for summer student to work on three different projects. The work location is the Argonne National Laboratory, near Chicago, Illinois. If you are interested in working with us on any of these, contact .

Investigating Hadoop dynamic scaling

We are creating a platform for running geospatial analysis operations in a scalable manner using cloud computing resources. In order to support large number of users with varying workloads, the platform must dynamically manage deployments of compute resources.

Hadoop is heavily used by applications running on this platform. The purpose of this project is to study scalability patterns of a geospatial analysis application, UrbanFlow, in order to derive scaling policies that will allow to dynamically vary the number of Hadoop workers in the system to provide a good response time. Since data and locality in particular is crucial to Hadoop, this project will evaluate how data placement patterns can help or prevent dynamic scaling.

The objectives of this project are:

  • Study data access and computing patterns of UrbanFlow
  • Propose scaling policies using these patterns that will optimize response time for various workloads
  • Develop a dynamic scaling engine that can enact such policies
Cloud workload trace archive

Traces from existing parallel and distributed computing systems are a useful resource for researchers to replicate real-life workloads for their experiments. However, there is little material available from cloud computing systems. We propose to develop a trace archive that will provide traces from various clouds systems combined with tools to replay them. This effort originally focuses on OpenStack clouds, but would eventually include other cloud technologies.

The objectives of this project are:

  • Define a cloud workload trace format after reviewing existing traces format. This format should be flexible enough to support other cloud technologies in the future.
  • Develop tools to extract workload from OpenStack systems, converting into the chosen trace format.
  • Develop tools to replay traces on an OpenStack deployments for experimental purposes. We will use the Chameleon testbed as a platform for deploying OpenStack.
  • Create a platform (potentially reusing existing software) for hosting traces and allowing others to contribute.
HPC/Cloud resource balancer

We are working on a platform that seeks to combine two types of scheduling: batch/best-effort scheduling typically used in HPC datacenters and on-demand scheduling available in commercial clouds. This project is developing a meta-scheduler that switches between these different modes of scheduling to ensure meeting both user satisfaction goals (in terms of resource availability) and provider satisfaction (in terms of utilization). The overall objective of this project is to use an existing implementation and a set of traces from on-demand and batch jobs and explore different usage scenarios in this context.

The relevant tasks are as follows:

  • Evaluate and potentially enhance existing implementation to add additional features
  • Define and run experiments evaluating features of the resulting platform
  • Contrast and compare the work with existing platforms such as Mesos
Categories: Open Source

Nodester CLI Reaches 1.0 Status!

We are happy to announce that the Nodester CLI (command line interface) has officially reached 1.0 status!  At the time of this article, we are actually on v1.0.2 which now supports Node.JS version 0.8.1 by default! 

Here is a list of the updates:

  • nodester app delete -> nodester app destroy (Because delete is reserved word in js) Almost every delete action is now destroy.
  • nodester app list: Alias to nodester apps (or viceverza)
  • nodester authors: Show the contributors for this tool.
  • Shorthands: e.g nodester app l maps to nodester app logs.¬†
  • Better help with headers and stuff.
  • JSHint code lint (All code is valid js)
  • Use of “use strict”
  • Partial node-0.8.1
  • Travis support
  • Bump to 1.0.0.
  • New “Hello World” app
  • More useful nodester app init command. Now it let you to choose between hello world or autoudpate remote (default to nodester)
  • Also no more “You need to restart your app” after npm installs
  • New Api nodester client with this command you are now able to interact with your personal instance easily. Running just nodester client set <endpoint> <brand> will setup your instance. Really useful.
The new CLI also creates a new and improved starter page / hello world app. This update includes links to getting started with websockets, tips and FAQs, as well as more information on the versions of Node.JS supported by our platform. 

The new CLI also walks you through deploying your app from scratch or as an existing application.  See below:

$ nodester app create 081

nodester info creating app: 081 server.js
nodester info successfully created app 081 to will run on port 19144 from server.js
nodester info run nodester app init 081 to setup this app.
nodester info ok!

$ nodester app init 081

nodester info What do you want to do:
(1) Setup a new app from scratch?
(2) You just want to setup your existent app?
note: if you choose 2 be sure that you are into your app’s dir
(1) 1

nodester info initializing git repo for 081 into folder 081
nodester warn this will take a second or two
nodester info cloning your new app in 081
nodester info clone complete
nodester info writing the default configuration
nodester info processing the initial commit
nodester info Nodester!
remote: 
remote:                           _          _
remote:           _ __   ___   __| | ___ ___| |_ ___ _ __
remote: ¬† ¬† ¬† ¬† ¬†| ‘_ \ / _ \ / _ ¬†|/ _ \ __| __/ _ \ ‘__|
remote:          | | | | (_) | (_| |  __\__ \ |_  __/ |
remote:          |_| |_|\___/ \__,_|\___|___/\__\___|_|   
remote: 
remote:           Open Source Node.js Hosting Platform
remote:               http://github.com/nodester
remote: 
remote: 
remote: Syncing repo with chroot
remote: From /node/git/topher/10991-985dc656de547235fe586e3debb0ce6b
remote:  * [new branch]      master     -> origin/master
remote: Attempting to restart your app: 10991-985dc656de547235fe586e3debb0ce6b
remote: {“status”:”restarted”}
remote: App restarted..
remote: 
remote:   \m/ Nodester out \m/
remote: 
To git@nodester.com:/node/git/topher/10991-985dc656de547235fe586e3debb0ce6b.git
 * [new branch]      master -> master

nodester info 081 started.
nodester info Some helpful app commands:

      cd ./081
      curl http://081.nodester.com/
      nodester app info
      nodester app logs
      nodester app stop|start|restart

nodester info ok!

Categories: Open Source

Nodester Running Node.JS 0.8.1

3-2-1 BLAST OFF!  Nodester now runs Node.JS version 0.8.1 by default if you upgrade to our latest CLI v1.0.2!  You can upgrade from 0.4.12 or 0.6.17 to 0.8.1 by simply opening your package.json file and changing the following entry to:

‚Äúnode‚ÄĚ:‚ÄĚ0.8.1‚ÄĚ

Commit your change to git and push it to nodester:

git commit -am ‚Äúupgrading to v0.8.1‚ÄĚ

git push

Hack the planet!  Thanks again to @_AlejandroMG for performing this update!

Categories: Open Source