Tuesday, September 30, 2014

FutureGrid Transition Plans for Hotel and Alamo Users


With the FutureGrid project ending, here is an outline of the transition to the Hotel and Alamo resources operated under the new Chameleon project. 

First, the FutureGrid management has graciously consented to operate their portal until the end of this week. This means that until the end of this week you will be able to manage your FutureGrid account. If you need to change the password or perform any other management actions, please do so at this time.

Alamo will become unavailable on October 1st as we take it down to reconfigure it for the new project. During this time, Hotel will be operating "as is" in order to provide some resources for existing users. You will be able to access Hotel with your existing FutureGrid account -- however beginning this Monday (October 6th) your account will be "frozen", i.e., you will no longer be able to manage your FutureGrid account, so please make sure that all is in order by then. During this time, we will also be able to provide only a limited amount of support as we develop the Chameleon portal. 

We are planning to make the Chameleon portal available around the end of October. It will include tools facilitating porting your FutureGrid account to the Chameleon account.  Alamo will become available around the same time via the new project. Once Alamo becomes available, we will take down Hotel for reconfiguration. We plan to complete the transition in November. 

To keep in touch, please visit our website, www.chameleoncloud.org. We will also post regular updates there as the project gets underway.

Kate Keahey
Mathematics and CS Division, Argonne National Laboratory Computation Institute, University of Chicago

Friday, September 12, 2014

Shutdown of Nimbus Hotel clouds

Dear Hotel users, 

Early this year, we announced a move towards shutting down the Nimbus-Xen cloud on Hotel and the creation of Nimbus-KVM in addition to the OpenStack-KVM cloud already operated on this resource. However, since most of our users chose to move to the OpenStack-KVM cloud as a result of this change, we decided to operate the Nimbus clouds for a longer time allowing the community to fully transition to OpenStack.

As most of the active Nimbus users have now moved to the OpenStack, we therefore plan to shut down the Nimbus clouds on Hotel (both Xen and KVM versions) by Friday, September 19, 2014 to facilitate the reconfiguration of the physical infrastructure to support OpenStack. These resources will continue to be operated as an OpenStack cloud under the new Chameleon project. Please contact us if this action will create any issue with your current or planned use of Nimbus Hotel.

Thursday, August 21, 2014

From FutureGrid to NSFCloud


Yesterday, the National Science Foundation announced the NSFCloud awards. FutureGrid partners, University of Chicago and TACC won one of them to build an experimental testbed called Chameleon. We are excited to be able to continue serving the FutureGrid community through this new project!

The FutureGrid resources at University of Chicago and TACC, Hotel and Alamo, will continue to be operated during the first year of the Chameleon project. We will provide documentation and tools as necessary to streamline user transition from FutureGrid to Chameleon so that FutureGrid users can keep their data and access these resources easily. We will also make every effort to ensure that the resource availability under the new project overlaps with FutureGrid.

Initially the resources will be operated in roughly the same way as they are now operated in FutureGrid with the exception that the "HPC partitions" will not be supported. We will continue the process of transitioning users to the OpenStack clouds. In Spring of 2015, we expect to introduce additional capabilities allowing users to work with bare metal reconfiguration while continuing to operate OpenStack clouds for research and educational projects as before. In the Fall of 2015, the existing Hotel and Alamo resources will be supplanted by new hardware consisting of over 650 multi-core nodes equipped with OpenFlow switches and a total of 5 PB of storage. The operational model of the resources will remain the same in the essentials but will be progressively refined to support increasingly more experiments.

To find out more, please visit our website, www.chameleoncloud.org. We will also post regular updates as the project gets underway.

Monday, August 11, 2014

Important Changes!

After almost five years of operation, the FutureGrid project will come to an end on September 30th. There will be several options for current users who wish to continue their work after that date.
The Indiana University machines Xray, India, Bravo, Delta, Echo will continue running roughly as is for both education and research (with greater use of Cloudmesh tool and a different access portal), and additional possibilities via other projects will be described in a detailed announcement in the next few weeks. In particular, we expect that testbeds funded by the NSFCloud solicitation (http://www.nsf.gov/pubs/2013/nsf13602/nsf13602.htm) will be available to welcome FutureGrid users as FutureGrid is ending. These testbeds will focus on supporting research and development in cloud computing. Further High performance computing and data intensive computing users can request time on other XSEDE resources.
See https://www.xsede.org/resources/overview for a description of XSEDE resources and https://portal.xsede.org/allocations-overview for information on how to request access.

Please submit a ticket or send email to help@futuregrid.org if you need help before details on future options are available.
Thank you, Geoffrey

Tuesday, July 15, 2014

New Phantomize feature: automatic installation of tcollector sensor agent

We are happy to announce "phantomize", a Phantom feature that will automatically install and run the tcollector sensor agent on the first boot of your virtual machines thereby automatically instrumenting your VMs to provide sensor measurements.

Phantom offers autoscaling based on sensor measurements from a variety of sources, including user's virtual machines. To collect these measurements, Phantom relies on the tcollector sensor agent being running on each of those virtual machines. Until now, users had to manually install tcollector in their virtual machines or use an image provided by us with tcollector already installed. The former requires extra effort and the latter restricts the user to the types of images provided by us. 

The phantomize feature addresses this problem. To use it, all the user needs to do is pick the "phantomize" contextualization type in their launch configuration settings. The only requirement is that the user's virtual machine image is capable of downloading and executing the user-data script on boot.

Phantomize has been tested successfully with Debian, Fedora, and Ubuntu virtual machines on FutureGrid clouds running Nimbus and OpenStack.

Friday, June 6, 2014

OpenStack Hotel now supports the native OpenStack APIs

We have changed the configuration of the OpenStack Hotel cloud so that users can now access the native OpenStack APIs over secure HTTP connections.

You can download your OpenStack credentials file from the web interface via the "Access & Security" link in the left of any page and then click on the "API Access" link on the top.

Wednesday, June 4, 2014

FutureGrid multi-cloud VM image generator now available

We are happy to announce the alpha release of our newest tool for FutureGrid users: a multi-cloud VM image generator.

FutureGrid offers access to multiple clouds based on several different technologies (Nimbus, OpenStack, and Eucalyptus) using different hypervisors (Xen or KVM). Users can also supplement the use of FutureGrid resources by bursting out to commercial clouds such as Amazon EC2. While this allows users to use multiple clouds, such access is often hard to leverage as VM images are generally not portable across different formats and cloud providers.

This presents users with a few problems. First, moving from one cloud to another means creating a new image; this is time-consuming and error-prone. Second, users typically want the VM images to represent a consistent environment independently of what type of cloud the image is deployed on; this is hard to achieve using a manual configuration process as even small differences in configuration can have significant consequences. Third, even if the user does produce a set of images that are initially consistent, as images subsequently evolve it is hard to keep track of which changes were applied to which image. In short, the problem is the lack of traceability and repeatability of VM image customizations.

Our image generator aims to solve these problems by providing an interface to specify a customization script that can be used to generate consistent images for many clouds. The service starts out with a set of consistent images uploaded to several clouds, applies them to those images, and creates a new VM image on each cloud.


We invite you to try our image generator by following our online tutorial, and please report any issue of request to nimbus-phantom@lists.mcs.anl.gov.

Thursday, March 6, 2014

Nimbus users on Hotel and Sierra should now start migrating to KVM

If you are currently using Nimbus on Sierra or Hotel, or if your work relies on an existing virtual machine image hosted on those clouds, then you need to take action.

In a few days, the Nimbus cloud on Sierra will become unavailable. As a result, following the recent announcement of changes in Nimbus cloud allocations on FutureGrid, the Hotel cloud will be gradually converted from Xen to KVM virtualization. In addition to being compatible with the Alamo Nimbus cloud, this allows us to support recent Linux distributions and brings better compatibility with virtual machine images built using other virtualization platforms (VirtualBox for example).

We have now made available a Nimbus cloud on Hotel using KVM virtualization. The existing Xen cloud on Hotel will remain available until the end of March to allow users to migrate their environments. To start using the Nimbus Hotel KVM cloud, extract a fresh copy of your credentials and use the hotel-kvm.conf configuration file.

As soon as possible, you should migrate your Xen images from Sierra and Hotel to a KVM cloud: either to Alamo or to the new Hotel KVM cloud.

If your virtual machine image only contains small modifications from the base environment, we advise to create a new one using the base images on those clouds.

If your virtual machine image contains heavy modifications, you can follow our instructions to convert your Xen image to KVM.

If you need any assistance for this process, please contact us through the FutureGrid help system.

Tuesday, February 18, 2014

Changes in Nimbus Cloud Allocations in FutureGrid

Dear Nimbus Users,

FutureGrid has decided to decommission the Nimbus Cloud on Sierra at San Diego Supercomputing Center. 

Over the last several years Sierra and Hotel have been the two most used FutureGrid clouds. Sierra has fulfilled an important role for many Nimbus-based projects as a complement to Hotel: both Sierra and Hotel are configured with the Xen hypervisor so it was easy for users to move their virtual machines between the clouds when one of them was down for maintenance or to configure distributed experiments. We will be sad to see Sierra go, but in the meantime we made plans to facilitate the transition for our users. 

In order to provide the same level of service, i.e., provide two closely integrated clouds such that users can move easily from one to the other, we will reconfigure Hotel such that the Nimbus Alamo cloud at TACC and the Nimbus Hotel cloud at University of Chicago support the same configuration. Since Alamo is configured with KVM, this will mean that the Hotel cloud will now also support KVM. 

What this means to you: If you are using Nimbus on Sierra or Hotel it means that you will need to convert your images from Xen to KVM. We will of course provide guides on how to make this conversion as well as base KVM images that can be configured with user software. To allow our users time to make the change we will also operate both the Nimbus-Xen and Nimbus-KVM clouds at University of Chicago side by side for a month. 

Timetable: The Hotel Nimbus-KVM cloud will become available on March 3rd and the Hotel Nimbus-Xen cloud will shut down on March 31st. The Sierra Nimbus-Xen cloud will become unavailable after March 3rd.

Both Alamo and Hotel also support OpenStack KVM-based clouds with complementary configurations used by an increasing number of projects. As you are migrating your work from the Hotel and Sierra Nimbus-Xen clouds, please consider using one of those clouds as well. The amount of physical resources allocated to Nimbus and OpenStack will be based on cloud utilization.

Thursday, January 16, 2014

OpenStack Havana is available on Hotel

We are happy to announce that our new OpenStack Havana cloud on Hotel is ready!

OpenStack is now available on Hotel for all FutureGrid users. You can access the OpenStack cloud on Hotel by using your FutureGrid username and password to log into the web interface. You will find various virtual machine images based on CentOS, Fedora, Scientific Linux, and Ubuntu. For more information follow the documentation on the FutureGrid portal.

Images available for OpenStack Havana on Hotel

We also provide an EC2-compatible API over HTTPS. Credentials for this API can be retrieved in the "Access & Security" section, under the "API Access" tab. Additionally, Nimbus Phantom also supports deploying instances on this cloud (look at our documentation to learn more about Phantom).

We have configured this cloud to behave similarly to the existing Nimbus cloud on Hotel, in order to provide a better user experience. As such, OpenStack instances get public IP addresses by default. Additionally, instances hostnames are configured to match the DNS name associated with the public IP address. For example, an instance which is assigned a public IP address of 149.165.149.2 gets configured with an internal hostname of vm-149-2.uc.futuregrid.org. Both can be used to connect to the instance via SSH from your workstation.

Any question or bug report about the new OpenStack cloud on Hotel should be sent to the FutureGrid help system.