1 - Inspect System Health

Eucalyptus provides access to the current view of service state and the ability to manipulate the state. You can inspect the service state to either ensure system health or to identify faulty services. You can modify a service state to maintain activities and apply external service placement policies.

View Service State

Use the euserv-describe-services command to view the service state. The output indicates:

  • Component type of the service
  • Partition in which the service is registered
  • Unique name of the service
  • Current view of service state
  • Last reported epoch (this can be safely ignored)
  • Service URI
  • Fully qualified name of the service (This is needed for manipulating services that did not get unique names during registration. For example: internal services like DNS) The default output includes the services that are registered during configuration, as well as information about the DNS service, if present. You can obtain additional service state information, such as internal services, by providing the -a flag.

You can also make requests to retrieve service information that is filtered by either:

  • current state (for example, )
  • host where service is registered
  • partition where service is registered
  • type of service (for example, CC or Walrus) When you investigate service failures, you can specify -events to return a summary of the last fault. You can retrieve extended information (primarily useful for debugging) by specifying -events -events-verbose .

Heartbeat Service

http://CLCIPADDRESS:8773/services/Heartbeat provides a list of components and their respective statuses. This allows you to find out if a service is enabled without requiring cloud credentials.

Modify Service State

To modify a service:

Enter the following command on the CLC, Walrus, or SC machines:

systemctl stop eucalyptus-cloud.service

On the CC, use the following command:

systemctl stop eucalyptus-cluster.service

If you want to shut down the SC for maintenance. The SC is SC00 is ENABLED and needs to be DISABLED for maintenance.

To stop SC00 first verify that no volumes or snapshots are being created and that no volumes are being attached or detached, and then enter the following command on SC00:

systemctl stop eucalyptus-cloud.service

To check status of services, you would enter:

euserv-describe-services

When maintenance is complete, you can start the eucalyptus-cloud process on SC00 , which will enter the DISABLED state by default.

systemctl start eucalyptus-cloud.service

Monitor the state of services using euserv-describe-services until SC00 is ENABLED .

2 - View User Resources

To see resource use by your cloud users, Eucalyptus provides the following commands with the flag.

  • : Returns information about security groups in your account, including output type identifier, security group ID, security group name, security group description, output type identifier, account ID of the group owner, name of group granting permission, type of rule, protocol to allow, start of port range, end of port range, source (for ingress rules) or destination (for egress rules), and any tags assigned to the security group.
  • : Returns information about your instances, including output type identifier, reservation ID, name of each security group the instance is in, output type identifier, instance ID for each running instance, EMI ID of the image on which the instance is based, public DNS name associated with the instance (for instances in the running state), private DNS name associated with the instance (for instances in running state), instance state, key name, launch index, instance type, launch time, availability zone, kernel ID, ramdisk ID, monitoring state, public IP address, private IP address, type of root device (ebs or instance-store), placement group the cluster instance is in, virtualization type (paravirtual or hvm), any tags assigned to the instance, hypervisor type, block device identifier for each EBS volume the instance is using, along with the device name, the volume ID, and the timestamp.
  • : Returns information about key pairs available to you, including keypair identifier, keypair name, and private key fingerprint.
  • : Returns information about EBS snapshots available to you, including snapshot identifier, ID of the snapshot, ID of the volume, snapshot state (pending, completed, error), timestamp when snapshot initiated, percentage of completion, ID of the owner, volume sized, description, and any tags assigned to the snapshot.
  • : Describes your EBS volumes, including volume identifier, volume ID, size of the volume in GiBs, snapshot from which the volume was created, availability zone, volume state (creating, available, in-use, deleting, deleted, error), timestamp of the volume creation, and any tags assigned to the volume.

3 - Change Network Configuration

Change Network Configuration

You might want to change the original network configuration of your cloud. To change your network configuration, perform the tasks listed in this topic.Log in to the CLC and open the /etc/eucalyptus/eucalyptus.conf file. Navigate to the Networking Configuration section and make your edits. Save the file. Restart the Cluster Controller.

systemctl restart eucalyptus-cluster.service

3.1 - Networking Configuration Options

All network-related options specified in /etc/eucalyptus/eucalyptus.conf use the prefix VNET_. The most commonly used VNET options are described in the following table.

OptionDescriptionComponent
VNET_BRIDGEThis is the name of the bridge interface to which instances’ network interfaces should attach. A physical interface that can reach the CC must be attached to this bridge. Common setting for KVM is br0.Node Controller
VNET_DHCPDAEMONThe ISC DHCP executable to use. This is set to a distro-dependent value by packaging. The internal default is /usr/sbin/dhcpd3.Node Controller
VNET_MODEThe networking mode in which to run. The same mode must be specified on all CCs and NCs in your cloud. Valid values: EDGEAll CCs and NCs
VNET_PRIVINTERFACEThe name of the network interface that is on the same network as the NCs. Default: eth0Node Controller
VNET_PUBINTERFACEThis is the name of the network interface that is connected to the same network as the CC. Depending on the hypervisor’s configuration this may be a bridge or a physical interface that is attached to the bridge. Default: eth0Node Controller

4 - Add a Node Controller

If you want to increase your system’s capacity, you’ll want to add more Node Controllers (NCs).To add an NC, perform the following tasks:

Log in to the CLC and enter the following command:

clusteradmin-register-nodes node0_IP_address ... [nodeN_IP_address]

When prompted, enter the password to log into each node. Eucalyptus requires this password to propagate the cryptographic keys.

5 - Migrate Instances Between Node Controllers

In order to ensure optimal system performance, or to perform system maintenance, it is sometimes necessary to move running instances between Node Controllers (NCs). You can migrate instances individually, or migrate all instances from a given NC.

To migrate a single instance to another NC, enter the following command:

euserv-migrate-instances -i INSTANCE_ID

You can also optionally specify --include-dest HOST_NC_IP or --exclude-dest HOST_NC_IP , to ensure that the instance is migrated to one of the specified NCs, or to avoid migrating the instance to any of the specified NCs. These flags may be used more than once to specify multiple NCs.

To migrate all instances away from an NC, enter the following command:

euserv-migrate-instances --source HOST_NC_IP

You can also optionally specify euserv-modify-service -s stop HOST_NC_IP , to stop the specified NC and ensure that no new instances are started on that NC while the migration occurs. This allows you to safely remove the NC without interrupting running instances. The NC will remain in the DISABLED state until it is explicitly enabled using euserv-modify-service -s start HOST_NC_IP .

In some cases, timeouts may cause a migration to initially fail. Run the command again to complete the migration.

If the migration fails, check the nc.log file on the source and destination NCs. If you see an error similar to:

libvirt: Cannot get interface MTU on 'br0': No such device (code=38)

… then ensure the NCs have the same interface and bridge device names, as described in .

6 - Remove a Node Controller

Describes how to delete NCs in your system.If you want to decrease your system’s capacity, you’ll need to decrease NC servers. To delete an NC, perform the following tasks.

Log in to the CC and enter the following command:

clusteradmin-deregister-nodes node0_IP_address ... [nodeN_IP_address]

7 - Restart Eucalyptus

Restart Eucalyptus

Describes the recommended processes to restart Eucalyptus, including terminating instances and restarting Eucalyptus components.You must restart Eucalyptus whenever you make a physical change (e.g., switch out routers), or edit the eucalyptus.conf file. To restart Eucalyptus, perform the following tasks in the order presented.

7.1 - Shut Down All Instances

To terminate all instances on all NCs perform the steps listed in this topic. To terminate all instances on all NCs:

Enter the following command:

euca-terminate-instances <instance_id>

7.2 - Restart the CLC

Log in to the CLC and enter the following command:

systemctl restart eucalyptus-cloud.service

All Eucalyptus components on this server will restart.

7.3 - Restart Walrus

Log in to Walrus and enter the following command:

systemctl restart eucalyptus-cloud.service

7.4 - Restart the CC

Log in to the CC and enter the following command:

systemctl restart eucalyptus-cluster.service

7.5 - Restart the SC

Log in to the SC and enter the following command:

systemctl restart eucalyptus-cloud.service

7.6 - Restart an NC

To restart an NC perform the steps listed in this topic.Log in to the NC and enter the following command:

systemctl restart eucalyptus-node.service

Repeat for each NC. Verify that the following is even needed. If so, replicate for other NC-tasks. You can automate the restart command for all of your NCs. Store a list of your NCs in a file called nc-hosts that looks like:

nc-host-00
nc-host-01
...
nc-host-nn

To restart all of your NCs, run the following command:

cat nc-hosts | xargs -i ssh root@{} systemctl restart eucalyptus-node.service

8 - Shut Down Eucalyptus

Shut Down Eucalyptus

Describes the recommended processes to shut down Eucalyptus.There may be times when you need to shut down Eucalyptus. This might be because of a physical failure, topological change, backing up, or making an upgrade. We recommend that you shut down Eucalyptus components in the reverse order of how you started them. To stop the system, shut down the components in the order listed.

8.1 - Shut Down All Instances

To terminate all instances on all NCs perform the steps listed in this topic.To terminate all instances on all NCs:

Enter the following command:

euca-terminate-instances <instance_id>

8.2 - Shut Down the NCs

To shut down the NCs perform the steps listed in this topic.To shut down the NCs:

Log in as root to a machine hosting an NC. Enter the following command:

systemctl stop eucalyptus-node.service

Repeat for each machine hosting an NC.

8.3 - Shut Down the CCs

To shut down the CCs:

Log in as root to a machine hosting a CC. Enter the following command:

systemctl stop eucalyptus-cluster.service

Repeat for each machine hosting a CC.

8.4 - Shut Down the SCs

To shut down the SC:

Log in as root to the physical machine that hosts the SC. Enter the following command:

systemctl stop eucalyptus-cloud.service

Repeat for any other machine hosting an SC.

8.5 - Shut Down Walrus

To shut down Walrus:

Log in as root to the physical machine that hosts Walrus. Enter the following command:

systemctl stop eucalyptus-cloud.service

8.6 - Shut Down the CLC

To shut down the CLC:

Log in as root to the physical machine that hosts the CLC. Enter the following command:

systemctl stop eucalyptus-cloud.service

9 - Disable CloudWatch

To disable CloudWatch, run the following command.

euctl cloudwatch.enable_cloudwatch_service=true