UECInstallerEnhancement

Differences between revisions 3 and 4
⇤ ← Revision 3 as of 2009-11-25 11:06:49 →
Size: 4935
Editor: 82-69-40-219
Comment: example topology
← Revision 4 as of 2009-11-25 11:58:19 → ⇥
Size: 5657
Editor: 82-69-40-219
Comment: finish initial draft
Deletions are marked like this. Additions are marked like this.
Line 25: Line 25:
 * Hierarchy of components designed around Ethernet broadcast domains, so multiple clusters are expected to be in separate broadcast domains while central infrastructure (cloud, Walrus) are expected to be in a single domain
Line 29: Line 30:
== Assumptions ==
Line 33: Line 32:
You can have subsections that better describe specific parts of the issue. This work builds on the work done in the 9.10 cycle to enable cloud setup in the installer (UbuntuSpec:foundations-karmic-cloud-setup-in-server-installer). That work permitted installation of either a "simple cluster" (CLC+Walrus+CC+SC) or a node. This will operate in much the same way, but with a greater variety of possible target systems.
Line 35: Line 34:
== Implementation == The installer should perform initial discovery to determine the topology, and should transfer public credentials from the next host up the connection tree so that that host can execute commands on the one currently being installed. Upstart jobs will complete registration at boot time, although that is explicitly out of scope for this specification; see UbuntuSpec:server-lucid-euca-remote-autoregister.
Line 37: Line 36:
This section should describe a plan of action (the "how") to implement the changes discussed. Could include subsections like: === Discovery ===
Line 39: Line 38:
=== UI Changes === We will continue to support Avahi for discovery. However, Avahi or indeed any other similar discovery system only works if broadcasts from the to-be-discovered host are visible on the discovering host. (Avahi does support "reflection" of discovery packets using a special proxy, but we ought not to require excessively complex network setup.) As such, wherever we support Avahi discovery, we will also support manual entry of a hostname or IP address. This is particularly relevant for the cloud controller location, since multiple clusters are particularly liable not to be on the same Avahi broadcast domain.
Line 41: Line 40:
Should cover changes required to the UI, or specific UI that is required to implement this Who needs to be able to discover whom:
{{attachment:discover.png|discovery paths}}
Line 43: Line 43:
=== Code Changes === Who needs to be able to connect to whom:
{{attachment:connect.png|connection paths}}
Line 45: Line 46:
Code changes should include an overview of what needs to change, and in some cases even the specific details. Currently, the installer only discovers the CC when installing the NC. We will expand this as follows:
Line 47: Line 48:
=== Migration ===  * The CLC and Walrus will each advertise their existence while running.
Line 49: Line 50:
Include:
 * data migration, if any
 * redirects from old URLs to new ones, if any
 * how users will be pointed to the new way of doing things, if necessary.		  * When the UEC installer boots:
  * Attempt to discover any of the CLC, Walrus, or CC.
  * If no CLC is discovered using Avahi:
   * Allow the user to enter a CLC address. If none is given:
    * Offer installation of CLC plus zero or more of Walrus, CC, and SC. (TODO: can't do SC without CC)
  * If a CLC was discovered, either using Avahi or by manual entry:
   * If a CC was discovered:
    * (TODO: SC? the registration notes say `SC looks for clusters on the local network, offers the user a choice of which one this SC "belongs" in`; also we should only offer NC if SC is installed on this cluster)
    * Offer either NC installation (default) or installation of a new CC/SC, plus Walrus if it was not discovered. This question will include the names of any discovered clusters as well as the CC addresses, to make the choice more meaningful.
   * Otherwise:
    * Offer installation of one or more of Walrus, CC, and SC. (TODO: can't do SC without CC)

 * When installing a CLC, generate a preseed file and expose it on port 80, in much the same way as we do for the CC.

 * When installing a Walrus, CC, or SC, fetch a preseed file from the CLC.
 * When installing an NC, fetch a preseed file from the CC, as before.
 * Each preseed file includes public credentials corresponding to keys owned by the host publishing it.

Finally, Avahi publication on the CC tends to have the effect of mistakenly advertising public IP addresses of nodes as well, since the CC does routing for those addresses (Bug:458904). We will add an extra TXT record to the advertisement containing the correct IP address.
Line 60: Line 76:
== Unresolved issues ==

This should highlight any issues that should be addressed in further specifications, and not problems with the specification itself; since any specification with problems cannot be approved.

== BoF agenda and discussion ==

{{{
== Discovery ==

Continue to support Avahi for discovery, but need to support manual entry of CLC location since CCs may not be on the same Avahi broadcast domain

"Who needs to be able to discover whom":
digraph euca {
 CLC;
 Walrus;
 CC1;
 SC1;
 CC2;
 SC2;
 NC1a;
 NC2a;
 CLC -> Walrus;
 CLC -> CC1;
 CLC -> SC1;
 CLC -> CC2;
 CLC -> SC2;
 CC1 -> NC1a;
 CC2 -> NC2a;
}

"Who needs to be able to connect to whom":
digraph euca {
 CLC;
 Walrus;
 CC1;
 SC1;
 CC2;
 SC2;
 NC1a;
 NC2a;
 CLC -> Walrus;

 CLC -> CC1;
 CLC -> SC1;
 CC1 -> NC1a;
 SC1 -> Walrus;
 NC1a -> Walrus;

 CLC -> CC2;
 CLC -> SC2;
 CC2 -> NC2a;
 SC2 -> Walrus;
 NC2a -> Walrus;
}

Where does registration need to happen:
 * NC needs to be registered on CC
 * all other components need to be registered on CLC
  * need to know which components belong to which cluster

What should the installer do?
1. discovery
2. registration?
3. push of credential

Installer should do discovery, transfer of credentials and write up config files (including discovered IPs), upstart scripts at boot should complete registration

currently push credentials to NC by way of preseed file created during cluster installation
will need to do this for other remote components (but with a preseed file on the cloud)
hierarchy of components designed around Ethernet broadcast domains, so multi-cluster setups are expected to be in separate broadcast domains while central infrastructure (cloud, walrus) are expected to be in a single domain		 == Notes from BoF ==
Line 133: Line 80:
 * MANAGED adds isolation  * MANAGED adds isolation, but requires more features from the underlying network
(The author's current belief is that we will not offer this choice.)
Line 135: Line 83:
Public/private interface selection bug: https://bugs.launchpad.net/ubuntu/+source/eucalyptus/+bug/455816

Avahi publication on cluster has a habit of picking up public IPs of nodes as well
solution: add TXT record with the IP you're actually supposed to use


Installation order:
1. CLC
2. CC1
3. SC1
4. NC1a

Offer next plausible choices:
 * If only CLC installed, offer Walrus or CC
 * If CLC and Walrus installed, offer CC
 * If CLC and CC installed, offer Walrus, SC, or new CC
 * If CLC, Walrus, CC, and SC installed, offer NC or new CC
 * ... etc.
}}}		 Public/private interface selection bug: Bug:455816

Summary

In the same way that in 9.10 we made it possible to install UEC on two servers, we will support a greater variety of configurations, such as those explained in https://help.ubuntu.com/community/UEC/Topologies. This includes updating the installer to support this.

Release Note

The UEC installer now supports installing clouds with a greater variety of topologies, including multiple clusters.

Rationale

This change expands UEC installation support to cover more sophisticated setups, as well as the simple single-cluster setup supported in 9.10. Multi-cluster support is valuable in more complex environments.

Target topologies

  • Multiple clusters
  • CLC/Walrus on single powerful machine, plus multiple CCs
  • Break up SCs to near file servers; CLC/CC elsewhere
  • Hierarchy of components designed around Ethernet broadcast domains, so multiple clusters are expected to be in separate broadcast domains while central infrastructure (cloud, Walrus) are expected to be in a single domain

  • see https://help.ubuntu.com/community/UEC/Topologies

example topology

Design

This work builds on the work done in the 9.10 cycle to enable cloud setup in the installer (foundations-karmic-cloud-setup-in-server-installer). That work permitted installation of either a "simple cluster" (CLC+Walrus+CC+SC) or a node. This will operate in much the same way, but with a greater variety of possible target systems.

The installer should perform initial discovery to determine the topology, and should transfer public credentials from the next host up the connection tree so that that host can execute commands on the one currently being installed. Upstart jobs will complete registration at boot time, although that is explicitly out of scope for this specification; see server-lucid-euca-remote-autoregister.

Discovery

We will continue to support Avahi for discovery. However, Avahi or indeed any other similar discovery system only works if broadcasts from the to-be-discovered host are visible on the discovering host. (Avahi does support "reflection" of discovery packets using a special proxy, but we ought not to require excessively complex network setup.) As such, wherever we support Avahi discovery, we will also support manual entry of a hostname or IP address. This is particularly relevant for the cloud controller location, since multiple clusters are particularly liable not to be on the same Avahi broadcast domain.

Who needs to be able to discover whom: discovery paths

Who needs to be able to connect to whom: connection paths

Currently, the installer only discovers the CC when installing the NC. We will expand this as follows:

  • The CLC and Walrus will each advertise their existence while running.
  • When the UEC installer boots:
    • Attempt to discover any of the CLC, Walrus, or CC.
    • If no CLC is discovered using Avahi:
      • Allow the user to enter a CLC address. If none is given:
        • Offer installation of CLC plus zero or more of Walrus, CC, and SC. (TODO: can't do SC without CC)
    • If a CLC was discovered, either using Avahi or by manual entry:
      • If a CC was discovered:

        • (TODO: SC? the registration notes say SC looks for clusters on the local network, offers the user a choice of which one this SC "belongs" in; also we should only offer NC if SC is installed on this cluster)

        • Offer either NC installation (default) or installation of a new CC/SC, plus Walrus if it was not discovered. This question will include the names of any discovered clusters as well as the CC addresses, to make the choice more meaningful.
      • Otherwise:
        • Offer installation of one or more of Walrus, CC, and SC. (TODO: can't do SC without CC)
  • When installing a CLC, generate a preseed file and expose it on port 80, in much the same way as we do for the CC.
  • When installing a Walrus, CC, or SC, fetch a preseed file from the CLC.
  • When installing an NC, fetch a preseed file from the CC, as before.
  • Each preseed file includes public credentials corresponding to keys owned by the host publishing it.

Finally, Avahi publication on the CC tends to have the effect of mistakenly advertising public IP addresses of nodes as well, since the CC does routing for those addresses (458904). We will add an extra TXT record to the advertisement containing the correct IP address.

Test/Demo Plan

It's important that we are able to test new features, and demonstrate them to users. Use this section to describe a short plan that anybody can follow that demonstrates the feature is working. This can then be used during testing, and to show off after release. Please add an entry to http://testcases.qa.ubuntu.com/Coverage/NewFeatures for tracking test coverage.

This need not be added or completed until the specification is nearing beta.

Notes from BoF

Network topology configuration:

  • MANAGED-NOVLAN enables all features, but no isolation between VMs
  • MANAGED adds isolation, but requires more features from the underlying network

(The author's current belief is that we will not offer this choice.)

Public/private interface selection bug: 455816

CategorySpec

FoundationsTeam/UECInstallerEnhancement (last edited 2009-12-01 18:14:40 by 82-69-40-219)