Terraform (henceforth TF) is an open-source infrastructure as code software tool that provides a consistent CLI workflow to manage hundreds of cloud services. Terraform codifies cloud APIs into declarative configuration files. For this page we refer to this GitHub project.
All the files and folders used here come from the same GitHub repository cloned in the KubeSpray chapter. That said, the necessary prerequisites are as follows:
- Install TF;
- Install Ansible;
- you already have a floating IP pool created;
- you have security groups enabled;
- you have a pair of keys generated that can be used to secure the new hosts.
Configuration
Inventory files
Create a directory for your cluster, mycluster for istance, by copying the existing sample-inventory and linking the hosts script, used to build the inventory based on TF state (this will be the base for subsequent Terraform commands).
| Code Block |
|---|
| language | bash |
|---|
| title | Inventory files |
|---|
| collapse | true |
|---|
|
# The following commands must be launched from inside the "kubespray" folder.
$ cp -LRp contrib/terraform/openstack/sample-inventory inventory/mycluster
$ cd inventory/mycluster
$ ln -s ../../contrib/terraform/openstack/hosts
$ ln -s ../../contrib |
TF supports various authentication methods for OpenStack. The recommended authentication method is to describe credentials in a YAML file clouds.yaml, that can be stored in the current directory (i.e. mycluster)
...
| Code Block |
|---|
| language | bash |
|---|
| title | environment variable OS_CLOUD |
|---|
| collapse | true |
|---|
|
# Insert this line in the .bashrc file for variable persistence
export OS_CLOUD=openstack
# To apply the changes
$ source .bashrc |
The construction of the cluster is driven by values found in../../contrib/terraform/openstack/variables.tf. You can consult this file to find out which variables are available for configuration, accompanied by a brief description, which values they accept and their defaults. For your cluster, edit cluster.tfvars. Let's take a look at some parameters in this file, which can serve as an example
| Code Block |
|---|
| language | bash |
|---|
| title | cluster.tfvars |
|---|
| collapse | true |
|---|
|
# list of availability zones available in your OpenStack cluster
az_list = ["nova"]
# SSH key to use for access to nodes
public_key_path = "/home/centos/.ssh/id_rsa.pub"
# image to use for bastion, masters, standalone etcd instances, and nodes
image = "centos-8-CNAF-x86_64"
# standalone etcds
number_of_etcd = 0
#flavor_etcd = "23e53bd6-be2f-4802-8126-3d7b367468f0" #m1.training
# masters
number_of_k8s_masters = 1
number_of_k8s_masters_no_etcd = 0
number_of_k8s_masters_no_floating_ip = 0
number_of_k8s_masters_no_floating_ip_no_etcd = 0
flavor_k8s_master = "23e53bd6-be2f-4802-8126-3d7b367468f0" #m1.training
#flavor_k8s_master = "3d022f94-02a0-46d9-a41a-0ee2aa95d5f1" #m1.medium
#flavor_k8s_master = "5037b00e-2917-449e-b40b-4e41c8dfea07" #m1.large
#flavor_k8s_master = "f322e78e-5ba7-4a00-ba31-fc025518a782" #m1.xlarge
# nodes
number_of_k8s_nodes = 3
number_of_k8s_nodes_no_floating_ip = 0
flavor_k8s_node = "23e53bd6-be2f-4802-8126-3d7b367468f0" #m1.training
# networking
router_id = "dec41a75-f1e1-4ec2-8c6a-bd87eb283bcc" # It is possible to use an existing router instead of creating one
network_name = "network-02"
external_net = "ac57f9a8-4349-4185-8d66-341d1b30a1bd"
subnet_cidr = "192.168.102.0/24"
floatingip_pool = "public"
master_allowed_remote_ips = ["192.168.0.0/16"]
k8s_allowed_remote_ips = ["192.168.0.0/16"] |
Note that the Ansible script will report an invalid configuration if you wind up with an even number of etcd instances since that is not a valid configuration. This restriction includes standalone etcd nodes that are deployed in a cluster along with master nodes with etcd replicas. As an example, if you have three master nodes with etcd replicas and three standalone etcd nodes, the script will fail since there are now six total etcd replicas. Moreover, You can force your existing FIPs by modifying the compute variables in ../../contrib/terraform/openstack/kubespray.tf as follows
| Code Block |
|---|
| language | bash |
|---|
| title | Force your FIPs |
|---|
| collapse | true |
|---|
|
k8s_master_fips = ["131.154.97.XXX"]
k8s_node_fips = ["131.154.97.YYY"] |
Provisioning VMs
Initialization
Before TF can operate on your cluster you need to install the required plugins. This is accomplished as follows (this should finish fairly quickly telling you TF has successfully initialized and loaded necessary modules)
| Code Block |
|---|
| language | bash |
|---|
| title | Initialization |
|---|
| collapse | true |
|---|
|
# Launch from the path inventory/mycluster
$ terraform init ../../contrib/terraform/openstack |
You can apply the Terraform configuration to your cluster with the following command issued from the usual path. The same command can be used to apply changes to an existing cluster after modifying the cluster.tfvars configuration file.
| Code Block |
|---|
| language | bash |
|---|
| title | Provisioning cluster |
|---|
| collapse | true |
|---|
|
$ terraform apply -var-file=cluster.tfvars ../../contrib/terraform/openstack |
...
| Info |
|---|
|
If you chose to create a bastion host, this script will create ../../contrib/terraform/openstack/k8s-cluster.yml with an ssh command for Ansible to be able to access your machines tunneling through the bastion's IP address. If you want to manually handle the ssh tunneling to these machines, please delete or move that file. If you want to use this, just leave it there, as ansible will pick it up automatically. |
You can destroy your new cluster with the following command
| Code Block |
|---|
| language | bash |
|---|
| title | Destroying cluster |
|---|
| collapse | true |
|---|
|
$ terraform destroy -var-file=cluster.tfvars ../../contrib/terraform/openstack |
You can enable debugging output from Terraform by setting OS_DEBUG to 1 and TF_LOG to DEBUG before running the Terraform command.