How to deploy a CoreOS Virtual Machine pre-configured with PyVSS in minutes

CoreOS Container Linux will reach its end of life on May 26, 2020 and will no longer receive updates. https://coreos.com/os/eol/

Nowadays, there are many lightweight Linux operating systems in the market, such as VMware PhotonOS, RancherOS, CoreOS, etc. They provide many installation methods from ISO, VMDKs, AMI, OVA, and other formats to make them available in nearly any platform whether physical, virtual, or public/private cloud.The OVA format is compatible with VMware vSphere and makes deployment faster and most of OVAs provide interfaces for pre-configuring the Operating System on first boot.Introduction

CoreOS differentiates from others as an operating system designed for clustered deployments providing automation, security, and scalability for your most critical applications (Why CoreOS). CoreOS is ready to deploy applications in Docker. The Cloud-init distribution of CoreOS supports configuration from the VMware RPC API's guestinfo facility, commonly known as the VMWare Guestinfo Interface, allowing the ITS Private Cloud API injecting them upon creation.

guestinfo variable	type
`hostname`	`hostname`
`interface.<n>.name`	`string`
`interface.<n>.mac`	`MAC address`
`interface.<n>.dhcp`	`yes \| no`
`interface.<n>.role`	`public \| private`
`interface.<n>.ip.<m>.address`	`CIDR IP address`
`interface.<n>.route.<l>.gateway`	`IP address`
`interface.<n>.route.<l>.destination`	`CIDR IP address`
`dns.server.<x>`	`IP address`
`coreos.config.data`	`string`
`coreos.config.data.encoding`	`base64 \| gzip+base64`
`coreos.config.url`	`URL`

In this tutorial, we will use many of the afore listed guestinfo variables injected via the extra_config attribute mentioned in Single Virtual Machine Deployment in order to configure networking and hostname. Those variables will not contain the guestinfo prefix, because the REST API automatically adds them before applying the configuration.

Also, we will provide valid user_data compressed and encoded in base64 in the coreos.config.data variable to do the following:

Add users.
Add SSH public keys to the coreos user.
Enable Docker service.

If you are not familiar with Cloud-config, please refer to the CoreOS Cloud-config documentation.

Pre-requirements

User keys

Contents of your SSH public key:
```
cat ~/.ssh/id_rsa.pub
```
If you don't have key-pairs, please generate one by opening a Terminal session in Mac or Linux and executing the following command:
```
ssh-keygen -t rsa -b 4096
```

Configuration

We will set the hostname, interface role, static IP address, and several other network interface parameters to the VM ethernet interface matching the name values given in the following table:

guestinfo variable	value
`hostname`	`cos1`
`interface.0.name`	`en*`
`interface.0.dhcp`	`no`
`interface.0.role`	`public \| private`
`interface.0.ip.0.address`	`128.100.72.x/24`
`interface.0.route.0.gateway`	`128.100.72.1`
`dns.server.0`	`128.100.72.8`

Guestinfo configuration file

Contents will be stored in a file named guestinfo.yml in YAML format as follows:

guestinfo.yml

hostname: cos1
interface.0.role: private
dns.server.0: 128.100.100.128
interface.0.route.0.gateway: 128.100.72.1
interface.0.route.0.destination: 0.0.0.0/0
interface.0.name: en*
interface.0.dhcp: 'no'
interface.0.ip.0.address: 128.100.72.148/24
coreos.config.data.encoding: gzip+base64

Cloud-config configuration file

With respect to user-data injected to the VM and then read by CoreOS in order to configure users, services and keys, we will use the coreos.config.data variable to store the following file compressed and encoded as specified in coreos.config.data.encoding: gzip+base64.

user_data.yml

#cloud-config
hostname: "cos1"
ssh_authorized_keys:
  - "ssh-rsa ..."
coreos:
  units:
    - name: docker-tcp.socket
      command: start
      enable: true
      content: |
        [Unit]
        Description=Docker Socket for the API

        [Socket]
        ListenStream=2375
        BindIPv6Only=both
        Service=docker.service

        [Install]
        WantedBy=sockets.target
users:
  - name: "clouduser"
    passwd: "hashed_or_plain_password"
    groups:
      - "sudo"
      - "docker"
    ssh-authorized-keys:
      - "ssh-rsa ..."

For further information about all features configurable via Cloud-Config, please refer to the official CoreOS Cloud-Config documentation.

Python packages

Python 2.7.10+
Access to the ITS Private Cloud API
P yVss v0.5+: Python client for the ITS Private Cloud API
```
pip install pyvss
```
ptpython (optional) - usual python console can be used
```
pip install ptpython
```
httpie (optional) - curl or wget can be used as well
```
pip install httpie
```

This tutorial uses pyvss to create a virtual machine from an existing CoreOS OVA (coreos_production_vmware_ova.ova) stored in your personal VSKEY-STOR space. The full specification is listed below:

name: cOS-VM1
built process: image
source image: coreos_production_vmware_ova.ova
billing department: VSS
description: virtual machine created from CoreOS OVA
usage: Test
cpu and memory: 2
disks: 10 GB
networks: any available network in your account
folders: any folder available in your account
os: other26xLinux64Guest

Step-by-step guide

Generate an API access token and save it to tk.json file:

http POST https://vss-api.eis.utoronto.ca/auth/request-token -a jm > tk.json
http: password for jm@vss-api.eis.utoronto.ca:

Start ptpython or python console:
```
ptpython
```

Import VssManager, compress_encode_string, yaml and json libraries and load tk.json and guestinfo.yml into a variable

from pyvss.helper import compress_encode_string
import yaml
import json 
 
token = json.load(open('tk.json'))   # load token contents
 
user_data = open('user_data.yml', 'rb').read()  # load user data
 
extra_config = yaml.load(open('guestinfo.yml', 'rb'))  # load extra_config

Compress and encode user_data to then update the extra_config dictionary with the proper key value item:

user_data_gz_b64 = compress_encode_string(user_data) 

extra_config.update({'coreos.config.data': user_data_gz_b64})

Import and create a VssManager instance with the previously loaded token:

from pyvss.manager import VssManager
vss = VssManager(tk=token.get('token'))

Refer to the official docs or use the help built in function to display what parameters are required by the create_vm_from_image function:
```
help(vss.create_vm_from_image)
```

Define the following variables

name='cOS-vm1'
usage='Test'
os='other26xLinux64Guest'
image='images/coreos_production_vmware_ova.ova'
client
='VSS'
description='Testing CoreOS VMware Guest Info interface implementation'
disks=[20]
cpu=2
memory=2

Get target folder, networks,operating system and iso image to mount for the installation

networks = vss.get_networks(name='1072') # filtering by name
networks = [net.get('moref') for net in networks] # we need only a moref and networks has to be a list
 
folders = vss.get_folders(name='APIDemo') # filtering by name
folder = folders[0].get('moref')

Create new vm request

r = vss.create_vm_from_image(name=name, os=os, client=client, description=description, folder=folder, networks=networks, disks=disks, usage=usage, cpu=cpu, memoryGB=memory, image=image, extra_config=extra_config)

We will use wait_for_request function to get new virtual machine Uuid when provisioned:
```
uuid = vss.wait_for_request(r.get('_links').get('request'), 'vm_uuid', 'Processed')
```

Once you get the uuid, get a summary of the VM:

vss.get_vm(uuid).get('state').get('powerState')
u'poweredOff'

Power on VM
```
vss.power_on_vm(uuid)
```

Validate new VM state:

vss.get_vm_state(uuid).get('powerState')
u'poweredOn'

Get guest ip address:

vss.get_vm_guest_ip(uuid)[0].get('ipAddress')
u'128.100.72.148'

We should be OK to SSH into the VM:

ssh clouduser@128.100.72.148

CoreOS stable (1068.8.0)
Last login: Fri Oct 28 18:53:28 2016 from 128.100.102.61
clouduser@cos1 ~ $

Check if docker is running

clouduser@cos1 ~ $ docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES

Done.

ITS Private Cloud API