Blog from November, 2020

We are pleased to announce we released the ITS Cloud API v3.14.2 with the following improvements and bug fixes:

Improvements

User/Personal VMDK file object, sync request and resources. (#1445)

Endpoints

  •  /v2/user/file (GET)
  •  /v2/user/file/vdmk (GET, PATCH)
  • /v2/request/vmdk-sync (GET)
  • /v2/request/vmdk-sync/<id> (GET)

Description

This improvement brings the ability submit a synchronization request to browse the vmdk files in the VSS API which were previously uploaded to the vskey-stor space. 

The first step is to upload a valid VMDK to vskey-stor. Then submit a sync request to the following URL:

curl -X PATCH https://vss-api.eis.utoronto.ca/v2/user/file/vmdk -H "Authorization: Bearer $TK"

Once the request has been processed, you will be able to browse the files as follows:

curl -X GET https://vss-api.eis.utoronto.ca/v2/user/file/vmdk -H "Authorization: Bearer $TK"
{
	"_links": {
		"self": "https://vss-api.eis.utoronto.ca/v2/user/file/vmdk"
	},
	"data": [
	{
		"created_on": "2020-11-16 Mon 11:28:52 EST",
		"data_store": "vssUser-xfers",
		"folder": "vskey/josem/50307927-887d-b04e-81a0-a8da57703590",
		"id": 24,
		"label": "vskey/josem/50307927-887d-b04e-81a0-a8da57703590/disk-0.vmdk",
		"name": "disk-0.vmdk",
		"path": "[vssUser-xfers] vskey/josem/50307927-887d-b04e-81a0-a8da57703590/disk-0.vmdk",
		"updated_on": "2020-11-16 Mon 11:28:52 EST"
	}
	],
	"meta": {
		"count": 1,
		"pages": {
		"first_url": "https://vss-api.eis.utoronto.ca/v2/user/file/vmdk?page=1&per_page=100&expand=True",
		"last_url": "https://vss-api.eis.utoronto.ca/v2/user/file/vmdk?page=1&per_page=100&expand=True",
		"next_url": null,
		"page": 1,
		"pages": 1,
		"per_page": 100,
		"prev_url": null,
		"total": 1
	},
	"time": "0.00731s",
	"user": "josem"
	}
}

VM/Add disk from User Vmdk file. (#1448)

Endpoints

  • /v2/vm/<id>/disk (POST)

Description

The improvement brings support to mount "external" vmdk drives which are useful when migrating virtual machines to the ITS Private Cloud. Before attempting to "mount" an external disk, you should have synchronized their available Vmdk files from vskey-stor. (PATCH to /v2/user/file/vmdk).

The action of "mounting" external disk is actually divided in the following steps:

  1. Copy the external disk to the target VM home.
  2. The VMDK in question will be named with the following format: %(vmx_home)s/%(vm_name)s-vsk-%(unit)s.vmdk
  3. A VMDK descriptor will be created at the final step of the copying process which will result in: %(vmx_home)s/%(vm_name)s-vsk-%(unit)s.vmdk and %(vmx_home)s/%(vm_name)s-vsk-%(unit)s-flat.vmdk.

To submit an automated mount process, submit a POST request to /v2/vm/<id>/disk adding the attribute backing_vmdk with either the vmdk id or full path with the following payload:

curl -X POST https://vss-api.eis.utoronto.ca/v2/vm/vm-1234/disk -d '[{"capacity_gb": 100, "backing_vmdk": '[vssUser-xfers] vskey/josem/50307927-887d-b04e-81a0-a8da57703590/disk-0.vmdk'}]' -H "Authorization: Bearer $TK"

It takes a fair amount of time to process depending on the VMDK size, but you can monitor the process with the request URL which was provided in previous POST action response. Once the file is copied to the target datastore, the task will automatically mount it and make it available to the virtual machine.

VM/Extra Configuration: allow set disk.EnableUUID. (#1447)

Endpoints

  • /v2/vm (POST)
  • /v2/vm/<id>/extra-config (GET, PUT)
  • /v2/vm/<id>/spec (GET)
  • /v2/vss/vm-extra-config (GET)

Description

VMWare by default does not provide information needed by udev to generate /dev/disk/by-id entries and in some Linux OS such as Ubuntu or Suse there could be entries in the syslog as follows:

failed to get udev uid: Invalid argument

The solution provided by VMware is to add the advanced setting disk.EnableUUID = "TRUE" to the virtual machine configuration file. This can be now accomplished by either including the extra_config attribute in the /vm creation payload or updating the settings in the /vm/<id>/extra-config resource.

curl -X POST https://vss-api.eis.utoronto.ca/v2/vm -d '{"usage": "Prod", "os": "ubuntu64Guest", "built_from": "os_install", "client": "EIS", "description": "Java web application.", "folder": "group-v4122", "networks": [{"network": "dvportgroup-95", "type": "vmxnet3"}, {"network": "dvportgroup-92"}], "disks": [{"capacity_gb": 100}, {"capacity_gb": 100, "backing_mode": "independent_persistent"})], "power_on": true, "extra_config": [{"disk.EnableUUID": "TRUE"}, {"customTag": "CustomTagValue"}]}' -H "Authorization: Bearer $TK"

As you can see in the previous example, the extra_config attribute also can take custom key-value tags that will be stored as guestinfo.* options. Existing Extra Config options are included in the /vm/<id>/extra-config and also in /spec resource.

The /v2/vss/vm-extra-config resource provides a list of supported extra configuration options:

curl -X GET https://vss-api.eis.utoronto.ca/v2/vss/vm-extra-config -H "Authorization: Bearer $VSS_API_TOKEN"
{
"_links": {
"self": "https://vss-api.eis.utoronto.ca/v2/vss/vm-extra-config",
"vss": "https://vss-api.eis.utoronto.ca/v2/vss"
},
"data": [
{
"description": "Enable disk UUID on virtual machines",
"option": "disk.EnableUUID"
},
{
"description": "Enable content Copy/Paste between VMRC client and Windows/Linux",
"option": "isolation.tools.copy.disable"
},
{
"description": "Enable content Copy/Paste between VMRC client and Windows/Linux",
"option": "isolation.tools.paste.disable",
"prefix": false
},
{
"description": "Enable content Copy/Paste between VMRC client and Windows/Linux",
"option": "isolation.tools.setGUIOptions.enable"
},
{
"description": "You can use the guestinfo variables to query information such as version and build information.",
"option": "guestinfo."
}
],
"meta": {
"count": 5,
"time": "0.00248s",
"user": "josem"
}
}

VM/deployment to support firmware type (efi, bios) (#1451)

Endpoints

  • /v2/vm (POST)
  • /v2/vm/<id>/spec (GET)
  • /v2/vss/vm-firmware (GET)

Description

For virtual machines runninng Windows 10 or Windows Server 2016 VM’s today, VMware HIGHLY recommends using the EFI firmware enabled. Moving from traditional BIOS/MBR to EFI (UEFI) firmware after the fact introduces some challenges later on down the line (Source VMware Blog).

From this version of the VSS API, you would be able to deploy virtual machines with an specific firmware option, either the default BIOS or EFI. 

To deploy a virtual machine with EFI, just include the attribute firmware=efi in the virtual machine creation payload POST /v2/vm:

{
	"built_from": "os_install",
	"client": "EIS",
	"cpu": 2,
	"description": "Staging Server",
	"disks": [
		{
		"capacity_gb": 127
		}
	],
	"firmware": "efi",
	"folder": "group-v12345",
	"memory": 4,
	"name": "WIN-EFI",
	"networks": [
		{
		"network": "dvportgroup-1234",
		"type": "vmxnet3"
		}
	],
	"os": "windows9Server64Guest",
	"usage": "QA",
}

For instance:

curl -x POST https://vss-api.eis.utornto.ca/v2/vm -H "Authorization: Bearer $TK" -d '{"built_from": "os_install", "client": "EIS", "cpu": 2, "description": "Staging Server", "disks": [{"capacity_gb": 127}], "firmware": "efi", "folder": "group-v12345", "memory": 4, "name": "WIN-EFI", "networks": [{"network": "dvportgroup-1234", "type": "vmxnet3"}], "os": "windows9Server64Guest", "usage": "QA"}'

VM/update Enable or Disable UEFI Secure Boot for a Virtual Machine (efi, bios) (#1452)

Endpoints

  •  /v2/vm/<id>/firmware (GET, PUT)
  • /v2/vss/vm-firmware (GET)

Description

Warning. If you have already deployed the virtual machine and would like to switch the firmware type, please test the process in a testing environment before attempting to perform it in a production VM.

To update an existent virtual machine firmware, simply make a PUT request to /v2/vm/<id>/firmware with the virtual machine powered off:

curl -x POST https://vss-api.eis.utornto.ca/v2/vm/vm-1234/firmware -H "Authorization: Bearer $TK" -d '{"value": "firmware"}'

Vm/Template conversion add more info to old/new value. (#1446)

Bug Fixes

VM/Template is marked on a domain without user permission (#1453)

VM/Disk backing_vmdk issue when mounting disk (#1454)

VSS-CLI v0.10.0 is available for download via PyPI or GitLab. Latest version includes the following improvements and bug fixes:

New Features

compute vmdkcommand to manage user vmdk files uploaded to vskey-stor (#371)

Commands:

  • compute vmdk

Just as how the user uploaded Images (ISO, VM, Floppy) work currently in the VSS API, the vmdk files uploaded to the User personal store aka vskey-stor, will show up as soon as a sync command is issued:

vss-cli compute vmdk personal sync

id : 6
status : SUBMITTED
task_id : 0bab1760-c667-48c6-a074-30597b0a1471
message : Request has been accepted for processing

The synchronization request will look for *.vmdk files in your personal store and create a reference in the VSS API which the CLI will produce when the following command is executed:

vss-cli compute vmdk personal ls

id path name
---- ----------------------------------------------------------------------- -----------------------------------
28 [vssUser-xfers] vskey/jm/50308ce8-725d-5fc0-5396-02fc57f31197/disk-2.v… disk-2.vmdk
29 [vssUser-xfers] vskey/jm/50308ce8-725d-5fc0-5396-02fc57f31197/disk-1.v… disk-1.vmdk
30 [vssUser-xfers] vskey/jm/501264bc-5d2d-3330-e0d9-562309e33331/disk-0.v… disk-0.vmdk
31 [vssUser-xfers] vskey/jm/501214bf-6869-554c-cdf6-2a0d88d21a9d/disk-0.v… disk-0.vmdk
32 [vssUser-xfers] vskey/jm/50121d83-c93b-0685-b54f-27cd8befc894/disk-0.v… disk-0.vmdk

The disks can now be usable to copy and mount with the vss-cli compute vm set disk mk.

compute vm set firmware: change virtual machine firmware configuration (#375)

Commands:

  • compute vm set firmware

There are two options to update a virtual machine firmware: BIOS and EFI. BIOS firmware is the default for new VM deployments in our environment, however as of this version of the VSS API/CLI, there is a method to change the firmware type to EFI and enable the UEFI Secure boot. Before updating the firmware, please the vm comply with the following prerequisites (source VMware):

  • Verify that the virtual machine operating system and firmware support UEFI boot.
  • EFI firmware Virtual hardware version 13 or later.
  • Operating system that supports UEFI secure boot.

In order to change the firmware type, the virtual machine must be powered off and then execute the following command:

vss-cli --wait compute vm set vm-21516 firmware efi

id : 25699
status : IN_PROGRESS
task_id : 30f1cb3f-6455-45db-95a1-4fac5b639c15
message : Request has been accepted for processing

⏳ Waiting for request 25699 to complete...
🎉 Request 25699 completed successfully:
warnings : Firmware updated from "bios" to "efi"
errors :


compute vm get firmware: display virtual machine firmware configuration (#376)

Commands:

  • compute vm get firmware

To obtain the current firmware of a given virtual machine, execute the following command:

vss-cli compute vm get 2011T-WIN2012-Tmp firmware

firmware : efi


Improvements

compute vm set extra-cfg: multiple options with a mix of guestinfo and allowed cfg (#366)

Commands:

  • compute vm set extra-cfg up
  • compute vm set extra-cfg mk

Virtual machine extra configuration management has been available in the VSS CLI for quite a while, but now, we have changed the approach and extended the feature not just for the guestinfo interface, but for a curated list of options that can be enabled. Available options are provided by the VSS API in the resource /vss/vm-extra-config. The list of allowed extra configuration options are:

  • disk.EnableUUID: Enable disk UUID on virtual machines
  • isolation.tools.copy.disable: Enable content Copy/Paste between VMRC client and Windows/Linux
  • isolation.tools.paste.disable: Enable content Copy/Paste between VMRC client and Windows/Linux
  • isolation.tools.setGUIOptions.enable: Enable content Copy/Paste between VMRC client and Windows/Linux

All other options are prefixed by guestinfo, assuming they will be consumed by the guest operating system via vmware tools.

For example, to enable disk.EnableUUID and adding a custom tag named custom.Tag=192.160.0.1, the vss-cli command would look like:

vss-cli --wait compute vm set vm-21925 extra-cfg mk disk.EnableUUID=TRUE custom.Tag=192.168.0.1

id : 25705
status : IN_PROGRESS
task_id : 11911abb-7764-4902-a17f-9fb5c1b850be
message : Request has been accepted for processing
⏳ Waiting for request 25705 to complete...
🎉 Request 25705 completed successfully:
warnings : VM extra-config options has been created disk.EnableUUID=TRUE; guestinfo.custom.Tag=192.168.0.1

To verify it, execute the counterpart get:

vss-cli compute vm get vm-21925 extra-cfg
options
---------------------------------------
{'disk.EnableUUID': 'TRUE'}
{'guestinfo.custom.Tag': '192.168.0.1'}

compute vm mk: multiple options with a mix of guestinfo and allowed cfg (#367)

compute vm mk: specify firmware type(#374)

Commands:

  • compute vm mk shell
  • compute vm mk from-clone
  • compute vm mk from-spec
  • compute vm mk from-image
  • compute vm mk from-template

Description:

Just like issue #366, you can now include multiple extra configuration options via the --extra-config option in the afore listed vm creation commands as follows:

vss-cli --wait compute vm mk shell --description "Testing vm deployment" --client EIS --os windows8Guest --iso 8 --disk 100 --net dvportgroup-11137 --folder group-v9271 --vss-service 1 --extra-config custom.Tag=192.168.0.1 --extra-config disk.EnableUUID=TRUE WIN2012-Tmp2

id : 4848
status : IN_PROGRESS
task_id : 3ad3ca99-5138-4c6c-8a21-773e0ab7773d
message : Request has been accepted for processing
⏳ Waiting for request 4848 to complete...
🎉 Request 4848 completed successfully:

warnings : Fault Domain: FD4 (domain-c66), Created in: VSS > Development > Testing > ut20170135153548 > Prd (group-v9271), Network adapter 1 (vmxnet3): 00:50:56:92:67:2a: Quarantine
errors :

Also, creating a new virtual machine with a specific firmware type can be done by including the option --firmware. For example:

vss-cli --wait compute vm mk shell --description "Testing vm deployment" --client EIS --os windows8Guest --iso 8 --disk 100 --net dvportgroup-11137 --folder group-v9271 --vss-service 1 --firmware efi WIN2012-Tmp3

id : 4849
status : IN_PROGRESS
task_id : 3ad3ca99-5138-4c6c-8a21-773e0a88173d
message : Request has been accepted for processing
⏳ Waiting for request 4849 to complete...
🎉 Request 4848 completed successfully:

warnings : Fault Domain: FD4 (domain-c66), Created in: VSS > Development > Testing > ut20170135153548 > Prd (group-v9271), Network adapter 1 (vmxnet3): 00:50:56:92:68:2f: Quarantine
errors :

compute vm set disk:  support backing_vmdk and flexible disk specification (#372)

Commands:

  • compute vm set disk mk

Adding disks to virtual machines with the mk command supports both text and JSON payload:

vss-cli compute vm set vm-21925 disk mk -i '{"capacity_gb": 100, "backing_vmdk": <id_or_path>}'

 Also, the CLI payload can specify a new attribute named backing_vmdk which has to be synchronized from vskey-stor (see issue #371). For example:

vss-cli compute vm set vm-21925 disk mk -i '{"capacity_gb": 100, "backing_vmdk": <id_or_path>}'

The specified VMDK will be copied over to the home virtual machine datastore, renamed and added as an existing hard disk. Depending on the size of the disk, this task may take longer to complete.

Update pyvss from 0.16.0-> 0.17.0 (#368)

Update docker base image upgrade to local python:3.8-alpine (#369)

Update ci docker services to use local (#370)

Update pyvss from 0.17.0-> 0.17.1 (#373)

Upgrade

Remember, VSS-CLI documentation is now available in VSS-CLI and the full Change Log is available here. Upgrade or install VSS-CLI as follows:

# using vss-cli
vss-cli upgrade 

# upgrade with pip
pip install vss-cli --upgrade

# install
pip install vss-cli 


For more information, please refer to the official documentation site.