Page Comparison

...

1. Download the vss-cli configuration spec:

   View file
   name ubuntu-llm-privategpt.yaml

   and update the following attributes:

   1. machine.folder: target logical folder. List available folders with vss-cli compute folder ls

   2. metadata.client : your department client.

   3. metadata.inform: email address for automated notifications

2. Deploy with the following command:

   Code Block
   vss-cli --wait compute vm mk from-file ubuntu-llm-privategpt.yaml

3. Add a virtual GPU of 16GB, specifically the 16q profile. For more information in the profile used, check the following document How to Request a Virtual GPU

   Code Block
   vss-cli compute vm set ubuntu-llm gpu mk --profile 16q

4. Once the VM has been deployed, a confirmation email will be sent with the assigned IP address and credentials.

5. Power on virtual machine

   Code Block
   vss-cli compute vm set ubuntu-llm state on

NVIDIA Driver and Licensing

NVIDIA Driver and Licensing

1. Login to the server via ssh. Note that the username may change if you further customized the VM with cloud-init

   Code Block
   ssh -p 2222 vss-admin@XXX.XXX.XXX.XXX

2. Download the NVIDIA drivers from VKSEY-STOR:

   Code Block
   scp {vss-user}@vskey-stor.eis.utoronto.ca:/ut-vss-lib/nvidia-grid-vpsherevsphere-7.0-525535.147183.0104-525535.147183.0501-529538.1967/Guest_Drivers/nvidia-linux-grid-525535_525535.147183.0501_amd64.deb /tmp/

3. Install the drivers as privileged user:

   Code Block
   apt install dkms nvtop dpkg -i  apt install /tmp/nvidia-linux-grid-525535_525535.147183.0501_amd64.deb

4. Create directory: ClientConfigToken

   Code Block
   mkdir /etc/nvidia/ClientConfigToken

5. Create the NVIDIA token file:

   Code Block
   echo -n -e $(vmware-rpctool "info-get guestinfo.ut.vss.nvidia_token") > /etc/nvidia/ClientConfigToken/client_configuration_token_12-05-2023-11-26-05.tok

6. Set permissions to the NVIDIA token:

   Code Block
   chmod 744 /etc/nvidia/ClientConfigToken/client_configuration_token_12-05-2023-11-26-05.tok

7. Set the FeatureType to 2 for NVIDIA “NVIDIA RTX Virtual WorkstationWorkstation” in /etc/nvidia/gridd.conf with the following command:

   Code Block
   sed -i 's/FeatureType=0/FeatureType=2/g' /etc/nvidia/gridd.conf

8. Restart nvidia-gridd service to pick up the new license token:

   Code Block
   systemctl restart nvidia-gridd

9. Check for any Error or successful activation:

   Code Block
   journalctl -u nvidia-gridd

   output:

   Code Block

   Dec 13 11:23:20 ubu-llm systemd[1]: Stopped NVIDIA Grid Daemon. Dec 13 11:23:20 ubu-llm systemd[1]: Starting NVIDIA Grid Daemon... Dec 13 11:23:20 ubu-llm systemd[1]: Started NVIDIA Grid Daemon. Dec 13 11:23:20 ubu-llm nvidia-gridd[2017]: Started (2017) Dec 13 11:23:20 ubu-llm nvidia-gridd[2017]: vGPU Software package (0) Dec 13 11:23:20 ubu-llm nvidia-gridd[2017]: Ignore service provider and node-locked licensing Dec 13 11:23:20 ubu-llm nvidia-gridd[2017]: NLS initialized Dec 13 11:23:20 ubu-llm nvidia-gridd[2017]: Acquiring license. (Info: vss-nvidia-ls.eis.utoronto.ca; NVIDIA RTX Virtual Workstation) Dec 13 11:23:22 ubu-llm nvidia-gridd[2017]: License acquired successfully. (Info: vss-nvidia-ls.eis.utoronto.ca, NVIDIA RTX Virtual Workstation; Expiry: 2023-12-14 16:23:22 GMT) Dec 13 14:59:24 ubu-llm nvidia-gridd[2017]: License renewed successfully. (Info: vss-nvidia-ls.eis.utoronto.ca, NVIDIA RTX Virtual Workstation; Expiry: 2023-12-14 19:59:23 GMT)

10. Verify GPU status with nvidia-smi:

    Code Block

    user@test:~$ nvidia-smi Wed Dec 13 14:18:27 2023 +-----------------------------------------------------------------------------+ | NVIDIA-SMI 525.147.05 Driver Version: 525.147.05 CUDA Version: 12.0 | |-------------------------------+----------------------+----------------------+ | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |===============================+======================+======================| | 0 GRID P6-16Q On | 00000000:02:00.0 Off | N/A | | N/A N/A P8 N/A / N/A | 6426MiB / 16384MiB | 0% Default | | | | Disabled | +-------------------------------+----------------------+----------------------+ +-----------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=============================================================================| | 0 N/A N/A 3458 C ...ld/cuda/bin/ollama-runner 6426MiB | +-----------------------------------------------------------------------------+

11. You can also monitor in console the gpu usage with nvtop:

    

...