Integrate VMware Cloud Director 10.5.x with OKTA IDP

Introduction to OIDC & OAuth 2.0

OpenID Connect (OIDC) is an identity authentication protocol that extends open authorization (OAuth) 2.0 to standardize the process for authenticating and authorizing users. The OAuth 2.0 protocol enables a third-party application (called a client) to access resources from a resource server (such as an API) on behalf of a user (referred to as a resource owner). The user provides the client with a limited access token, which it can use to request resources from the resource server.

The OAuth 2.0 protocol provides security through scoped access tokens, and OIDC provides user authentication and single sign-on (SSO) functionality. The access token issued by an authorization server verifies the identity and consent of the user. 

VMware Cloud Director can be integrated with an external OIDC provider to import users/groups created in the upstream IDP.  The Service Provider imports users/groups in VCD and associates them with appropriate roles.Read More

How to Delete MQTT Enabled App Launchpad in VCD

Starting with VCD 10.2 and App Launchpad 2.0.0.1, it is possible to deploy App Launchpad using MQTT for communication with VCD.

VCD 10.5 introduced a new feature called Content Hub as a replacement for App Launchpad. Service providers running VCD 10.5.x are encouraged to provide container/vm applications to tenants by integrating Content Hub with VMware MarketPlace and Helm repositories.

In this post, I will demonstrate how you can delete MQTT enabled App Launchpad extension from VCD.

Step 1: List Installed Extensions

The GET call returns a json in response listing all installed extensions and its ID. From the extensions list filter the ID of the App Launchpad extension.Read More

How to Delete Legacy App Launchpad (AMQP Enabled) from VCD

App Launchpad is a VMware Cloud Director service extension that service providers can use to create and publish catalogs of deployment-ready applications. Tenant users can then deploy the applications with a single click.

  • App Launchpad supports applications from the Bitnami applications catalog that are available in the VMware Cloud Marketplace. 
  • You can create catalogs of your custom, in-house applications and configure App Launchpad to work with these catalogs.

The older versions of the App Launchpad (<=2.0), use AMQP to communicate with VCD. Starting with App Launchpad v2.0.0.1, the MQTT protocol is also supported.  

If you are using AMQP for the App Launchpad and running version > 2.0.0, you can reconfigure App Launchpad to use the MQTT protocol. 

You have to first delete AMQP enabled App Launchpad before you can reconfigure it to use MQTT. 

Step 1: Find App Launchpad Extension ID

Read More

Delete AMQP Broker Settings in VCD

The older versions of VMware Cloud Director used AMQP protocol to exchange messages (such as system notifications or any other update) with another VCD cell. Starting with VCD 10.1, MQTT  replaced AMQP. To learn more about how VCD used MQTT, see product documentation.

If you have an environment that still uses AMQP (e.g., VCD upgraded from version <=10.1) and wants to replace it with MQTT, you must first delete the AMQP broker settings from VCD. Unfortunately, it is not currently feasible to delete the settings from the GUI and must be done through APIs.

In the VCD GUI, you only see 2 options for the AMQP broker: Edit settings and Test AMQP config. There is no delete option.

In this post, I will show what APIs you need to delete AMQP broker settings.

Step 1: Get AMQP Broker Configuration

Note: The below APIs are applicable for VCD 10.5.1. If you are running an older version of VCD, check the supported API versions that you can use.Read More

Error Publishing TMC Self-Managed to Tenants in VCD

Welcome to yet another troubleshooting post for tmc self-managed operation in VCD. In the last post, I discussed the tmc self-managed deployment issue and how I fixed it. In this post, I will discuss another issue that I encountered with the solution. 

After successfully deploying TMC Self-Managed, you must publish the solution to tenants so that they can attach their TKG clusters to TMC. When the publishing operation is performed, the TMC Self-Managed Add-On solution creates a temporary VM known as the solution agent vm, which is subsequently destroyed once the task is complete.

In my lab, the publishing task was completed for a couple of tenants and later when I tried publishing it to another tenant, the task got stuck (VCD was acting cranky at that time).

This behavior is encountered when the solutions process in the VCD cell gets killed or there is network interruption between the cell and the VCD public address during the operation execution.
Read More

Troubleshooting TMC Self-Managed Stuck Deployment in VCD

My previous blog post discussed the VCD Extension for Tanzu Mission Control and covered the end-to-end deployment steps. In this post, I will cover how to troubleshoot a stuck TMC self-managed deployment in VCD.

I was deploying TMC self-managed in a new environment, and during configuration, I made a mistake by passing an incorrect value for the DNS zone, leading to a stuck deployment that did not terminate automatically. I waited for a couple of hours for the task to fail, but the task kept on running, thus preventing me from installing it with the correct configuration.

The deployment was stalled in the Creating phase and did not fail.

On checking the pods in the tmc-local namespace, a lot of them were stuck in either ‘CreateContainerConfigError” or “CrashLoopBackOff” states.

In VCD, when I checked the failed task ‘Execute global ‘post-create’ action,” I found the installer was complaining that the tmc package installation reconciliation failed.Read More

TKG Cluster Deployment Gotchas with Node Health Check in CSE 4.2

Recently, I upgraded Container Service Extension to 4.2.0 in my lab and was trying to deploy a TKG 2.4.0 cluster with node health check enabled. The deployment got stuck after deploying one control plane and worker node, and the cluster went into an error state.

Clicking on the Events tab showed the following error:

I checked the CSE log file and the capvcd logs on the ephemeral vm (before it got deleted) and found no error that would make sense to me.

I contacted CSE Engineering to discuss this issue and opened a bug for further analysis of the logs.

Root Cause

CSE Engineering debugged the logs and found that it was a bug in the product version. Here is the summary of the analysis done by Engineering. 

Read More

How to Integrate TMC Self-Managed 1.0 with VCD

Introduction

VMware Tanzu Mission Control is a centralized hub for simplified, multi-cloud, multi-cluster Kubernetes management. It helps platform teams take control of their Kubernetes clusters with visibility across environments by allowing users to group clusters and perform operations, such as applying policies, on these groupings.

VMware launched Tanzu Mission Control Self-Managed last year for customers running their Kubernetes (Tanzu) platform in an air-gapped environment. TMC Self-Managed is designed to support organizations that prefer to maintain complete control over their multi-cluster management hub for Kubernetes to take full advantage of advanced capabilities for cluster configuration, policy management, data protection, etc.

The first couple of releases of TMC Self-Managed only supported TKG clusters that were running on vSphere. Last month, VMware announced the release of the VMware Cloud Director Extension for Tanzu Mission Control, which allows installing TMC Self-Managed in a VCD environment to manage TKG clusters deployed through the VCD Container Service Extension (CSE).Read More

VCD (10.5) Service Crashing Continuously in CSE Environment

After updating my lab’s Container Service Extension to version 4.2.0, I observed that the VMware VCD service was frequently crashing. Restarting the cell service did not help much, as the VCD user interface (UI) died again after five minutes. The cell.log was throwing below exception

You will find similar log entries in the cell-runtime.log file.

Read More

Container Service Extension 4.0 on VCD 10.x – Part 4: Tenant Operations

In the previous post in this series, I discussed the CSE configuration options that a service provider can use to provide Kubernetes-as-a-service to their tenants. In this post, I’ll go over how tenants can use the Container Service Extension plugin for Kubernetes cluster deployment in a self-service manner.

If you haven’t read the previous posts in this series, you can do so by clicking on the links provided below.

1: CSE Introduction & Architecture

2: NSX Advanced Load Balancer Configuration & VCD Integration

3: Container Service Extension Configuration by Service Provider

Log in to the tenant’s org to deploy a Kubernetes cluster. The user should be assigned the “Kubernetes Cluster Author” role. To begin with the cluster creation wizard, navigate to Home > More > Kubernetes Container Clusters and click the New button.

Select the Kubernetes runtime for the cluster. CSE 4.0 only supports Tanzu Kubernetes Grid runtime.  

Choose the Kubernetes version and give the Kubernetes cluster a name.Read More