Container Service Extension 4.0 on VCD 10.x – Part 1: Introduction & Architecture

26-01-202327-01-2023 Manish Jha

Introduction

VMware Container Service is an extension to VMware Cloud Director which enables cloud providers to offer Kubernetes-as-a-Service to their tenants. CSE helps tenants quickly deploy the Tanzu Kubernetes Grid clusters in their virtual data centers in just a few clicks directly from the tenant portal. By using VMware Cloud Director Container Service Extension, customers can also use Tanzu products and services such as Tanzu Mission Control to manage their clusters.

Container Service Extension (CSE) has come a long way and with each release, the product is getting better and better. Folks who have worked on the older versions of CSE knows how painful it the setup process was and involved too many manual steps. With CSE 4.0, the provider workflow is simplified and the installation can be done in less than 30 minutes. Kudos to the CSE engineering team.

CSE 4.0 Benefits

I want to list a few benefits that CSE 4.0 offers before getting into the architecture.… Read More

Resizing TKGm Cluster in VCD

28-12-202127-02-2022 Manish Jha

This blog post explains how to resize (horizontal scale) a CSE provisioned TKGm cluster in VCD.

In my lab, I deployed a TKGm cluster with one control plane and one worker node.

[root@cse ~]# kubectl get nodes
NAME        STATUS   ROLES                  AGE   VERSION
mstr-4fxl   Ready    control-plane,master   21h   v1.21.2+vmware.1
node-cstl   Ready    <none>                 21h   v1.21.2+vmware.1

[root@cse ~]# kubectl get nodes

NAME STATUS ROLES AGE VERSION

mstr-4fxl Ready control-plane,master 21h v1.21.2+vmware.1

node-cstl Ready <none> 21h v1.21.2+vmware.1

To resize the cluster through the VCD UI, go to the Kubernetes Container Clusters page and select the TKGm cluster to resize. Click on the Resize option.

Select the number of worker nodes you want in your TKGm cluster and click the Resize button.… Read More

Error Deploying Container Service Extension 3.1.1 – No module named ‘_sqlite3’

28-12-202127-02-2022 Manish Jha

Container Service Extension 3.1.1 was released a few days back with new enhancements. The release announcements were made here and here.

Although the deployment procedure hasn’t changed much, mine was not smooth and I faced a couple of hiccups. This blog post discusses the problem I experienced and how I resolved it.

After installing VCD-CLI using pip, I was unable to execute any VCD command. The command was throwing an error as shown below:

[root@cse ~]# vcd version

Traceback (most recent call last):
  File "/root/.local/lib/python3.7/site-packages/vcd_cli/browsercookie/__init__.py", line 18, in <module>
    from pysqlite2 import dbapi2 as sqlite3
ModuleNotFoundError: No module named 'pysqlite2'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/root/.local/bin/vcd", line 5, in <module>
    from vcd_cli.vcd import vcd
  File "/root/.local/lib/python3.7/site-packages/vcd_cli/vcd.py", line 121, in <module>
    from vcd_cli import login  # NOQA
  File "/root/.local/lib/python3.7/site-packages/vcd_cli/login.py", line 24, in <module>
    from vcd_cli import browsercookie
  File "/root/.local/lib/python3.7/site-packages/vcd_cli/browsercookie/__init__.py", line 20, in <module>
    import sqlite3
  File "/usr/local/lib/python3.7/sqlite3/__init__.py",

Unable to delete TKGm clusters in VCD

28-12-202127-02-2022 Manish Jha

I encountered an issue while playing with Container Service Extension 3.1.1 in my lab where I was unable to construct TKGm clusters. During troubleshooting, I discovered that the Rights Bundle “cse:nativeCluster Entitlement” was missing certain critical rights that are newly added with CSE 3.1.1.

On attempting to delete the failed clusters, the clusters stuck in the state “DELETE:IN_PROGRESS”.

On attempting to delete the failed cluster via vcd-cli, the operation failed with the error “RDE_ENTITY_NOT_RESOLVED“

[root@cse ~]# vcd cse cluster delete mj-tkc01 --yes

Usage: vcd cse cluster delete [OPTIONS] [NAME]
Try 'vcd cse cluster delete -h' for help.

Error: [ 0870ce65-3c19-4faa-a483-9ff52f4e9f06 ] RDE_ENTITY_NOT_RESOLVED [ 556ce860-1b72-4365-b327-3b38af622eca ]

[root@cse ~]# vcd cse cluster delete mj-tkc01 --yes

Usage: vcd cse cluster delete [OPTIONS] [NAME]

Try 'vcd cse cluster delete -h' for help.

NSX ALB Integration with VCD-Part 5: Load Balancing in Action

26-12-202127-02-2022 Manish Jha

Welcome to the last post of this series. I am sure if you are following this blog series, then you have got yourself familiar with how NSX ALB integrates with VCD to provide “Load Balancing as a Service (LBaaS)”

In this post, I will demonstrate how tenants can leverage NSX ALB to create load balancer constructs (Virtual Services, Pools, etc)

If you haven’t read the previous posts in this series, I recommend you do so using the links provided below.

1: NSX ALB Integration with VCD – Supported Designs

2: NSX ALB Integration in VCD

3: Implementing Dedicated Service Engine Groups Design

4: Implementing Shared Service Engine Groups Design

Tenant vStellar has deployed a couple of servers that are connected to a routed network “Prod-GW” and have got IP addresses 192.168.40.5 and 192.168.40.6 respectively.

Both servers are running an HTTP web server and are accessible via their local IP.

The tenant is looking for load balancing these web servers by leveraging NSX ALB.… Read More

NSX ALB Integration with VCD-Part 4: Shared Service Engine Groups

26-12-202127-02-2022 Manish Jha

Welcome to the 4th part of the NSX Advanced Load Balancer Integration with VMware Cloud Director series. The first post in this series covered Service Engine design topologies, while the second covered the processes for enabling “Load Balancing as a Service” in VCD. The deployment of the Dedicated Service Engine design was demonstrated in the third post.

This post will talk about the implementation of the Shared Service Engines design.

If you haven’t read the previous posts in this series, I recommend you do so using the links provided below.

1: NSX ALB Integration with VCD – Supported Designs

2: NSX ALB Integration in VCD

3: Implementing Dedicated Service Engine Groups Design

In Shared Service Engine Group design, tenant’s Edge Gateways can leverage a common Service Engine Group for the load balancer and virtual services placement. Since VCD tenants can have overlapping org networks implemented in their respective org’s, data traffic segregation is achieved by implementing VRF’s in NSX ALB. … Read More

NSX ALB Integration with VCD-Part 3: Dedicated Service Engine Groups

11-12-202127-02-2022 Manish Jha

I discussed the supported design for NSX ALB integration with the VMware Cloud Director in the first post of this series. Part 2 of this series described how to enable “Load Balancing as a Service” in VCD.

If you missed any of the previous posts in this series, I recommend that you read them using the links provided below.

1: NSX ALB Integration with VCD – Supported Designs

2: NSX ALB Integration in VCD

This blog post is focused on implementing the Dedicated Service Engine Groups design.

The below diagram shows the high-level overview of Dedicated SEG in VCD.

In this design, the management network of Service Engine (eth0) is attached to the tier-1 gateway dedicated for NSX ALB management and provisioned by the service provider. When a Virtual Service is created by the tenant, a logical segment corresponding to the VIP network is automatically created and gets attached to the tenant’s tier-1 gateway.… Read More

NSX ALB Integration with VCD-Part 2: NSX ALB & Infra Configuration

28-11-202127-02-2022 Manish Jha

In the first post of this series, I discussed the design patterns that are supported for NSX ALB integration with VCD.

In this post, I will share the steps of the NSX ALB & Infra configuration, before implementing the supported designs.

Step 1: Configure NSX-T Constructs

1a: Deploy a couple of new Edge nodes to place the Tier-0 gateway that you will be creating for the NSX ALB consumption.

Associate the newly deployed edge nodes with the existing Edge Cluster.

1b: Create a Tier-0 and configure BGP. Also, ensure that Tier-1 connected segments are allowed to be redistributed via BGP.

1c: Create a Tier-1 gateway and associate it with the Tier-o gateway that you created in the previous step.

Ensure that the tier-1 gateway is configured to redistribute connected routes to the tier-0 gateway.

1d: Create a DHCP-enabled logical segment for the Service Engine management and connect the segment to the tier-1 gateway which you created in the previous step.… Read More

NSX ALB Integration with VCD-Part 1: Design Patterns

28-11-202127-02-2022 Manish Jha

Overview

NSX Advanced Load Balancer provides multi-cloud load balancing, web application firewall, application analytics, and container ingress services from the data center to the cloud. It is an Intent-based software load balancer that provides scalable application delivery across any infrastructure. NSX ALB provides 100% software load balancing to ensure a fast, scalable and secure application experience. It delivers elasticity and intelligence across any environment.

With the release of VCD 10.2, NSX Advanced Load Balancer integration is available for use by the tenants. Service Provider configured NSX ALB and exposes load balancing functionality to the tenants so that tenants can deploy load balancers in a self-service fashion.

The latest release of VCD (10.3.1) supports NSX ALB versions up to 21.1.2. Please check the VMware product interop matrix before planning your deployment.

In this blog post, I will be talking about the NSX ALB design patterns for VCD and the ALB integration steps with VCD.… Read More

Tanzu Kubernetes Grid multi-cloud Integration with VCD – Greenfield Installation

22-09-202123-02-2022 Manish Jha

With the release of Container Service Extension 3.0.3, service providers can integrate Tanzu Kubernetes Grid multi-cloud (TKGm) with VCD to offer Kubernetes as a Service to their tenants. TKGm integration in addition to existing support for Native K8 and vSphere with Tanzu (TKGS) has truly transformed VCD into a developer-ready cloud.

With Tanzu Basic (TKGm &TKGS) on VCD, tenants have a choice of deploying K8s in three different ways:

TKGS: K8 deployment on vSphere 7 which requires vSphere Pod Service
TKGm: Multi-tenant K8 deployments that do not need vSphere Pod Service.
Native K8: Community supported Kubernetes on VCD with CSE

By offering multi-tenant managed Kubernetes services with Tanzu Basic and VCD, Cloud providers can attract developer workloads starting with test/dev environments to their cloud. Once developers have grown confidence in the K8 solution, application owners can leverage the VCD-powered clouds to quickly deploy test/dev K8s clusters on-premise and accelerate their cloud-native app development and transition to production environments.… Read More

VCD

Container Service Extension 4.0 on VCD 10.x – Part 1: Introduction & Architecture

Introduction

CSE 4.0 Benefits

Like this:

Resizing TKGm Cluster in VCD

Like this:

Error Deploying Container Service Extension 3.1.1 – No module named ‘_sqlite3’

Like this:

Unable to delete TKGm clusters in VCD

Like this:

NSX ALB Integration with VCD-Part 5: Load Balancing in Action

Like this:

NSX ALB Integration with VCD-Part 4: Shared Service Engine Groups

Like this:

NSX ALB Integration with VCD-Part 3: Dedicated Service Engine Groups

Like this:

NSX ALB Integration with VCD-Part 2: NSX ALB & Infra Configuration

Like this:

NSX ALB Integration with VCD-Part 1: Design Patterns

Overview

Like this:

Tanzu Kubernetes Grid multi-cloud Integration with VCD – Greenfield Installation

Like this:

Introduction

CSE 4.0 Benefits

Spread the Love

Like this:

Spread the Love

Like this:

Spread the Love

Like this:

Spread the Love

Like this:

Spread the Love

Like this:

Spread the Love

Like this:

Spread the Love

Like this:

Spread the Love

Like this:

Overview

Spread the Love

Like this:

Spread the Love

Like this: