Custom Certificate update/renew on External PSC, VCSA appliances and ESXi hosts on 6.5 U3 version.

-
Hello Everyone,

This post is about how to update/renew custom certificates on External PSC, vCenter appliance and ESXi hosts.


So, i am writing this post so that you can easily understand how to update the custom certificates in a distributed environment.

Let me give you a brief overview about my distributed environment which is as follows.

2 X PSC appliances 6.5 U3 version (External PSC behind F5 load balancer)
1 X vCenter appliance 6.5 U3 
3 X ESXi hosts 6.5 U3

Alright, Lets start the certificate update/renew in below order only.

1. Update/Renew certificates on External PSC appliances.
2. Update/Renew certificates on vCenter appliance.
3. Update/Renew certificates on ESXI hosts.


First, please take snapshots on all PSC and VCSA appliances.

Here, we are going to update/renew the Machine SSL certificates for PSC, VCSA and ESXi host. 

=====================================================================

                                     External PSC Certificate Update/Renew

For PSC certificates update/renew, please follow below VMware KB.


Above VMware KB article explains, how to generate .cfg file then .csr and .key files.

Once the .csr file generated then send it to your Internal Microsoft CA to generate the certificates and also ask them to share the Root CA certificate and Intermediate CAs certificates as well.

In many organisation, there might be 2 Intermediate CAs then take the certificates of both Intermediate CAs as well.


Once you have server, Root CA and Intermediate CAs certificates ready then Copy these certificates files on first PSC appliances under /certs directory and follow the above KB article section starting from this subject (Generating a certificate from an external certificate authority)

You need to update/renew certificates on all PSCs appliances one by one and reboot them as well. Please make sure all services are UP on both the PSC appliances with this command (service-control --status --all) after reboot.

Post PSCs certificate update, please STOP (service-control --stop --all) and START (service-control --start --all) all services on VCSA appliance or you can reboot the VCSA appliance as well if you want. 

Please make sure all VCSA services are UP post reboot and you are able to connect to vCenter via GUI as well.

that's all for PSCs certificate update/renew part.

================================================================

                                        VCSA Certificate Update/Renew

Let's start the certificate update on VCSA appliance. We will update the Machine SSL certificate on VCSA.

I hope you took snapshot on VCSA appliance already if not please do it now.

Please follow the below KB article for VCSA appliance certificate update/renew.


Above KB article will tell you how to generate .cfg file and then create a .csr and .key file. Once you have .csr file ready, sent it to Internal Microsoft CA to generate the server certificate and also get the relevant Root CA and Intermediate CAs certificate as we did above during PSC certificate task.

One thing, i want to highlight here that .cer and .crt extension of certificate are same, so don't confuse about these.

Once you have server certificate, ROOT CA and Intermediate CAs certificate ready then you have to create below certificates file now.

1. Machine_SSL.cer : This is a complete chain of server certificate + intermediate CAs(if applicable) + Root CA

2. Root64.cer: This is a chain of intermediate CAs(if applicable) + Root CA

3. SSL.key : this is .key file which we created earlier during .csr file creation.

Now, follow the below steps to update/renew the certificate on VCSA server. 

Login on VCSA via SSH and copy the above certificates files under directory /certs via WinSCP or SCP.
  1. Launch the vSphere 6.x Certificate Manager:

    /usr/lib/vmware-vmca/bin/certificate-manager
     2. Select Option 1 (Replace Machine SSL certificate with Custom Certificate).

     3. Provide the administrator@vsphere.local password when prompted.

      4. Select Option 1 (Continue to importing Custom certificate(s) and key(s) for Machine SSL certificate).


    5. It will ask the PSC IP address now, please provide the PSC load balancer IP or PSC                appliance IP whatever is applicable here. In my case, my PSCs are behind the F5 load            balancer so i gave Load Balancer IP.

    6. It will ask for below files location one by one like below.

Please provide valid custom certificate for Machine SSL.
File : /certs/machine_ssl.cer
 
Please provide valid custom key for Machine SSL.
File : /certs/ssl.key
 
Please provide the signing certificate of the Machine SSL certificate.
File : /certs/Root64.cer


Now, certificate update task will start and it will take some time to complete. If any issues then this certificate task will rollback the VCSA on previous certificates. If any issues occurs the you can check the below logs for more information.

/var/log/vmware/certificate-manager.log

Once the certificate task completed successfully then reboot your VCSA appliance once and make sure that all services should be UP.

That's for all for VCSA certificate update/renew part.

=================================================================

                                   ESXI Certificate Update/Renew

Let's start the ESXi custom certificate update/renew process.

ESXi don't requires custom certificate as by default they always get the VMCA certificate when added to the vCenter server. 

But in some cases specially any organisation security policy requires custom internal organisation certificates on ESXi hosts as well. So, in this case we need to apply the custom certificates on all ESXi as well.

It is not possible that we are using VMCA certificates on VCSA and but want to apply custom certificates on ESXi hosts. VCSA also needs to have custom certificates as well before update the custom certificate on ESXi hosts otherwise you cannot add the ESXi hosts to vCenter server because of different certificate providers.

Before apply the ESXi certificates, we need to change one advance setting (Certificate Management mode for ESXi hosts) on vCenter Server which is VMCA by default but we need to change it to "custom".

Below is the steps how to change this setting on vCenter Server.

Procedure:

1. Select the vCenter Server that manages the hosts and click Settings.

2. Click Advanced Settings, and click Edit.

3. In the Filter box, enter certmgmt to display only certificate management keys.

4. Change the value of vpxd.certmgmt.mode to "custom" if you intend to                    manage your own certificates.

5. Restart the vCenter Server service.


Now, lets generate the ESXi certificates from Internal Microsoft CA as per below process.


Procedure:


1. Create a .cfg file and then .csr and .key files. (Process is same as we did in PSC and VCSA case)

2. Send them to Internal Microsoft CA to generate the certificates.

3. Put the ESXi host into maintenance mode.

4. Log in to the ESXi Shell, either directly from the DCUI or from an SSH client, as a user with administrator privileges.

5. In the directory on ESXi host /etc/vmware/ssl, rename the existing certificates using the following commands.

        mv rui.crt orig.rui.crt
        mv rui.key orig.rui.key

6. Copy the certificates that you want to use to /etc/vmware/ssl.

7. Rename the new certificate and key to rui.crt and rui.key.

8. Restart the host after you install the new certificate.

Alternatively, you can put the host into maintenance mode, install the new certificate, use the Direct Console User Interface (DCUI) to restart the management agents, and set the host to exit maintenance mode.


        9. Check the certificate status on ESXI, click on ESXi on vCenter server, go to                      Configure Tab, select Certificate option under Systems, you will see the new                    updated custom certificate there. 

I saw one weird thing as well, if you still see the VMCA certificate on ESXi host then you need to disconnect and reconnect the ESXi host.

Also, when you will copy the certificate and key files on ESXi directory under /etc/vmware/ssl then edit these files with vi editor and remove the ^M character from all lines as shown in below screenshot.






That's all for this certificate update/renew post. 

I know this post seems to be very long but i hope it will be informative to you as well.


Cheers.............

Comments

  1. rickicurriFebruary 27, 2022 at 8:45 PM

    Thanks for sharing this informative I want to buy Wholesale Diploma Covers online. Keep sharing such posts.
    Custom diploma covers

    ReplyDelete

Post a Comment

Popular posts from this blog

How to migrate the N-VDS as the host switch to VDS 7.0 in NSX-T 3.x

-
Image
  Hello There, In this article, i am covering how to migrate the ESXi host switch from N-VDS to VDS 7.0 switch in NSX-T 3.2.x version. When using N-VDS as the host switch,  NSX-T  is represented as an opaque network in  vCenter Server . N-VDS owns one or more of the physical interfaces (pNICs) on the transport node, and port configuration is performed from  NSX-T Data Center . You can migrate your host switch to  vSphere  Distributed Switch (VDS) 7.0 for optimal pNIC usage and manage the networking for  NSX-T  hosts from  vCenter Server . When running  NSX-T  on a VDS switch, a segment is represented as an  NSX-T  Distributed Virtual Port Groups. Any changes to the segments on the  NSX-T  network are synchronized in  vCenter Server. We have an NSX-T environment running with NSX-T 3.2.2.1 version. this environment was designed and implemented with NSX-T 2.x version and that time we used the N-VDS as host switch configuration on the transport nodes bcuz VDS was not supported that time. 
Read more

vROPS appliances password remediation tasks failed from SDDC manager

-
Image
Issue details:-   Password remediation tasks on SDDC manager getting failed with below error. However, we are able to connect SSH with root password on the vROPS appliances, no issues with credentials. I checked Operations logs on SDDC manager and found below logs, indicating the SSH connectivity issues from SDDC to vROPS appliances. Tried to do SSH vROPS appliance from SDDC manager, getting below error... Seems some issue with ECDSA key.. Resolution:- SSH to vROPS appliance and retrieve the ECDSA ssh keys as below. Now, we have 2 options to update the correct SSH keys for vROPS appliance on SDDC manager known_Hosts files located at below location.   /root/.ssh/known_hosts   /etc/vmware/vcf/commonsvcs/known_hosts   /home/vcf/.ssh/known_hosts   /opt/vmware/vcf/commonsvcs/defaults/hosts/known_hosts First option is manually copy and paste the SSH key on all the known_hosts files and restart the SDDC manager services and then try to remediate the password again. it will be successful this
Read more

How to Import/Register a VM into vRA portal

-
Image
You can import an unmanaged virtual machine to a vRealize Automation environment. An unmanaged virtual machine exists in a vCenter but is not managed in a vRealize Automation environment and cannot be viewed in the vRA portal . Once you import an unmanaged virtual machine, the virtual machine is managed by using the vRealize Automation management interface. you can see the virtual machine on the Managed Machines tab or the Items tab. Please follow below steps to Import/Register VM into a vRA portal. Log in to vRealize Automation as a fabric administrator and as a business group manager. If you are importing virtual machines that use static IP addresses, prepare a properly configured address pool.  If you use bulk import to import a virtual machine with a static IP address that is allocated to another virtual machine, the import fails. Create a Blueprint and should not have even network attached as well. See below snapin. Create Network profile that shoul
Read more