I’ve come across a few slightly odd things that I hadn’t accounted for during a recent vSphere 5.5 U2 deployment and thought it would be handy to document them. In this post (which is hopefully the last one) I’d like to cover off SSL certificates.
A lot of people don’t bother trying to deploy custom certificates because it invariably involves interaction with an in-house InfoSec team. This can be a royal pain in the arse. I understand completely. That said, getting custom certs into your vSphere environment has become a lot easier in recent times.
Firstly, there’s a few KB articles you should read:
- Deploying and using the SSL Certificate Automation Tool 5.5 (2057340)
- Configuring CA signed SSL certificates for vSphere Update Manager in vCenter Server 5.1 and 5.5 (2037581)
- Implementing CA signed SSL certificates with vSphere 5.x (2034833)
- Configuring OpenSSL for installation and configuration of CA signed certificates in the vSphere environment
- Creating a Microsoft Certificate Authority Template for SSL certificate creation in vSphere 5.x (2062108)
Here’s the output from the Certificate Automation Tool
================================================================== Main menu Enter the action you want to run 1. Plan your steps to update SSL certificates(Update Steps Planner) 2. Generate Certificate Signing Requests 3. Update Single Sign-On 4. Update Inventory Service 5. Update vCenter Server 6. Update vCenter Orchestrator(vCO) 7. Update vSphere Web Client and Log Browser 8. Update vSphere Update Manager(VUM) 9. End the update process and exit The chosen action is: 1
And here’s what the Update Steps Planner gives you to work through.
The chosen action is: 1 ================================================================== 1. Plan your steps to update SSL certificates(Update Steps Planner) Choose the services you want to update: 1. Single Sign-On 2. Inventory Service 3. vCenter Server 4. vCenter Orchestrator 5. vSphere Web Client 6. Log Browser 7. vSphere Update Manager 8. All services(listed above) 9. Return to the main menu Example: To choose the certificate update of Inventory Service, vCenter Server and vSphere Web Client you would enter: 2,3,5 You chose (enter comma-separated list of numbers): 8 Input arguments: [8] Selected services: Single Sign-On, Inventory Service, vCenter Server, vCenter Orchestrator, Web Client, Log Browser, vSphere Update Manager Detailed Plan to follow: 1. Go to the machine with Single Sign-On installed and - Update the Single Sign-On SSL certificate. 2. Go to the machine with Inventory Service installed and - Update Inventory Service trust to Single Sign-On. 3. Go to the machine with Inventory Service installed and - Update the Inventory Service SSL certificate. 4. Go to the machine with vCenter Server installed and - Update vCenter Server trust to Single Sign-On. 5. Go to the machine with vCenter Server installed and - Update the vCenter Server SSL certificate. 6. Go to the machine with vCenter Server installed and - Update vCenter Server trust to Inventory Service. 7. Go to the machine with Inventory Service installed and - Update the Inventory Service trust to vCenter Server. 8. Go to the machine with vCenter Orchestrator installed and - Update vCenter Orchestrator trust to Single Sign-On. 9. Go to the machine with vCenter Orchestrator installed and - Update vCenter Orchestrator trust to vCenter Server. 10. Go to the machine with vCenter Orchestrator installed and - Update the vCenter Orchestrator SSL certificate. 11. Go to the machine with vSphere Web Client installed and - Update vSphere Web Client trust to Single Sign-On. 12. Go to the machine with vSphere Web Client installed and - Update vSphere Web Client trust to Inventory Service. 13. Go to the machine with vSphere Web Client installed and - Update vSphere Web Client trust to vCenter Server. 14. Go to the machine with vSphere Web Client installed and - Update the vSphere Web Client SSL certificate. 15. Go to the machine with Log Browser installed and - Update the Log Browser trust to Single Sign-On. 16. Go to the machine with Log Browser installed and - Update the Log Browser SSL certificate. 17. Go to the machine with vSphere Update Manager installed and - Update the vSphere Update Manager SSL certificate. 18. Go to the machine with vSphere Update Manager installed and - Update vSphere Update Manager trust to vCenter Server.
And then you have a nice list of stuff to work through. I’m not going to dump the whole process here, but here’s a grab of what updating your vCenter cert looks like.
==================================================================
Main menu
Enter the action you want to run
1. Plan your steps to update SSL certificates(Update Steps Planner)
2. Generate Certificate Signing Requests
3. Update Single Sign-On
4. Update Inventory Service
5. Update vCenter Server
6. Update vCenter Orchestrator(vCO)
7. Update vSphere Web Client and Log Browser
8. Update vSphere Update Manager(VUM)
9. End the update process and exit
The chosen action is: 5
==================================================================
5. Update the vCenter Server SSL Certificate
1. Update the vCenter Server Trust to Single Sign-On
2. Update the vCenter Server SSL Certificate
3. Update the vCenter Server Trust to Inventory Service
4. Rollback to the previous vCenter Server SSL Certificate
5. Return to the main menu to update other services
The chosen service is: 2
[Thu 28/05/2015 - 10:39:54.86]: The services that are restarted as a part of this operation are: VMware VirtualCenter Server, VMware VirtualCenter Management Webservices and VMware vSphere Profile-Driven Storage Service.
Enter location to the new vCenter Server SSL chain: C:\Install\ssl-certificate-updater-tool-1308332\vCenterServer-VC4002\chain.pem
Enter location to the new vCenter Server private key: C:\Install\ssl-certificate-updater-tool-1308332\vCenterServer-VC4002\rui.key
Enter vCenter Server administrator user name: domain\svc_vmware
Enter vCenter Server administrator password (will not be echoed):
"Important: Enter the password carefully. The Certificate Automation Update Tool does not check the validity of the vCenter Server database password."
"A blank or incorrect password will leave the system in an inconsistent state, which will cause the vCenter Server to become unavailable. "
"If the system becomes unstable due to a bad password, see the Troubleshooting Section of KB 2041600."
Enter the vCenter Server original database password (will not be echoed):
Enter Single Sign-On Administrator user: Administrator@vsphere.local
Enter Single Sign-On Administrator password (will not be echoed):
[.] WARNING: Certificate's `CN=VC4002.racqgroup.local, OU=vCenterServer-VC4002, O=Company, L=Location, ST=QLD, C=AU' signature uses weak one-way h
ash (SHA-1). In a secure environment it is recommended to use SHA2-256 or a stronger hash algorithm.
[.] The supplied certificate chain is valid.
Loading 'screen' into random state - done
"Restarting services... (This can take some time)"
"Stopping vCenter Web Services..."
"Stopping vCenter Server..."
"Starting vCenter Server and other services..."
[Thu 28/05/2015 - 10:45:42.32]: Last operation update vCenter Server SSL certificate completed successfully.
[Thu 28/05/2015 - 10:45:42.33]: Go to the next step in the plan that was received from Update Steps Planner.
Once you’ve had your way with vCenter, etc, you can do your ESXi hosts. The following link has info on that – Configuring CA signed certificates for ESXi 5.x hosts, and you can grab the appropriate version of Win32 OpenSSL from here. Here’s what it looks like when you use OpenSSL to generate the requests for your ESXi hosts.
Microsoft Windows [Version 6.1.7601] Copyright (c) 2009 Microsoft Corporation. All rights reserved. C:\Users\Player1>cd \ C:\>cd OpenSSL\bin C:\OpenSSL\bin>openssl req -new -nodes -out rui.csr -keyout rui-orig.key -config openssl.cfg Loading 'screen' into random state - done Generating a 2048 bit RSA private key ........+++ ..........................................+++ writing new private key to 'rui-orig.key' ----- C:\OpenSSL\bin>openssl rsa -in rui-orig.key -out rui.key writing RSA key C:\OpenSSL\bin>
One thing to note. I found that HA got a bit irritable until all hosts in the cluster had custom certs installed. So it’s worth turning HA off until you’re finished. If, for some reason something goes wrong wit the ESXi certs, you can re-generate the default self-signed ones with the following command:
/sbin/generate-certificates
Updates In some of my previous posts, I talked about a few things that I had to do to get things working. In this post, I discussed the “Missing VMware Tools ISO”. I still don’t know why the tools files were missing from the installation, but I do know that once we applied some more recent vSphere Update Manager baselines to those hosts the correct ISO files were added to the hosts.
I also covered “HP Legacy BIOS Mode and ESXi” in this post. Interestingly, you’ll need to change back to UEFI BIOS mode if you’re trying to make VirtualConnect changes to a host, as my client found out the hard way.
I also spoke about ESXi hosts and Active Directory authentication in this post. I should point out that this post by Joseph also came in handy. If you find that when you restart the services on the host it bombs out, you’ll need to manually create /var/lock/subsys. There’s a KB article from VMware that says the same thing here.
mkdir /var/lock/subsys /etc/init.d/netlogond restart /etc/init.d/lwiod restart /etc/init.d/lsassd restart
And you should then be right.