vCenter 8.0 Update 1 was released on April 18th, and I quickly jumped to upgrading my existing home lab vCenter, but ran into an issue that prevented the upgrade from completing, namely the error message “Invalid Type, expected String, instead got NoneType”.
Obviously something was wrong with my current vCenter installation, but finding out what was required to correct it wasn’t straight forward. Thanks to my partner in crime, and notorious human log file parser, Espen Ødegaard, I found the following in /var/log/vmware/applmgmt/PatchRunner.log:
2023-04-19T09:10:39.560Z vpxd:Expand ERROR vmware_b2b.patching.executor.hook_executor Patch hook 'vpxd:Expand' failed.
(...)
Traceback (most recent call last):
xml.etree.ElementTree.ParseError: not well-formed (invalid token): line 40, column 7
Note: The log output has been truncated for legibility.
Clearly something is not conformant in the vpxd.cfg file on my vCenter, on line 40.
Opening /etc/vmware-vpx/vpxd.cfg showed the following:
38 <vcls>
39 <clusters>
40 <78ed4e3c-4f28-4d27-a0d5-01182e957269>
41 <enabled>true</enabled>
42 </78ed4e3c-4f28-4d27-a0d5-01182e957269>
43 </clusters>
44 </vcls>
This is not correct, it should be referencing <domain-id> and not an UUID on line 40/42. It seems I have messed up the procedure for cleaning up vCLS VMs at some point, putting in the wrong configuration as a vCenter Advanced setting.
Info
Check Putting a Cluster in Retreat Mode for the correct procedure and how to obtain the correct domain-id
I tried cleaning this up via the vCenter Web Client, but could not find the config.vcls.clusters.domain-c(number).enabled configuration parameter at all, even when searching for only cls.
Manually cleaning up vpxd.cfg did the trick though, correcting the configuration to the correct settings:
38 <vcls>
39 <clusters>
40 <domain-c59002>
41 <enabled>true</enabled>
42 </domain-c59002>
43 </clusters>
44 </vcls>
Once this was fixed, I restarted the vCenter appliance and restarted the upgrade process, but since there was a failed upgrade attempt already I had to clean up the failed upgrade state by running the following via SSH on the vCenter appliance:
service-control --stop applmgmt
rm -rf /storage/core/software-update/updates/*
rm -rf /storage/core/software-update/stage/*
rm -rf /storage/db/patching.db
mv /storage/core/software-packages/staged-configuration.json /storage/core
mv /etc/applmgmt/appliance/software_update_state.conf /storage/core
service-control --start applmgmt
Once that step was performed, a new upgrade attempt was performed, this time with a successful result!
Obviously putting wrong configuration settings in either via vCenter Advanced Parameters or directly via editing vpxd.cfg can get you into trouble, and should be avoided if possible in production environments — Especially if you mess them up.
Related Posts
- VMware by Broadcom Promises Free Security Updates for vSphere — Published
- VMware vCenter CVE-2022-31697 — Published
- ESXi5.5 to 6.0 Upgrade From Local HTTP Daemon — Published
- Automatically Name Datastores in vSphere? — Published
- Can you combine vSphere Host Cache and vFlash on a single SSD? — Published