Upgrading to vCenter 8 Update 1: Invalid Type, expected String, instead got NoneType

Published by Christian Mohn · Read in about 2 min (415 words)

vCenter 8.0 Update 1 was released on April 18th, and I quickly jumped to upgrading my existing home lab vCenter, but ran into an issue that prevented the upgrade from completing, namely the error message “Invalid Type, expected String, instead got NoneType”.

Upgrading to vCenter 8 Update 1: Invalid Type, expected String, instead got NoneType error message

Obviously something was wrong with my current vCenter installation, but finding out what was required to correct it wasn’t straight forward. Thanks to my partner in crime, and notorious human log file parser, Espen Ødegaard, I found the following in /var/log/vmware/applmgmt/PatchRunner.log:

2023-04-19T09:10:39.560Z vpxd:Expand ERROR vmware_b2b.patching.executor.hook_executor Patch hook 'vpxd:Expand' failed.
(...)
Traceback (most recent call last):
xml.etree.ElementTree.ParseError: not well-formed (invalid token): line 40, column 7

Note: The log output has been truncated for legibility.

Clearly something is not conformant in the vpxd.cfg file on my vCenter, on line 40.

Opening /etc/vmware-vpx/vpxd.cfg showed the following:

 38   <vcls>
 39     <clusters>
 40       <78ed4e3c-4f28-4d27-a0d5-01182e957269>
 41         <enabled>true</enabled>
 42       </78ed4e3c-4f28-4d27-a0d5-01182e957269>
 43     </clusters>
 44   </vcls>

This is not correct, it should be referencing <domain-id> and not an UUID on line 40/42. It seems I have messed up the procedure for cleaning up vCLS VMs at some point, putting in the wrong configuration as a vCenter Advanced setting.

Info

Check Putting a Cluster in Retreat Mode for the correct procedure and how to obtain the correct domain-id

I tried cleaning this up via the vCenter Web Client, but could not find the config.vcls.clusters.domain-c(number).enabled configuration parameter at all, even when searching for only cls.

Manually cleaning up vpxd.cfg did the trick though, correcting the configuration to the correct settings:

 38   <vcls>
 39     <clusters>
 40       <domain-c59002>
 41         <enabled>true</enabled>
 42       </domain-c59002>
 43     </clusters>
 44   </vcls>

Once this was fixed, I restarted the vCenter appliance and restarted the upgrade process, but since there was a failed upgrade attempt already I had to clean up the failed upgrade state by running the following via SSH on the vCenter appliance:

service-control --stop applmgmt
rm -rf /storage/core/software-update/updates/*
rm -rf /storage/core/software-update/stage/*
rm -rf /storage/db/patching.db
mv /storage/core/software-packages/staged-configuration.json /storage/core
mv /etc/applmgmt/appliance/software_update_state.conf /storage/core
service-control --start applmgmt

Ref: vCenter Server update fails with error “Installation failed - Install in progress - You have reached maximum number of retries to resume the patching. Please restore the vCenter using the backup.” (87238).

Once that step was performed, a new upgrade attempt was performed, this time with a successful result!

Upgrading to vCenter 8 Update 1: Upgrade Successful

Obviously putting wrong configuration settings in either via vCenter Advanced Parameters or directly via editing vpxd.cfg can get you into trouble, and should be avoided if possible in production environments — Especially if you mess them up.

Post last updated on January 2, 2024: Add author

About

vNinja.net is the digital home of Christian Mohn and Stine Elise Larsen.

The primary focus is on IT architecture and data center technologies like virtualization and related topics, but other content also pops up from time to time.

Sponsors