VMware Update Manager: Unsupported Configuration

During an upgrade from vSphere 5.1 to 5.5, I ran into a rather strange issue when trying to utilize VMware Update Manager to perform the ESXi upgrade.

During scanning, VUM reported the ESXi host as “Incompatible”, without offering any other explanation. I spent ages looking through VUM logs, trying to find the culprit, suspecting it was an incompatible VIB. Without finding anything that gave me any indication on what the problem might be, I moved on to looking at the ESXi image I had imported into VUM.

As this was on a Dell PowerEdge R710, I was utilizing the Dell Customized Image of VMware ESXi 5.5 Update 2, which got an updated A02 version last night (27th of August) – I downloaded my image, VMware-VMvisor-Installer-5.5.0.update02-2068190.x86_64-Dell_Customized-A00.iso on the 27th, but before the VMware-VMvisor-Installer-5.5.0.update02-2068190.x86_64-Dell_Customized-A02.iso image was available. Thinking this would resolve the issue, I imported the new image into VUM, and created a new Upgrade Baseline. Sadly, I was still greeted by the non-gracious “Incompatible” warning after performing a new scan.

After some more digging, I found the following entry in the Events pane, for the given host, in vCenter:

Error in ESX configuration file (esx.conf).
error
28.08.2015 11:51:25
Scan entity
[hostname]
[username]

Naturally I go digging into the  /etc/vmware/esx.conf file, and found the following entries:

/nas/[oldserver]/readOnly = "false"
/nas/[oldserver]/enabled = "true"
/nas/[oldserver]/host = "[oldserver.fqdn]"
/nas/[oldserver]/share = "/VeeamBackup_[oldserver]/"

 

These references to [oldserver] pointed to an old Veeam Backup & Replication server that was decommissioned ages ago. Veeam B&R adds these to a host if vPowerNFS has been used to mount a backup share, and the entries had not been removed when removing the old Veeam B&R server. DNS resolution for the old server failed, as it has been completely removed from the infrastructure, thus causing the VUM update to fail.

Manually removing these lines from /etc/vmware/esx.conf fixed the problem, and VUM was able to scan and remediate without issues.

Update:

After writing this, I saw Jim Jones had the same experience, for more details read his post:
Unsupported Configuration when using VUM for a Major Upgrade

Update ESXi Embedded Host Client Fling

VMware ESXi Host Client 2015-08-26 18-42-24The ESXi Embedded Host Client Fling got an upgrade today, and in addition to new features it now works properly on ESXi 5.5. In addition to this, it’s also available as an offline bundle so you can distribute it with Update Manager.

Since I’ve spent most of my day in esxcli, here is a quick post on how to perform the upgrade from a local http repository hosting the .vib file.

Install .vib file

  1. Download the esxui-3015331.vib file and place it somewhere accessible via http.
  2. SSH to your ESXi host, and run the following command (remember to enable maintenance mode if needed).
    esxcli software vib install -v http://[yourip:port/path]/esxui-3015331.vib
  3. Wait for the installation to finish
    Installation Result
    Message: Operation finished successfully.
    Reboot Required: false
    VIBs Installed: VMware_bootbank_esx-ui_0.0.2-0.1.3015331
    VIBs Removed: VMware_bootbank_esx-ui_0.0.2-0.1.2976804
    VIBs Skipped:
  4. Access the updated Embedded Host Client via http://hostip/ui/

 

ESXi5.5 to 6.0 Upgrade From Local HTTP Daemon

I was recently involved with consulting for a Norwegian shipping company who has quite a few remote vSphere installations, most of them with a couple of ESXi hosts, but no vCenter and hence no Update Manager. While looking at methods for managing these installations, in particular how to facilitate patching and upgrading scenarios, I remembered that way back in 2013, I posted Quick and Dirty HTTP-based Deployment which shows how to use the Python to run a simple http daemon, and serve files from it.

Surely something similar can be used to maintain a central repository for vSphere patches? While I don’t recommend using your MacBook as a permanent source for these updates, you really should set up a proper http server in your network and utilize that, it works as a proof of concept.

So here it is, a recipe for using a simple http daemon to host your own ESXi Offline Bundles, and how to upgrade from 5.5 to 6.0 from the command line.
    1. Download the offline bundle you want to use, in my case I used ESXi600-201507001.zip (ESXi 6.0.0b).
    2. Extract the .zip file into it’s own directory, served by the http daemon, and rename it for simplicity.
    3. Connect to your target ESXi host via SSH
    4. Check the current running ESXi version by running esxcli system version get

      The time and date of this login have been sent to the system logs.VMware offers supported, powerful system administration tools. Please
      see www.vmware.com/go/sysadmintools for details.The ESXi Shell can be disabled by an administrative user. See the
      vSphere Security documentation for more information.
      ~ # esxcli system version get
      Product: VMware ESXi
      Version: 5.5.0
      Build: Releasebuild-1623387
      Update: 1
      ~ #
    5. Enable outgoing http requests from the ESXi host by running esxcli network firewall ruleset set -e true -r httpClient
    6. Determine which profile to use by running esxcli software sources profile list -d [http://daemon-ip:port/directory/]
      ~ # esxcli software sources profile list -d http://172.29.100.248:8000/6.0/
      Name Vendor Acceptance Level
      -------------------------------- ------------ ----------------
      ESXi-6.0.0-20150701001s-standard VMware, Inc. PartnerSupported
      ESXi-6.0.0-20150701001s-no-tools VMware, Inc. PartnerSupported
      ESXi-6.0.0-20150704001-no-tools VMware, Inc. PartnerSupported
      ESXi-6.0.0-20150704001-standard VMware, Inc. PartnerSupported
      ~ #

    7. Run vim-cmd hostsvc/maintenance_mode_enter to put ESXi host into maintenance mode.
    8.  Run esxcli software profile update -d [http://daemon-ip:port/directory/] -p [profilename] to fetch and install from http daemon
      # esxcli software profile update -d http://172.29.100.248:8000/6.0/ -p ESXi-6.0.0-20150704001-standard
      Update Result
      Message: The update completed successfully, but the system needs to be rebooted for the changes to be effective.
      Reboot Required: true
      VIBs Installed: [long list of vibs removed for brevity]
      VIBs Skipped:
      ~ #
    9. Disable outgoing http client traffic by running esxcli network firewall ruleset set -e false httpClient
    10. Reboot host by running the reboot command
    11. When host has rebooted, connect to ESXi host again via SSH and run esxcli system version get
      [root@localhost:~] esxcli system version get
      Product: VMware ESXi
      Version: 6.0.0
      Build: Releasebuild-2809209
      Update: 0
      Patch: 11
      [root@localhost:~]

    12. Verify that it runs the correct version, and run vim-cmd hostsvc/maintenance_mode_exit to put the host out of maintenance mode.

And that’s it, the host has now been upgraded from ESXi 5.5 to 6.0, from a local centralized http-based repository without the need to connect to the outside world. All done via the command line, and without a vCenter with Update Manager. Pretty neat.