VSAN – The Unspoken Future

This rather tongue-in-cheek title, is a play on Maish recent VSAN – The Unspoken Truth post where he highlights what he thinks is one of the hidden “problems” with the current version of VSAN is it’s inherent non-blade compatibility and current lack of “rack based VSAN ready nodes”.

Of course, this is a reality; If you base your current infrastructure on blade servers, VSAN probably isn’t a good match as it stands today. Chances are that if you are currently running a blade-based datacenter, you have traditional external storage on the back end of that, and that you for quite some time will be running a form factor that VSAN simply isn’t designed for. I don’t disagree with Maish in that conclusion, not a bit.

But what about the next server refresh? One of the things that VSAN is a facilitator for, along with enhancements in the storage industry, is the ability to move to other form factors. Currently Supermicro offers their rather nice looking FatTwin™ Server Solution. If we look at what the SYS-F617R2-R72+ box offers, in the total rack space of 4U (less than most blade chassis), it is clear that the form factor choices will not just be tower or blade, the will also include other new form factors that are currently not in the forefront of peoples minds when designing their data center.

Looking at the Supermicro box again, in a 4U rack footprint, it offers these maximums pr node:

  • 2 x Intel® Xeon® processor E5-2600
  • 6x Hot-swap 2.5″ SAS2/SATA HDD trays

So, in 4U, you can get 16 CPU, 8TB RAM and 48 SAS2/SATA bays.  Stick a couple of those in your rack, with a few 10GbE ports, and then try to do something similar with a blade infrastructure!

Now, of course, VSAN isn’t for everyone, nor is it designed to be. In a way VSAN offers a peak to the future of datacenter design, in the same way that it shows us that Software Designed Data Center (SDDC) is not just about the software, it’s about how we think, manage AND design our back-end infrastructure. It’s not just storage vendors that need to take heed and look at what they are offering, the same also goes for “traditional” server/node vendors.

That’s right, a server is becoming a node and which vendors sticker is in the front, might not matter that much in the future.

The future is already here – it’s just not evenly distributed.
— William Gibson

[Header photo credit: www.LendingMemo.com]

Random VCDX Tips and Quotes?

vNinja-LabsI needed something to spruce up my desktop environment, and it seems that one of the more popular ways to do that is to display random quotes and such on your desktop. Instead of hooking up to an existing quotes database, I simply made my own.

I have collected a few VCDX related tips and quotes, primarily from the archive Duncan Epping put together of VCDX Tips from VCDX 001 John Arrasjid, but also from the “submissions” I got from VCDX? Give Me a Quote! If anyone else wants their quote/tip/hint added to the database, please let me know!

But gathering all of these in a single location has no real purpose, so I managed to code up a really small PHP script that picks a random quote, and return it with attribution. The output is available at http://vninja.net/labs/quotes/quote.php and should return a random line from the db on each query.

I use this in combination with GeekTool to display a random, changing, quote right on my desktop:

VCDX Quotes on the Desktop


Inside GeekTool, I run the following command to generate the output:

curl http://vninja.net/labs/quotes/quote.php

This  is set to refresh every 120 seconds, as long as I have internet connectivity.

Feel free to use the script to do something similar, or if you have a better idea for it’s usage let me know!

Seen “In the Wild” Gallery:

Fixing “Could not create vFlash cache” error when powering on a VM

During some way overdue housekeeping in my HP Microserver-based “Homelab” today, I ran into a rather annoying issue that prevented me from starting one of my more important VMs; namely my home Domain Controller.

In short, I removed an old SSD drive that I’ve used for vFlash Read Cache (vFRC) testing and installed a new 1TB drive instead. Since I have a rather beefy work lab now, I need space more than speed at home, so this seemed like a good idea at the time.

A good idea that is, until I tried starting my DC VM, which also is my DNS and DHCP server, and got greeted with this little gem:


The “Could not create vFlash cache: msg.vflashcache.error.VFC_FAILURE” error is understandable, the SSD drive was removed, but I honestly did not think that even if a VM was configured to use it, that it’s absence would prevent a power-on operation on that VM. I would have expected it to throw a warning about the cache location missing, and warn me that acceleration was not happening, not a flat out “cannot power on“.

Normally the fix for this would be quick and easy, edit the VM and remove the vFRC configuration, but since my host has a whopping total of 8GB of memory I don’t have vCenter running at all times.  Editing the VM settings through the vCenter Legacy Client (C#) does not work, since vFRC requires Virtual Hardware Version 10, which it cannot edit.

Once I got the vCenter Server Appliance (vCSA) fired up, I realised that I have somehow forgotten the admin AND root passwords, and was completely unable to log-on. How that has happened is beyond me, but for the life of me I was not able to log on.

Next step was to try and edit the VM from VMware Workstation 10 installed on one of the Windows boxes in my network. Sadly Workstation has no concept of vFRC, and it is not possible to edit that particular VM setting, even if you connect it to the vSphere host. I later on also realized that VMware Workstation 10 is also unable to host connect  USB peripherals, like printers, to a VM, but that’s beside the point right now.

So, either trying to hack the VM Hardware Version to a value that the vSphere Client can handle, of  deploying a new vCSA instance, I was left with editing the VMs vmx file directly.

Thankfully this was an easy way to fix the problem, and get the VM powered on. For each vmdk file that is configured to use vFRC, there is a corresponding entry in the vmx file, that controls vFRC.

In order to turn off vFRC acceleration for a given disk, download the vmx file, and change the value for <device>.vFlash.enabled from

sched.scsi0:2.vFlash.enabled = "TRUE"


sched.scsi0:2.vFlash.enabled = "FALSE"

Re-upload the vmx file, and try to power on. In my case, this fixed the problem of powering on the VM.

This problem again highlights one of the problems with the dependancy on the new vSphere Web Client. In a real production environment, the vCenter would be up and running at all times, and editing the VM would have been a small task, but what if you had used vFRC to speed up vCSA itself, and you had a failed SSD drive?

Of course, in this case the fault is mostly mine. I removed a “production SSD”, without first removing the vFRC configuration. I did not have a working vCSA with Web Client up and running when this was done, and I had forgotten my passwords. A pretty specific error generating procedure if there ever was one.

It’s an easy fix to edit the vmx file, but it does at times feel a little bit like you are now able to cut of the branch you are sitting on with the new vSphere Web Client. In simpler days, you could pretty much fix anything by connecting the vSphere Client to the host and fix any errors there, but now the dependancy on the vCenter Server is stronger than ever.

Before upgrading your VMs to Hardware Version 10, make sure you understand the implications of going all-in with dependancies on the Web Client and vCenter. It might just come back and bite you if you haven’t thought through your design.