Virtual Connect FlexFabric and Direct-Attach FC 3Par Caveat

When configuring a new C7000 Blade Enclosure with a couple of FlexFabric 10Gb/24-port modules I ran into a rather annoying issue during setup.

HP Virtual Connect 3.70 introduced support for Direct-Attach setups of HP 3Par StoreServ 7000 storage systems, where you can eliminate the need for dedicated FC switches. For full details, have a look at Implementing HP Virtual Connect Direct-Attach Fibre Channel with HP 3PAR StoreServ Systems.

This is excellent for setups where all your hosts are HP Blades, and you have a Virtual Connect FlexFabric setup. After all, less components means less complexity, right?

The problem I ran into is a bit strange though, and it took some time figuring out what was wrong. The HP 3Par StoreServ 7200 was racked, stacked and configured when the FC SFP+  modules where installed in the FlexFabric module, and I pretty much thought it would be plug and play from there to get the blades to talk FC to the HP 3Par after going through the Virtual Connect setup.

Sadly, that was not the case. It seems there is a bug in the web GUI for VC 3.70 that prevents getting a working setup. I know 3.75 is released, but nothing in the release notes seem to indicate that this has been fixed in that release.

For some reason, the “Fabric Type” dropdown where you should be able to select either “FabricAttach” or “DirectAttach” is greyed out, thus preventing the proper configuration of the SAN Fabric in “DirectAttach” mode. It defaults to “FabricAttach”, and in a Direct-Attach scenario that simply does not work. You will not be able to get a FC link and the SFP+ module will be listed as “unsupported”.

SanFabric-1a

The solution was to create the SAN Fabric manually by using the Virtual Connect CLI interface. T
he following commands created the two fabrics required for redundancy (VC module in Bay 1 and in Bay 2)

1
2
add fabric Fabric-1-3PAR Bay=1 Ports=1 Type=DirectAttach
add fabric Fabric-2-3PAR Bay=2 Ports=1 Type=DirectAttach

As you can see, by using the add fabric command it was possible to define the correct Fabric Type and I could then proceed to add Port 2 from Bay 2 to Fabric-1-3PAR and vice versa to create a fully redundant setup.

SanFabric-2

Why the VC GUI prevented me from setting the correct fabric type is beyond me, but for some reason it simply did not allow me to change this rather important setting, and prevented the setup from working without using the CLI for configuration.

17 Comments

  1. Hi Christian – did you talk to HP Support? I wasn’t aware of this issue (not that I get much exposure to these kinds of things unless I see them in a blog like yours). I want to be sure the right people are aware.

    1. Calvin, I have not yet talked to support about this as we got the situation fixed for the client in question. I would not mind filing a bug report about it though, if only I knew where to do that?

  2. Hi Christian, Calvin asked me to provide you with some clarification. By default, all ports on the face plate of the FlexFabric module are configured for Ethernet. The insertion of the FC SFP+ transceiver does not change that. I agree that it may be counter-intuitive, but you have to configure the ports as FC for them to become enabled and log into the SAN or allow 3PAR target ports to log into FlexFabric module. To make be able to select fabric type as Direct Attach, you have to select uplink ports first. Once you selected those, the drop down will become enabled and allow you to set fabric type to DirectAttach.

    I would highly recommend for you review the Virtual Connect Fibre Channel Networking Cookbook http://bizsupport1.austin.hp.com/bc/docs/support/SupportManual/c01702940/c01702940.pdf which covers how to configure for Direct Attach with 3PAR.

    Also, one of our solution architects published an excellent guide detailing Virtual Connect and 3PAR config requirements.
    https://dl.dropboxusercontent.com/u/8069972/hp-virtual-connect-3par-direct-attach-config-walkthrough.pdf

  3. Thanks for the comment Alex, much appreciated! In this case, even after we set ports X1 and X2 as FC ports, the dropdown did not activate. We tried deleting and re-creating the SAN Fabric several times, but we were never able to configure it as DirectAttach since the dropdown stayed unavailable.

    Either way, we got around it by creating the SAN Fabric by using the command line interface, and the system has since been set in production. Since it´s now in production use, it is not possible for me to retry this again to verify.

    As for the “Virtual Connect Fibre Channel Networking Cookbook”, I have read it and in fact I have a printout of it right here on my desk at the moment.

    The “HP Virtual Connect 3PAR Direct Attach Config Walkthrough” you link to is great too, but it does not mention the need to set the FC ports before changing the FC connection type.

    Perhaps this is all a browser issue, but the fact still remains that we were not able to select the correct SAN Fabric mode in the GUI, and had to use the command line interface to do so.

  4. Just wanted to say thanks to you, Christian, for this post. Last week we installed two HP 3PAR 7400 4-Node systems in our two main datacentres. Each would be connected via Direct-Attach to the FlexFabric Virtual Connects across two c7000 Enclosures (two enclosures and a single 3PAR at each datacentre).

    I read all the HP documentation I could find. THe varying documents all seemed to contradict each other, some would say the 3PAR Host Ports needed to be setup as a Loop port, some would say Point. Even my HP Technical Account Manager said they needed to be Loop ports.

    Well, I could never get the FlexFabric ports to show they were even connected when the 3PAR was set to Loop ports. When set to Point the Virtual Connect would show the FlexFabric ports as “incompatible” and have on orange warning icon on each port in the SAN Fabric in Virtual Connect Manager.

    It was only after removing the fabrics and re-creating using your commands via SSH to the Virtual Connects that they all worked.

    Thank you for this information, I was beginning to tear my hair out as to why something that should be so simple was causing me so much grief. You’re a legend!

    Also, for your reference, I am using Firmware 4.01 on the Virtual Connects.

  5. Christian’s post worked for me, we had to delete the actual SAN Fabric and recreated it. However after creating the new one it did allow me to select the ports and change them to directAttached. Thanks again.

    1. I have never tried that. I have direct attached a MSA/P2000 FC array to some VMware hosts, but never through Virtual Connect. I suspect it would work though, since it’s just a FC connection.

    2. The only supported array I see mentioned is a 3PAR for direct-attach. Best to consult HP as the only documentation talks about 3PAR.

  6. Hi Christian,
    We already have a c7000 flexfabric directAttach to a 4nodes 3PAR. We are configuring a new c7k FF DA. We have defined SAN fabrics and links seem ok (status=ok in vc manager, ports are in ready/free state in 3PAR), but none of the fabric can get the wwn of the 3PAR: “Connected to” still remain at 00:00:00….
    I have compared with the existing c7000 and everything is the same
    Do you know why the vc can not get the 3PAR port WWN?
    Thanks

    1. Are you sure the Host Ports on the 3PAR Controllers are configured correctly? I had this when I first setup my direct-attach, the ports on the 3PAR had to be configured a certain way.

      In our setup the ports must be configured as:

      Connection Mode: Host
      Connection Type: Point

      In earlier versions of the 3PAR Management Console there was a bug where the Connection Type was greyed out after you select the Connection Mode, though we received an update from HP which resolved this.

  7. Hi,
    Thanks for reply. We take care of checking everything twice by 2 different guys (we have also secleted a different range of wwn for the san configuration). We are going, in few days, to check cables…
    Guy

  8. Hi,
    After verifying everything, changing gbic, cables, tyring port X3 or X4, nothing changed. I decided to reboot each VC and … reboot is the first thing to do when configuration seem to be ok but still not work. After reseting the vc, the field “Connected to” is filled with the (good) wwn.
    Guy

  9. I know this isn’t exactly related but wondering if anyone can help me out.

    We have four c7000 enclosures each with 2 flexfabric modules and 2 4Gb Fibre Channel modules. We are going to be replacing the fibre modules with the 8Gb Fibre Channel 20-port modules in the next month.

    Reading the HP Virtual Connect Setup guide for firmware 4.01-4.10 they do mention such a replacement.

    The guide is very brief and says moving the 8Gb 20-port module is simply a straight swap. If you use the 24-port module you have to delete the fabric and recreate, which is a lot of work and downtime.

    Has anyone upgraded from 4Gb modules to 8Gb? Since the modules are redundant in n enclosure I’m hoping I can remove one, ensure login to the fabric, then replace the other one. I’m hoping this will involve no downtime as blades will only lose half their paths to the storage at any time.

    Has anyone had any experience with this? I’ve asked our HP account techs and they are unsure as suggest I have a full enclosure outage and recreate the entire VC Domain. I had to do this when we moved from VC Ethernet Modules to FlexFabric and really don’t want to do it again.

    Thanks,

    Chris

  10. merci pour cet article. j’ai suivi les recommandations, et j’ai fait mes configurations. Fidèle

Leave a Reply