CategoryvSphere

Using OVFtool via Powercli with a session ticket – lessons learned

Powercli is a really great tool for automation. Nevertheless from time to time we need other tools as well to fullfil our needs. If you want to automize the distribution of templates in your environment the OVFtool is a really nice way to achieve this. Since I wanted to deploy the template to multiple Clusters in an environment, the following would do the trick.

Having 2 arrays which stores all the vCenter and all the Clusters in the environment the following would do the trick.

function DistributeTemplates 
(
	[Parameter(Mandatory=$True)]
	[String]$templateLocation
)
 
$ovftool = "C:\Program Files\VMware\OVFtool\ovftool.exe"
 
foreach ($vCenter in $vCenterlist) {
    Connect-VIServer $vCenter
    foreach ($cluster in $clusterList) {
 
        $arglist = ' --name=TemplateName --network=NetworkName -ds=DatastoreName $($templateLocation) vi://vCenterUser:Password@$($vCenter)/DCNAME/host/$($cluster)'  		 		
        $process = Start-Process $ovftool -Argumentlist ($arglist) -wait
    }
}

Eventhough it was working I was not happy about how the authentication mechanism was used (Password in cleartext…nooooo way).

Luckily I found a post at geekafterfive.com who explained how we can use a ticketing automatism in OVFTool.

$Session = Get-View -Id Sessionmanager
$ticket = $session.AcquireCloneTicket()
Unfortunatley struggeld with two things:

1. Make sure you are only connected to one vCenter, otherwise

 $ticket = $session.AcquireCloneTicket()

will throw an error Method invocation failed because [System.Object[]] doesn’t contain a method named ‘AcquireCloneTicket’.”

powercli_error2. I could only upload the templates to the first cluster. The second one was always failing. It seems that my ticket was not valid anymore. Luckily a closer look to the vSphere SDK Programming guide told me that “A client application executing on behalf of a remoteuser can invoke the AcquireCloneTicket operation of SessionManager to obtain a onetime user name and password for logging on without entering a subsequent password” …. ahhh…one time password…I thought the ticket will be valid for multiple operations once I’m connected to a vCenter. But since my thoughts don’t count on this topic (*yeahyeah…what a rough world) I needed to create a new ticket before every OVFTool operation.

So the following script was completly satisfying my (template-automation) needs.

function DistributeTemplates 
(
	[Parameter(Mandatory=$True)]
	[String]$templateLocation
)
 
$ovftool = "C:\Program Files\VMware\OVFtool\ovftool.exe"
 
foreach ($vCenter in $vCenterlist) {
    Connect-VIServer $vCenter
    foreach ($cluster in $clusterList) {
        $Session = Get-View -Id Sessionmanager
        $Ticket = $Session.AcquireCloneTicket()
        $arglist = ' --I:targetSessionTicket=$($Ticket) --name=TemplateName --network=NetworkName -ds=DatastoreName $($templateLocation) vi://$($vCenter)/DCNAME/host/$($cluster)'  		 		
        $process = Start-Process $ovftool -Argumentlist ($arglist) -wait
    }
}

Now a ticket of the vCenter authentication is generated after the usage and I haven’t had to deal with storing any credentials while deploying a template to the whole wide world :)…yeha…

Import vCenter Operations (vCOps) as an OVF into a vCloud Director environment

Having a standard in the virtualization world like OVF or OVA is a real cool thing. Nevertheless from time to time I stumble about some issues with it. Just happened while trying to import vCenter Operation to a vCloud environment at a customer site.

Even though you can import OVF (unfortuneatly no OVA files -> extract the OVA first, e.g. with the ovftool or tar) it hasn’t worked in the vCloud environment, since the vCOPS vAPP consists of two virtual machines and wants to have additional information (EULA, network, etc.) during the deployment process.

So my next idea was to deploy the OVF in the vCenter just to import it afterwards into the vCloud director.

0005

After the importing process i wasn’t able to start the VM either

001

Unable to start vAPP ….., Invalid vApp properties: Unknown property vami.ip0.VM_2 referenced in property ip.

0021

To solve this problem, I moved the two virtual machines (analytics and UI) out of the vApp in the vCenter and deactivated their vApp properties in the virtual machine settings.

003

If any error message occur during this configuration part and prompt wants you to refresh the browser, ignore this by clicking NO (worked only in Chrome for me)

004

Now you can navigate to the vCloud Director, import both virtual machines to your vApp.

0005

and voila. The vCenter Operations VMs can boot in your vCloud environment.

006

Keep in mind that you eventually need to change the IP addresses of your vCenter Operations components. Just follow the instructions on virtualGhetto

Linux and the Web Client in vSphere 5.5

If you use Linux and read the vSphere 5.5 release notes you might have noticed this:

vSphere Web Client. Because Linux platforms are no longer supported by Adobe Flash, vSphere Web Client is not supported on the Linux OS. Third party browsers that add support for Adobe Flash on the Linux desktop OS might continue to function.

Puhh, pretty bad news! I found out VMware’s vSphere Web Client in 5.5 requires Adobe Flash 11.5 whereas Adobe provides flash for Linux only up to version 11.2. As a result, starting Web Client with Firefox on Linux you see this:

Screenshot-from-2013-10-12-215820

Here is the good news: It works with Google Chrome:

Screenshot-from-2013-10-12-215828

For your information I copy my software package versions so you know what exactly I am working with:

vxltsupport@desktop:~$ dpkg -l | grep -E "flash|firefox|google-chrome"
ii  firefox                              26.0+build2-0ubuntu0.13.04.2           i386         Safe and easy web browser from Mozilla
ii  firefox-globalmenu                   26.0+build2-0ubuntu0.13.04.2           i386         Safe and easy web browser from Mozilla (transitional package)
ii  firefox-gnome-support                22.0+build2-0ubuntu0.13.04.2           i386         Safe and easy web browser from Mozilla - GNOME support
ii  firefox-locale-de                    26.0+build2-0ubuntu0.13.04.2           i386         German language pack for Firefox
ii  firefox-locale-en                    26.0+build2-0ubuntu0.13.04.2           i386         English language pack for Firefox
ii  flashplugin-installer                11.2.202.332ubuntu0.13.04.1            i386         Adobe Flash Player plugin installer
ii  google-chrome-stable                 31.0.1650.63-1                         i386         The web browser from Google
vxltsupport@desktop:~$

Should you be aware of any other browser or flash plugins that work or do not work, please drop me a mail to mathias.ewald@vxpertise.net and I will add the information here!

vOpenData

Some time ago, a service called vOpenData started providing information about vSphere infrastructures to the public followed by a blog post by Duncan Epping about it. Just a couple ago, I stumbled across it again thinking “What a great idea!!!” The dashboard is amazing! For example, ever wondered what is the average VMDK disk size or the most commonly used OS for a VM? Take a look!

Web Client becoming more and more important

I have to admit.: Although I know I would have to get used to VMware Web Client, I’m still doing a lot of tasks in the good old .NET client. But I just recognized that I couldn’t edit the virtual machines hardware settings of a hardware version 10 VM:

hw10

So the vSphere Web Client Server is getting more and more a very critical component in your environment, since one day we probably won’t be able to use .NET Client even for basic administration tasks.

Thin Provisioned LUNs with TPSTUN and TPST

Thin provisioned LUNs have been a problem for a long time with vSphere. Allowing overcommitment of storage capacity on the storage array, you have to keep in mind that the LUN will grow creating a new VM but not shrink deleting one. As a result, the LUN will keep growing as VMs come and go, plus: there is a good chance that at some point your array might run out of capacity as storage admins may overcommit physical storage capacity. Take a look at the example blow:

tplun

The RAID group has a capacity of 25GB. With two 20GB LUNs, ESXis will see a total capacity of 40GB (overcommitted). Right now, ESXis use a total of 20GB and think they have another 20GB of free capacity. Looking at the RAID group, it cannot bare more than another 5GB, though. One of the VMDK files was already deleted in VMFS but as you can see this did not free capacity on the RAID group level.

What is going to happen when the RAID group runs out of capacity, while ESXi hosts think they would still have some free space? Well, lets say a new SCSI command arrives from a VM which would require the allocation of another physical disk block. The VMkernel would try to allocate space and make that data persistent on the array but can’t as there is no more space. But the VM cannot continue before the data was written rendering the VM frozen. What about VMs with preallocated / thick VMDK files? No difference. Their blocks are preallocated in VMFS but not in the thin provisioned LUN. Writing to a new block would have the array allocate additional space which is just not here.

What is going to happen to a write command overwriting existing data? Well, obviously no additional space needs to be allocated. But still I would assume the VM will freeze to, as the VMkernel is just not able to get other commands (see section above) out of the SCSI queue. So other VMs will probably block that VM, too. But I am not entirely sure about this and unfortunately I cannot test this in my labs as I have not storage that allows over commitment and disabling VAAI.

Bottom line:

When you run out of capacity on the RAID group, VMs will start to freeze.

Obviously, you will want to monitor your storage array for a out of space condition like this and be able to react preventing VM from freezing. To help relaxing the situation, VAAI brings three primitives of interest in the context: Unmap, Thin Provisioning Stun (TPSTUN) and Thin Provisioning Soft Threshold (TPST). While Unmap allows the reclamation of unused but mapped space inside VMFS (the deleted VMDK file in the figure above), TPSTUN and TPST allow better handling of the problem itself.

Note:
Interested in a complete list of VAAI primitives? Check http://blog.fosketts.net/2011/11/10/complete-list-vmware-vaai-primitives/!

Note:
Storage vendors seem to have trouble implementing UNMAP efficiently. As a result, VMware decided to remove automatic UNMAP. Right now, UNMAP is only possible to trigger manually using the vmkfstools -y command.

TPSTUN

TPSTUN is a VAAI feature officially introduced with vSphere 5. Unlike other VAAI primitives, TPSTUN is not block related but an error core sent from the storage array via a SCSI command. The command is sent when the storage array reaches 100% capacity usage (no more physical blocks can be allocated) entering an out-of-space condition. ESXi will receive the command and pause the affected VMs allowing other VMs to continue to run.

copy-of-Untitled-1

TPST

TPSTUN is nice as it allows unaffected VMs to continue to run pausing others. But wouldn’t it be nice for a vSphere admin to know about this before it actually happens? Absolutely! vSphere 5.1 comes with support for TPST which actively notifies ESXi about an upcoming out-of-space condition and allows it to query the thin provisioning state in the storage array, too. The threshold at which the storage array triggers this notification can vary from array to array but 80% seems to something commonly accepted as a good value.

Demonstration

As I never ran into the situation on a production environment, yet, I simulated this in my labs just to see what is looks like in vSphere Client, vSphere Web Client and ESXi log files.

Storage Array RAID Group and LUNs

In my virtual labs, I have no way to use FibreChannel with my virtual ESXi hosts – obviously. Luckily, I found QUADstor. Their software has always been available for free and was just open sourced a couple of weeks ago.

  • QUADstor storage array with 100GB total capacity
  • 2 LUNs (VDisks), 100 GB each

Screenshot-08292013-075126-PM

Screenshot-08302013-082826-AM

So we definitely overcommit storage here providing 200% of available capacity to vSphere. Looking at the VDisk (LUN) configuration, we can see that the TP Soft Threshold is set to 80%:

Screenshot-08302013-083440-AM

The QUADstor documentation describes this setting like this:

A threshold is an indication to a client that a out of space condition will soon be reached. For example for a 100 GB VDisk if the threshold is set at 80 % if the available space in the corresponding storage pool cannot store at least 20 GB more of VDisk data a notification is sent to the client on the next write command

Testing TPST

Right now, QUADstor uses 26.33GB of physical disk space. According to the description, we should see TPST when 80GB physical disk usages is reached. Now let us bring the used capacity slightly above the 80% threshold and see what happens. I am going to do this simply by cloning VMs. For this purpose, I disabled QUADstor’s deduplication feature.

~ # cat /var/log/vmkernel.log | grep "0x6 0x38 0x7. Act:NONE"
...
2013-08-30T06:25:22.357Z cpu0:705035)NMP: nmp_ThrottleLogForDevice:2319: Cmd 0x2a (0x4124007f4800, 2050) to dev "naa.6ed5603489a66917daa052a5de9197ad" on path "vmhba33:C1:T0:L0" Failed: H:0x0 D:0x2 P:0x0 Valid sense data: <strong>0x6 0x38 0x7. Act:NONE</strong></code>
...
~ #

I learned that 0×6 (UNIT ATTENTION) with 0×38 0×7 indicates a logical unit soft threshold was reached (TPST).

Testing TPSTUN

Completely filling up physical capacity on the array, this is what I see in ESXi’s vmkernel.log:

2013-08-30T08:59:39.059Z cpu0:2050)NMP: nmp_ThrottleLogForDevice:2319: Cmd 0x2a (0x4124007fec80, 2905) to dev "naa.6ed5603489a66917daa052a5de9197ad" on path "vmhba33:C1:T0:L0" Failed: H:0x0 D:0x2 P:0x0 Valid sense data: 0x7 0x27 0x7. Act:NONE
2013-08-31T08:59:39.060Z cpu0:2050)ScsiDeviceIO: 2316: Cmd(0x4124007fec80) 0x2a, CmdSN 0x2 from world 2905 to dev "naa.6ed5603489a66917daa052a5de9197ad" failed H:0x0 D:0x2 P:0x8 Possible sense data: 0x7 0x27 0x7.

In this line 0×7 (DATA PROTECT) with 0×27,0×7 indicate SPACE ALLOCATION WRITE FAILURE (TPSTUN).

Conclusion

Being a big fan of VAAI, I have to admit I was a bit disappointed: The only reaction to TPSTUN and TPST in vSphere seems to be log messages like the ones above. There are a couple of cool thing you could to with this, though:

Indicators:

  • vSphere Client / vSphere Web Client could show an indicator about occurrences of TPST on a LUN …
  • … or a stun of a VM in response to TPSTUN.

Proactive reactions:

  • Receiving TPST, SDRS could svMotion to get blow the threshold.
  • In response to TPSTUN, we could automatically migrate to a different LUN which is not located on the same RAID group (SDRS already uses a correlation detector to figure out things like this!).

Let’s see what the next release will bring icon wink Thin Provisioned LUNs with TPSTUN and TPST

vSphere Testing vSAN in a Nested vSphere Environment

The new vSAN feature coming up soon is REALLY exciting me! That is such a cool feature 8-)! Of course, I was interested in testing it in my lab right away when I could get access to it and faced a problem: vSAN requires the availability of a locally attached SSD drive. Well, I guess most of us build virtual (nested) labs and right now vSphere cannot be configured to present a virtual SSD to a VM – damn! Fortunately, there is a way to fake one icon wink Testing vSAN in a Nested vSphere Environment I remembered that is was possible to mark a disk as SSD in case ESXi didn’t recognize it. So I thought maybe we can use this to fake an SSD for virtual ESXi hosts and guest what – yes we can!

Recipe:

  1. Get a shell on your ESXi host (ESXi Shell or SSH)
  2. Find the canonical name of the locally attached regular disk
  3. Create a claim rule that tags the drive as SSD.
  4. Reclaim the device.
  5. Verify.

Find the canonical name of the locally attached regular disk:

~ # esxcli storage core device list | grep -E "(^\s+Display Name)|(^\s+Size)|SSD"
Display Name: QUADSTOR iSCSI Disk (naa.6ed5603489a66917daa052a5de9197ad)
Size: 204800
Is SSD: false
Display Name: Local VMware Disk (mpx.vmhba1:C0:T1:L0)
Size: 8192
Is SSD: false
Display Name: Local VMware Disk (mpx.vmhba1:C0:T0:L0)
Size: 16384
Is SSD: false
Display Name: Local NECVMWar CD-ROM (mpx.vmhba0:C0:T0:L0)
Size: 300
Is SSD: false
Display Name: QUADSTOR iSCSI Disk (naa.6edd1360f61f663586050a01b6571f84)
Size: 204800
Is SSD: false

Create a claim rule that tags the drive as SSD:

~ # esxcli storage nmp satp rule add --satp VMW_SATP_LOCAL --device mpx.vmhba1:C0:T1:L0 --option=enable_ssd

Reclaim the device:

~ # esxcli storage core claiming reclaim -d mpx.vmhba1:C0:T1:L0

Verify:

~ # esxcli storage core device list | grep -E "(^\s+Display Name)|(^\s+Size)|SSD"
Display Name: QUADSTOR iSCSI Disk (naa.6ed5603489a66917daa052a5de9197ad)
Size: 204800
Is SSD: false
Display Name: Local VMware Disk (mpx.vmhba1:C0:T1:L0)
Size: 8192
Is SSD: true
Display Name: Local VMware Disk (mpx.vmhba1:C0:T0:L0)
Size: 16384
Is SSD: false
Display Name: Local NECVMWar CD-ROM (mpx.vmhba0:C0:T0:L0)
Size: 300
Is SSD: false
Display Name: QUADSTOR iSCSI Disk (naa.6edd1360f61f663586050a01b6571f84)
Size: 204800
Is SSD: false
esxcli storage core claiming reclaim -d mpx.vmhba1:C0:T2:L0 – See more at: http://www.virtuallyghetto.com/2011/07/how-to-trick-esxi-5-in-seeing-ssd.html#sthash.O0jydZb0.dpuf

Screenshot-08302013-124859-PM

Tadaaa! Like this we can test vSAN and other cool stuff already around: Swap to SSD and Host Caching in VMware Horizon View VDI environments! Awesome!

For simulating the vSAN feature in a nested environment just use the detailed guide from David Hill at virtual-blog.

Once you followed the mentioned steps, you will se a shared vSAN datastore and the corresponding Storage Provider entries.

vSAN_datastore

Now you are able to create a VM Storage Policy out of the vendor specific vSAN capabilities.

vSAN_Storage_Profile

Just attach the newly created profiles to your Virtual Machine hard disk and voila…. That’s it icon wink Testing vSAN in a Nested vSphere Environment Have fun of testing this really cool new feature in your nested environment.

Register for the public beta right here

What’s New in vSphere 5.5

With VMworld 2013 in progress and the new version of vSphere becoming GA soon, I found it was time to publish a quick overview of the features that come with the new vSphere 5.5 release. I will try to summarize each feature with a couple of words only and go into details with a dedicated article when necessary. Also take a look at What’s New in vSphere 5.5 Platform!

ESXi Features

Hot-Pluggable PCI SSD Devices: 
ESXi now allows hot-plugging of SSD devices in addition to already supported SATA and SAS disks.

Support for Reliable Memory Technology:
Currently, the VMkernel runs in memory in a non-redundant fashion. Memory errors can potentially crash the hypervisor and VMs with it. RMT (Reliable Memory Technology) is a CPU feature that reports a certain memory reagion to be reliable. The VMkernel uses this information to place critical components such as the init thread, hostd and watchdog in this reagion guarding them against failure.

Enhancements to CPU C-States:
While earlier releases supported CPU P-states (frequency scaling), ESXi 5.5 now supports turning off entire CPU components (C-states) in order to save power.

VM Features

Virtual Machine Compatibility:
Formerly just know as the “Virtual Machine Version”, VMware changed the name of this to Virtual Machine Compatiblity. VMs on ESXi 5.5 bring the new VM Version 10. This allows for

  • support for LSI SAS for Solaris 11
  • new CPU architectures
  • new AHCI: supports up to 30 disks and CD/DVD drive

Therefore, the new maximum disk devices limitation per VM went up from 60 to 120.

Expanded vGPU Support:
Support for virtual GPU was limited to NVIDIA-based GPUs up until vSphere 5.1. Version 5.5 brings support for multiple Intel and AMD-based GPU chips, while still allowing vMotion migrations. Using vSphere Web Client or Horizon View, this feature can be enabled for Windows 7, Windows 8, Fedora 17, Ubuntu 12, RHEL 7 and later versions. Three different modes (automatic, hardware, software) govern vMotion migration compatibility to hosts with different or even without a supported GPU.

Graphic Acceleration for Linux Guests:
A new guest driver for virtual video cards is provided and was fully contributed to the open source community allowing every distribution to integrate the driver into their kernel. With the new driver every Linux VM can now support OpenGL 2.1, DRM kernel mode setting, Xrandr, XRender and Xv.

vCenter Server Features

vCenter SSO:
SSO can now connect to its Microsoft SQL database without requiring the customary user IDs and passwords. After joining the hosting system to the AD domain, vCenter SSO interacts with the database using the identity of the machine itself.

vSphere Web Client:
VMware continues to integrate all new features into the Web Client only enhancing its features and making it more usable for daily administration. This includes full client supports for Mac OS X, full support for Firefox and Chrome web browsers and usability features like drag and drop, filters and recent items.

vCenter Server Appliance:
The Linux-based vCenter appliance now uses vPostgres as an embedded database supporting up to 500 ESXi hosts or 5000 Virtual Machines. This new scalability maximum enabled vCenter Server Appliance to be used for most environments out there! Great job, VMware!

vSphere App HA:
Support for Application monitoring in an HA cluster was around for a while but applications of third-party monitoring software had to implement the vSphere Guest SDK in order to monitor application health. vSphere App HA renders this concept obsolete. Two appliances must be deployed per vCenter:

  • vSphere App HA appliance: stores and manages policies
  • vFabric Hyperic appliance: uses agents installed in guest OSs to monitor health and enforce policies stored in vSphere App HA

VM (Anti-)Affinity Rules and HA/DRS Clusters:
vSphere 5.5 HA now takes DRS VM rules into account during a failover process. This removes the requirement for DRS to rebalance workloads once up and running allowing better compliance with multitenancy or compliance restrictions and higher availability for latency sensitive workloads that suffer from vMotion migrations.

vSphere Big Data Extensions:
BDE comes as a Web Client plugin allowing simplified provisioning of multinode Hadoop clusters and ecosystem components such as Apache Pig, Apache Hive and Apache HBase.

vSphere Storage Enhancements

62TB VMDK:
The former VMDK size limitation of 2TB – 512 Bytes was now broken. With 5.5 VMDKs and virtual compatibility mode RDMs support up to 62TB while still supporting snapshots.

MSCS:
Earlier, MSCS was supported with FC shared storage only. This support was extended to iSCSI, FCoE and Round Robin path policy for Windows Server 2012.

16GB E2E Support:
Before 5.5, VMware did not support end-to-end 16Gbit FC connectivity from ESXi hosts to storage arrays. This changed in 5.5.

PDL AutoRemove:
Once a LUN enters the “permanent device loss” state, ESXi now removes the device from the host freeing one of the available 255 spaces for LUNs instead of letting it sit there occupying the slot.

vSphere Replication Interoperability:
Primary-site VMs replicated with VSR can now be migrated with Storage vMotion without any penality for the replication process. Earlier, .psf files where not relocated but deleted during a Storage vMotion migration. As a result, the VM had to go through a full sync including tons of checksum tests on both ends. As of 5.5, migration of the replica disk is still not possible.

vSphere Replication Multi-Point-in-Time (MPIT) Snapshot Retention:
vSphere Replication 5.5 now supports multiple restore points for a single VM. While ealier releases retained the state after the last replication only, a retention policiy now dictates the amount of recovery points to be stored.

VAAI UNMAP Improvements:
The esxcli storage vmfs unmap command, now supports the ability to specify the reclaim size in blocks rather than a percentage value. Dead space reclamation can now the reclaimed in increments rather than all at once.

VMFS Heap Improvements:
Former releases of VMFS hat problems with open files of more than 30TB in size from a single ESXi host. vSphere 5.0 p5 and vSphere 5.1 U1 introduced larger VMFS heap spaces to address the issue at the cost of higher memory usage mounting a VMFS. vSphere 5.5 improved the heap eviction process rendering a larger heap size obsolete. With a maximum memory consumption of 256MB, an ESXi host can now access all the address space of a 64TB VMFS volume.

vSphere Flash Read Cache:
ESXi 5.5 hosts now discover locally attached flash devices and pool them into a “vSphere Flash Resource”. A flash resource can be consumed in two difference ways:

  1. vSphere Flash Swap Cache: replaces the previously introduced Swap to SSD feature
  2. vSphere Flash Read Cache: significantly improves read intensive VM workloads

Networking Features

LACP:
The LACP protocol supported between a physical and vSphere Distributed Switch now supports

  • 22 new hashing algorithms
  • multiple link aggregation groups (LAG): 64 LAGs per host and 64 LAGs per vDS
  • workflow to configure LACP across a large number of hosts

Traffic Filtering:
Network packets can now be filtered based on various qualifiers: MAC (src and dst), system traffic qualifiers (vMotion, iSCSi, …) and IP qualifiers (protocol type, src, dst, port number). Matching packets can be either filtered or tagged. This way VMware implements Access Control Lists (ACL) for vDS.

Quality of Service Tagging:
In addition to the previously available IEEE 802.1q Class of Service tagging of Ethernet frames (Layer 2) as part of Network I/O Control (NIOC), vDS now supports Differentiated Service Code Point (DSCP). This standard inserts QoS tags into the IP header allowing for QoS information to be available across the entire networking environment including routers.

SR-IOV:
Single Root I/O Virtualization was improved by simplifying the configuration process and allowing propagation of Port Group settings to virtual functions of SRIOV devices.

Enhanced Host-Level Packet Capture:
This fancy name represents the functionality of a simple command-line based traffic sniffer like tcpdump available in ESXi hosts. It allows detailed sniffing of vSS and vDS traffic.

40GB NIC Support:
Support for 40GB NICs was introduced supporting Mellanox ConnextX-3 VPI adapters in ethernet mode.

vDS Rollback – Changing the Management VLAN

Introduced in vSphere 5.1, VMware’s vSphere Distributed Switch (vDS) brings a feature called “Rollback”. In KB 2032908 VMware provides a good overview of the features, to please read this first before you continue.

The Problem

While the advantages of this outweigh the disadvantages by far, there are situations when vDS Rollback can drive you nuts. Just happend today: We had ESXi server in VLAN 10 while vCenter, database, etc reside in VLAN 20. For VLAN 20 we had a vDS Portgroup with a VLAN configuration of “none” as the physical Switch (pSwitch) defined VLAN 20 to be the “native VLAN”

The native VLAN, by definition, is the network carried over a VLAN trunk connection without a 802.1q tag.

So this was the initial situation:

rollback

Now, we wanted the Management VLAN to be tagged to the pSwitch. In earlier releases the process looked like this:

  1. Change vDS configuration to VLAN: 20 and lose connection to vCenter (planned)
  2. Go to pSwitch and add VLAN 20 to the tagged VLANs on the trunk
  3. Connection will be restablished.

The problem with rollback is, that when you lose the connection to vCenter, ESXi are going to lose connection, too, triggering the rollback feature. As a result, your configuration which lead to losing the connection will be undone.

Solution 1: Disable Rollback

One solution is to disable rollback, take the “old” way to changing the VLAN configuration, then re-enable rollback:

Disable rollback:

  1. On your vCenter server nagivate to %programdata%\VMware\VMware VirtualCenter
  2. Open vpxd.cfg for edit.
  3. Change  <rollback>false</rollback> to  <rollback>true</rollback>
  4. Restart vCenter services from services.msc

Now follow the instructions as mentioned in the first section of this article. Finally, follow the steps above to enable rollback.

Solution 2: Native and Tagged VLAN 20

Another solution involves some pSwitch configuration: We have to configure the pSwitch to send untagged traffic to VLAN 20 (native VLAN) and accept traffic tagged with 20, too! Like this we can change the VLAN configuration of the Portgroup from “none” to VLAN 20 without a network interruption. Like this rollback will not kick.

rollback2

Finally, the pSwitch can get a different native VLAN configuration without further interruption of vSphere management traffic.

Hope that helped!

PowerCLI – Waiting for Guest Customization to Finish

Only a couple of days ago, a customer asked for a script to deploy hundreds of VMs with a couple of tasks to be done after the VM was available on the network. Like move into a different OU, or register with XenDesktop, etc. The challenge about this was to have the PowerCLI script wait for the VM to be actually done with the Sysprep process and be available on the network before the remaining tasks are executed.

I found an article from Alan Renouf on blogs.vmware.com about this topic but was unable to do it the way he suggested. I didn’t really figure out why it seemed to work for him but not in my case, but for example the way he tested for the Event type and a couple of other things did not work out and created syntax errors etc. While his code showed me how to do it in general, I decided to write my own code. Own code is always better for personal understanding, too icon wink PowerCLI – Waiting for Guest Customization to Finish AND my code is way simpler as it does not simultaneously monitor multiple VMs but only one at a time.

function StartVMAndWaitForSysprep
(
	[Parameter(Mandatory=$True)]
	[Vmware.VIMAutomation.ViCore.Impl.V1.Inventory.VirtualMachineImpl]$vmRef
)
{
	$vm = Start-VM $vmRef -Confirm:$False -ErrorAction:Stop
 
	# wait until VM has started
	Write-Host "Waiting for VM to start ..."
	while ($True)
	{
		$vmEvents = Get-VIEvent -Entity $vm
 
		$startedEvent = $vmEvents | Where { $_.GetType().Name -eq "VMStartingEvent" }
 
		if ($startedEvent) 
		{
			break
		}
		else
		{
			Start-Sleep -Seconds 2	
		}	
	}
 
	# wait until customization process has started	
	Write-Host "Waiting for Customization to start ..."
	while($True)
	{
		$vmEvents = Get-VIEvent -Entity $vm 
		$startedEvent = $vmEvents | Where { $_.GetType().Name -eq "CustomizationStartedEvent" }
 
		if ($startedEvent)
		{
			break	
		}
		else 	
		{
			Start-Sleep -Seconds 2
		}
	}
 
	# wait until customization process has completed or failed
	Write-Host "Waiting for customization ..."
	while ($True)
	{
		$vmEvents = Get-VIEvent -Entity $vm
		$succeedEvent = $vmEvents | Where { $_.GetType().Name -eq "CustomizationSucceeded" }
		$failEvent = $vmEvents | Where { $_.GetType().Name -eq "CustomizationFailed" }
 
		if ($failEvent)
		{
			Write-Host "Customization failed!"
			return $False
		}
 
		if($succeedEvent)
		{
			Write-Host "Customization succeeded!"
			return $True
		}
 
		Start-Sleep -Seconds 2			
	}
}

Finally, but not not worth its own posting, we have a function to wait for the guest to have an IP address:

function WaitForToolsToReportIP
(
	[Parameter(Mandatory=$True)]
	[Vmware.VIMAutomation.ViCore.Impl.V1.Inventory.VirtualMachineImpl]$vmRef
)
{
	$ping = New-Object System.Net.NetworkInformation.Ping
 
	while ($True) 
	{	
		$vm = Get-VM $vmRef
 
		$guestInfo = $vm.guest
 
		if($guestInfo.IPAddress -and $guestInfo.IPAddress[0])
		{
			return $True
		}
		else
		{
			Start-Sleep -Seconds 2
		}
	}	
}

As you can see, I check whether VMware Tools report an IP address. The advantage of this is that I don’t have to rely on the OS to respond to pings.

Hope that helps!

© 2019 v(e)Xpertise

Theme by Anders NorénUp ↑