Automatic virtual hardware change after shutdown – vCenter alarms and #PowerCLI

Teaching an Optimize and Scale class this week I was asked if there is chance to automatically change the virtual hardware once a VM has been shut down or powered off.

Scenario – Customer wants to change the CPU / RAM of their XenDesktops with minimal impact on the end-user. If a VM is powered off vCenter should change the hardware and wait until another components powers the VM on again.

At first I was not sure how this can be done. Using PowerCLI for changing the virtual hardware? – easy… but how can we call this script once the VM is powered off? After a minute I remebered that we can use vCenter alarms to trigger more than just snmp/email actions if a specific Event/Condition occurs. We can also run cmds and therefore call a PowerCLI script.


Using AD-based authentication for the PowerCLI script makes it easier regarding the Authentication mechanism. Therefore the vCenter server must run with Active Directory Service Account that has the proper permissions on the vCenter level



Chapter 1:  Easy going – Hardcode the CPU count in the script

Create the PowerCLI script

Now we create the modify-cpu.ps1 script that will be stored in the C:\scripts\

with the following content. (Note: the CPU count must be hardcoded in the script by changing the $numCPU parameter. Be aware of that this script changes the count of Cores and stays with 1 virtual socket.

Complete Script – modify-cpu.ps1:


$vCenter = 'localhost'

$numCPU = 4
# Include VMware-PowerCLI SnapIn
if(!(Get-PSSnapin | Where {$ -eq "vmware.vimautomation.core"}))
        Write-Host "Adding PowerCLI snap in"
        Add-PSSnapin VMware.VimAutomation.Core -ea 0| out-null
        throw "Could not load PowerCLI snapin"

Connect-VIServer $vCenter

$vm = get-vm $vmname
$spec=New-Object –Type VMware.Vim.VirtualMAchineConfigSpec –Property @{"NumCoresPerSocket" = $numCPU;"numCPUs" = $numCPU}
$vm.ExtensionData.ReconfigVM_Task($spec) | out-null

Disconnect-VIServer $vCenter -confirm:$false


Create the vCenter alarm that will call the script above with the VM name as parameter

Select the VM you want the virtual CPU count be changed after a shutdown and create a vCenter alarm (wow I am really using the webclient)

Give it a valuable name and describiton and select to monitor for a specific event


As an event where the alarm and therefore the action will be triggered we select VM – Powered off


And finally as an action we call our created PowerCLI script in C:\scripts with the following statement:

“c:\windows\system32\cmd.exe” “/c echo.|powershell.exe -nologo -noprofile -noninteractive C:\scripts\modify-cpu.ps1 {targetName}”

With the {targetName} variable we can transfer the name of the virtual machine that caused the Trigger.


And voila. If the VM is now getting powered-off -> the change of the virtual hardware will be done.


(German language, hm!? Doesn’t sound it quiet nice :P)


If the configuration is not working as expected, validate the functionality by call cmd with the user that is running the vCenter service:

> Open CMD

> runas /user:Domain\vCenteruser cmd

> “c:\windows\system32\cmd.exe” “/c echo.|powershell.exe -nologo -noprofile -noninteractive C:\scripts\modify-hardware.ps1 {targetName}”

Chapter 2:  Make it more awesome by using VMs and Templates Folders

To be more flexible on the hardware configuration I wanted to have the following functionality:

Drag and Drop a VM in a specific folder. If the VM is powered off change the virtual hardware based on specific settings. The settings should be extracted from the folder name.

Foldername specification: modify_ressourcetype_value

for example I create 4 Folders:

  • modify_cpu_2
  • modify_cpu_4
  • modify_ram_4
  • modify_ram_8

If you drag a VM in the modify_ram_8 folder and power it off, the VM will be configured with 8GB memory.

So I needed to change my script in a way that it gathers the folder the of the VM that is transmitted with the script call:

Complete Script – modify-hardware.ps1:

$vm = get-vm $vmname
$vmFolderName = $vm.Folder.Name

and split the foldername at the ‘_’ chars:

$items = $vmFolderName.Split('_')
$ressourceType = $items[1]
$amount = $items[2]

Now I can select the change RAM or CPU logic with a ‘switch’ statement:


         $vCPu= $amount
         $spec=New-Object –Type VMware.Vim.VirtualMAchineConfigSpec –Property @{"NumCoresPerSocket" =          
         $vCPu;"numCPUs" = $vCPu}
         $vm.ExtensionData.ReconfigVM_Task($spec) | out-null
          $NmbrRam = $amount
          set-vm -VM $vm -memoryGb $NmbrRam -confirm:$false
         write-verbose 'Wrong folder name specification'

If the VM is not in a folder that matches the specification. Nothing will happen.

Transfer the following modify-hardware.ps1 script into C:\scripts\ and configure this time the alarms in the same way as described above – but now on the specific folder we have created for this solution.

The action call this time needs to be renamed since I created another script:

“c:\windows\system32\cmd.exe” “/c echo.|powershell.exe -nologo -noprofile -noninteractive C:\scripts\modify-hardware.ps1 {targetName}”


Complete Script – modify-hardware.ps1:


$vCenter = 'localhost'


# Include VMware-PowerCLI SnapIn
if(!(Get-PSSnapin | Where {$ -eq "vmware.vimautomation.core"}))
        Write-Host "Adding PowerCLI snap in"
        Add-PSSnapin VMware.VimAutomation.Core -ea 0| out-null
        throw "Could not load PowerCLI snapin"

Connect-VIServer $vCenter

#Gather Objects and Data
$vm = get-vm $vmname
$vmFolderName = $vm.Folder.Name

#Split the folder name to extract parameter - Foldername must be modify_ressource_x
#while ressource must be ram or cpu and X the number of CPU or amount of RAM in GB

$items = $vmFolderName.Split('_')
$ressourceType = $items[1]
$amount = $items[2]


         $vCPu= $amount
         $spec=New-Object –Type VMware.Vim.VirtualMAchineConfigSpec –Property @{"NumCoresPerSocket" =          
         $vCPu;"numCPUs" = $vCPu}
         $vm.ExtensionData.ReconfigVM_Task($spec) | out-null
          $NmbrRam = $amount
          set-vm -VM $vm -memoryGb $NmbrRam -confirm:$false
         write-verbose 'Wrong folder name specification'

Disconnect-ViServer $vCenter -confirm:$false

Maybe some of you will benefit from this little script-collection. Have fun with it 😉

IMO: Is SMP fault tolerance even useful? My view on it!

Maish Saidel-Keesing has written a post about the fault-tolerance topic with multiple vCPUs a few weeks ago. He has valid points in his argumentation, but anyway I want to give you a little bit of my view on this topic (IMO).

With fault-tolerance two VMs are running nearly symmetrical on 2 different ESXi hosts with one (primary) processing IO and the other one dropping it (secondary). With the release of vSphere 6.0 VMware will support this feature with a VM of up to 4vCPU and 64 Gbyte memory. [More Details here]

I try to summarize the outcome Maish’s argumentation:

FT is not the big deal feature since it only protects against a hardware failure of the ESXi host without any interruptions in the service of the protected VM. It does NOT detected or deal with a failure at Operating Systems and Application level.

So what Maish think we really need are cluster-mechansims on application level and if legacy applications don’t.

I would in general not disagree with this opinion. In an ideal world all applications would be stateless, scaleable and protectable with a load-balancer in front of them. But we will need 1X or more years until all applications are built in such a new ‘modern’ way. We will not get rid of the legacy applications in the short-term.

Within the last 4 years of beeing an instructor I received one questions nearly every time when delivering a vSphere class:

‘Can we finally protect our SMP-VMs now with Fault Tolerance? No?! Awww :(‘

So I would not say there is a not a need out there for this feature. Being involved in some bidding last year we had very often the requirement to deliver a system for automation-solutions within large building-complexes (airports, factories, etc.).

Software being used in such domains are sometimes legacy application par excelente (ironic) programmed with a paradigm long before agile/restful/virtualization played a role in the tech-world.  Sometimes you can licence a cluster feature (and pay 10 time as much as for a 1-node licence) – sometimes you can’t cluster it and need other ideas or workaround to increase the availability.

Some biddings were not won because of opponents who where able to deliver solutions that can (on the paper) tolerate an hardware outage without any service-/session impact.

For me with SMP-FT typical design-considerations come into play:

  • How does the cluster work? Does it work on application/OS-level or does it only protect for a general outage?
  • What were failure/failover reasons in the past? (e.g. vCenter – in most cases I had a failure here it was because of Database problem [40%], Active Directory / SSO problem [10%], a hardware failure [45%] or rest [5%])  -> A feature like FT would protected against a huge amount of failure experienced in the past. Same considerations can be taken into account for all kind of applications (e.g. virtual load-balancer, Horizon View Connection Server etc.)
  • How much would a suitable solutions cost to make, buy or update?

Sure we need to get rid of legacy applications, but to be honest… this will be a very long road (the business decides and pays it) and once we have gotten to the point where the legacy applications are gone – the next generation of legacy applications is in place that need to be transformed (Docker?! 😉 ).

We should see FT as it is. A new tool within our VMware toolkit to fit specific requirements and protect VMs (legacy/new ones) on a new level with pros- and cons (as always). IMO every tool / feature that gives us more opportunities to protect the IT is very welcome.

VMware Update to vSphere 5.5 and Horizon View 6.0 – vCenter service not working properly

A few days ago I received a mail of a former student of mine. They have updated their VMware environment to the latest vSphere 5.5U2 and afterwards Horizon View from 5.2 to 6.0.

From a procedural point of view it has seemed that everything has worked fine. But on a second look he has realized that in the Horizon View Manager dashboard the vCenter was marked red (‘service is not working properly ‘) and pool operations were not working anymore.

vCenter service not working properly

From a systematic troubleshooting perspective I recommended him to check that the connectivity between the Connection and Server was doing fine. OSI Layer 1-4 were working well (ports haven’t been changed as well between the VMware versions). For the connectivity check of layer higher than 4 I told him to check the ‘classical-access-logs’ to see a problem with the authentication.



and to verify that the service-account has proper vCenter access and the correct permissions set within a role.

And voila –> the service user’s vCenter permission was removed during the upgrade (-> All other permissions were still in place).  Maybe a malfunction during the SSO / AD-LDS upgrade. Unfortuneatly I am not able to have closer look to do a root-cause analysis of it.

Anyway! If you observe similiar issues –> a) Use a systematic approach to verify system-communication or b) check directly the vCenter permissions.

A generic IT-Infrastructure operations manual model – A place to start

Last year I spent a lot time at the operations side of life. Bringing new IT-systems into a productive and operational state is a pretty interesting and challenging topic. Since I am a freelancer I need to get very often a pretty quick insight into new environments. And the first touching points are always important documents like the design document and ……

The operations manual

Having such a document has a lot of benefits:

  • Given a first insight to new members of the Operations team (IT- and Operating has a high fluctuation, hm?)
  • Offers a change to neutrally audit/review operations tasks by 3rd parties
  • Having all necessary information to operate a system in 1 document (yeah I know… the design documentation would be veerrry beneficial as well)
  • …………… so many more (please comment)

Even though many companies struggle to create one (sure it costs time and therefore money) I will try to give a good starting point with the following operating model I have created. This model can be used for creating a new operations manual from scratch or if you just want to audit your existing manual. Please be aware of that this a generic model and is not specific to certain environments.

I don’t claim that I know and included every important thing that must be included in such a manual… so feel free to give me feedback and I will update the document accordingly (if the discussion is leading to a conclusion which let the model evolve and bring more benefit to all of us).

I divided the model in 3 different sections that must be in each operations manual.

  1. General information: Which IT-Service is delivered? Which communication channels are used? Which persons are important for the operating and during escalations?
  2. Functional tasks / requirements of an ops-manual (Does anyone has a better wording to describe those ?) : Concrete tasks and information that are done or used by operators/administrators to keep the basic functionality of the IT-systems.
  3. Non-Functional tasks : Tasks to ensure performance and availability of the solution. Those tasks ensure the quality of the environment and are typically separated in two phases – detecting and acting (e.g. failure & recovery, performance problem & fix). IMO those are the tasks that are really important to grow from a pure cost-driver within a company to a service-provider. A lot of organizations are having structured methods for detecting, but missing a well a structured process afterwards.


At the end of the day I believe that each operations manual should give information about the mentioned items.  Having a structured document with all of those information of the environment separates boys from men (from an organisational/maturity point of view 😉 ) –> so let’s grow up, create one and give me feedback about your experiences with those type of documents.

VMware VCAP5-DCD (Datacenter Design) – Exam experience and learning philosophies

To be honest, I didn’t wanted to write a blog post about my VCAP5-DCD exam experience, since there are soooo many good articles and posts already online. Anyway, a lot of people were asking me what resources I used to be prepared for this exam to achieve the ….


First of all… thanks a lot to everyone else who has created posts about their experiences. I think I read all of the existing posts about the VCAP5-DCD exam.

The following content is structured in the following way. If you are just interested in real facts… please go directly to the resources part 😉

  • What type of ‘learner’ am I
  • Why I did the exam
  • What resources are useful to pass the Exam
  • Personal hints and tipps on doing the exam

What type of ‘learner’ am I?

I don’t even know if the expression learner exists in the English language. Anyway I believe that everyone who is extending his knowledge needs to find out HOW he is learning in the best way. I don’t just focus here on VMware, I try to be as general as possible in the following description.

My school career showed me, that I am not a good learner in a traditional way.  If I have to read book with pure theory where I don’t have any practical relevant relationship I cannot focus on more than 2 pages. Even if I read 50 pages out of the book my mind was only really active for the first 3 minutes.

So what does this mean? I personally need a practical relationship to some ‘issues’/’events’ I experienced in my life. This experience can derive from the following:

1. having experienced something in the real world (“live the challenge”) -> Maximum personal involvement, but pretty expensive (time/cost consuming)

2. talking and listening to someone who had an experience of something similar (“feel the challenge”) -> Medium personal involvement, but it might be hard to find the right people in the correct domain (e.g. User groups, conventions, tech talks, vBeer, …)

3. reading from someone who had an experience (“read the challenge”) -> personal involvement is low, inexpensive within the world wide web.

The more personal an information is absorbed by me, the more I realize the challenge, the better and more attentive I can read/learn new things.

This is exactly the reason why I love technical blogs. People are describing things more concrete and related to their experiences in a very personal way. This is something a technical documentation or book typically does not (of course there are exceptions).

Technical documentations and books are pretty good resources and very important as well, but in my case I need a personal experience first and afterwards I can read the technical documentation much more attentive with much more take-aways, since I am than aware and can think about concrete usage of the information.

Another important thing for me is the following. If I am confronted with a lot of new information over multiple days in a specific time (Web-ex, classroom teach, breakout-sessions) I personally need a few weeks to handle all this data.

When I am ‘attacked’ by a lot of information which I was not able to process (e.g. during the class) I need a break from those topics. Even though I am not mentally- and active working on those things my brain seems to make progress on the data subconsciously (‘excuse me for the non-scientifical correctness’). And in many cases suddenly something is happening with me that I call ‘illumination’ … everything out of nothing makes totally sense  from one second to another (‘no joke, from time to time some mathematical facts I have never understood in school are suddenly illuminated in my mind 😉 you see it might take a verrry long time…. next step…  find out to accelerate the illumination phase).

As a third thing important fact it is mentionable, that I need pressure. Without time pressure, my efficiency is typically decreasing a lot.

To summarize it all…. What do I personally need to extend my knowledge in the best way? Personal involvement AND time for the illumination….

So let’s see how this works all out for the VCAP-DCD exam.

Why I did the exam

The why is always important. During my career I have met so many people with all kind of environments, I worked in a lot of projects and talked to so many experts and there is one thing I realised pretty soon.

It is incredibly important to have a good architectural design of an IT system. And it is so easy to screw IT systems up if you don’t do it right. Since I am working in the IT field as a professional (and not only as a geek/nerd who loves technologies) I was always impressed about people working in a very structured/methodological way. Today (until SKYNET rises) IT systems are supporting people AND/OR businesses. This leads to the fact that an IT system needs to align to a business. If you only look at an IT system as a summary of technical best practice you will probably have a great technical solution, but it will not be the best solution for a business itself.

The idea of creating/collecting business requirements and design/transform these information (and probably even implement them) into a solid technical solution is in my opinion a skill every architect should be capable of.

I knew from several discussions, blogs, books that VMware’s highest certification (#VCDX) is exactly about approving this skill set. Since the VCDX is still a long term goal for me, the VCAP-DCD exam was the right one to take.

So I decided to take the VCAP-DCD exam in the beginning of 2013. And I found so many good excuses to postpone it month for month since than (Projects, Master thesis, …). Since I am only focussing on VMware in my job I have already read the most common VMware literature that is recommend for any kind of vSphere related exam (VMware vSphere Design, Clustering Technical Deep Dive, …). I was often involved in Design tasks/creations within my job (projects/trainings/discussions) so the exam preparation was kind of long-term preparation with many situations where suddenly the (knowledge-) illumination has kicked in (ILLUMINATION).

As time was passing in August 2014 I was giving myself a deadline that I MUST pass the VCAP-DCD until December 2014 (PRESSURE).  So I started to learn more concrete to the blueprint…and to be honest… it was a real exciting AND effective way of learning since I already had the personal involvement and practical relationship during all of these years.

What resources are useful to pass the Exam

Now I am getting concrete about the resource I used to learn and pass the exam.

  • Exam blueprint : First of all – as for each VMware exam, the exam blueprint is the baseline for each kind of certification. I decided to take the 5.1 version because of the fact that I have worked almost one year in a very large vSphere 5.1 environment and requested the exam authorization somewhen in 2013.
  • Clustering Technical Deepdive & vSphere Design : IMO those 2 books are a compulsory reading if you want to extend your knowledge in the vSphere field.
  • #vBrownbag VCAP5-DCD Video sessions : When I met Alistaire Cooke (a big contributor of vBrownbag), during the vRockstar party at VMworld I was not aware that 1 week later I the video sessions will have a large portion   of my successfully passed exam. In those video session very good experts are talking about every objective of the exam blueprint. A must watch for everyone who wish to take the VCAP-DCD exam. Nick Marhshall has collected all Video parts together on his blog . Personally I have only focussed on those topics where I thought that I have the least knowledge.
  • VMware VCAP-DCD51 Doc package: Jason Langer created a great document about VCAP-DCD relevant documents and structured them according the exam blueprint. Most of the files are official VMware documents, that are probably out-dated now if you want to take the 5.5 exam. But anyway since the design methodologies (Paper about differences of conceptual vs. logical vs. physical design) are not pinned to a specific version it is a really useful resource as well.
  • VMware Best practices Technical Whitepaper
  • Blogs Blogs Blogs about VCAP-DCD : I will not be able to mention all I have read, but just google VCAP-DCD and you will find a lot of entries. Everyone has made different experiences with the exam… some are focussing on the timing, on the technical challenges in the exam or how they have learned for it. Just a few links I had in my bookmark list:
  • Gather hands-on design and technology experience

I know a lot of architects, telling me that an architect must not know too much technical details about their solution. Honestly I agree to a specific point of the view that an architect MUST not know every configured item in the physical design by heart. But at the same time I believe am sure the more detailed knowledge an architect has, the better his design decisions will be. So it is a big advantage if you want to study for the DCD exam that you are professional on a vSphere Operational/Administrational point-of-view. I would recommend everyone to do the VCAP-DCA exam first. It makes life and the learning much easier if you are very familiar with the technical details of a vSphere environment.

Personal hints and tipps on doing the exam

Doing a design exam is hard to learn for, since this something where the personal experience plays an important role. If you have worked in the VMware design field for a specific time, you know the technology very well and you want to improve this knowledge, define a concrete time frame when you want to do the exam. One month before the exam, start to use the resources mentioned above. Read them, understand them, try to think how these design methodologies, technical best practices would have changed your past projects.  Try not just to learn those things, understanding and illumination is the key topic to successfully pass the exam. The DCD is an exam where you always will be up to a point that you do not know everything.

One more hint… stay calm…the technical implementation especially of the design parts where you drag and drop items into a logical design is pretty bad… The flash application was hanging up 3 times during my exam and I needed to talk all the time with the pearson administrator so that he was able to restart the system and test. Those things are pretty pretty annoying…but IMO if you can’t change things, make the best out of it…. stay calm..don’t get nervous and take a mental break.

Make sure that the items are connect to each other (if you move one item all other connected items should move as well) BUT DON’T try to mark all of them in the end… the system always crashed in my case…

I am not sure if that was an unlucky accident in my exam or is fixed in the 5.5 version… but I would not risk it anymore.

So everyone who is going to go the VCAP-DCD journey… good luck and if you want more information about it, feel free to ask me.


Killing me softly/hardly/forcely. How to kill a VM via #PowerCLI

Having observed some problems with VMs that were not able to be shutdown/powered-off properly via PowerCLI I tried to find a solution.

From time to time Shutdown-VMGuest didn’t worked and even an Stop-VM with the kill option were not working as expected. I knew that ESXTOP and ESXCLI have the options to kill a VM process/world if there are no other options. But since I wanted to achieve this in PowerCLI this blog post from the year 2011 gave me the correct hint.

We can use ESXCLI via PowerCLI to fulfil that task  😉 *whoopwhoop*.

I was missing a feature to kill those worlds without authenticating directly against an ESXi-host and since ESXCLI and it’s namespaces have changed a little within the last years I wanted to document now how this can be achieved in vSphere 5.X.

First of all connect to the vCenter via

Connect-VIServer $vCenterFQDN

Get the VM-Object you are going to kill

$vm = Get-VM $nameOfTheVM

find the ESXi-Host where the VM is running on

$esxi = Get-VMHost -VM $vm

and load the ESXCLI functionality

$esxcli = Get-EsxCli -VMhost $esxi

Now it’s time to extract the WorldID out of the ESXCLI VM PROCESS LIST data

$worldid = $esxcli.vm.process.list() | where{$_.Displayname -eq $hostname} | Select WorldID

and kill the VM with the options soft, hard, force


VOILA the VM should be definitely killed right now. This ESXCLI commands is  not being tracked by the VPXA, so no events of the ‘kill’ are written down in the database. (With great knowledge comes great responsibility, right? ;-))

If you are running this command against a VM as part of an HA-Cluster. The HA-mechanism will reboot the VM after the kill. In this scenario you need to disable the HA-protection of the VM (so it is removed from the HA protected list) before you are going to kill it via.

$vm | Set-VM -HARestartPriority Disabled

I hope this information might be useful to some of you guys.

Please use the Code-Snippet here to see the fully-functional (Kill-VM.ps1) script.



CPU ready spikes & Host Power Management

We just observed some strange frequently occurring CPU ready spikes in our environment (screenshot).

cpu ready spikes

This effect occurred very frequently on each virtual machine. What caused it?

-> The Host Power Management mode which was balanced. After setting it to high-performance the spikes disappeared.

I know this might have to do with the specific hardware we are using, but since I heard about such effects from time to time, think about disabling the power management mode on the ESXi once you observe strange performance symptoms. IMO server are deserving high performance and nothing less 😉

I will change my mind once I have measured the financial benefit that might occur with the balanced power management mode. So if you have any concrete facts and number. Please post it here.



IMO: #VMworld 2014 recap VMware EVO:RAIL (part 2)

This is part 2 of my IMO #VMworld wrap up. Read my about thoughts of a new product called EVO:RAIL

IMO: #VMworld 2014 recap on VMware NSX (part 1)

IMO: #VMworld 2014 recap VMware EVO:RAIL (part 2)

IMO: #VMworld 2014 recap VMware vCloud Air (part 3)

IMO: #VMworld 2014 recap vSAN and vVol (part 4)

IMO: #VMworld 2014 recap Automation & Orchestration (part 5)



EVO:RAIL is a pretty cool so called hyper-converged solution provided by VMware and partner vendors like (DELL, EMC, Fujitsu, INSPUR, net one, SUPERMICRO, HP, Hitachi). Summarized Evo:Rail delivers a complete vSphere-suite (including vCenter, vSAN, Enterprise+ & vRealize suite) bundled with 4 computing nodes which is from a technical perspective ready to be productive in less than 30minutes (the record at the EVO:RAIL challenge was <16 minutes).

Such a solution is a thing I thought about a long time ago (it was one of the outcomes of my master-thesis on the software-defined datacenter in small-/medium sized enterprises) especially for small environments where the admins want to focus on operating the running systems (or better: delivering an IT-service) rather than implementing, installing and configuring basic infrastructure (Yeah I know this is going to be a shift in the future for me as a trainer who delivers a lot of install, configure manage classes and did installations as part of my consultancy/implementation jobs).

IMO VMware did a very smart move not to get into the role of a hardware vendor and did a cooperation with existing and well-known partners to deliver the solution specified/managed via the EVO:RAIL engine by VMware. The established sales channel to customer and companies can be used. Especially small- and medium sized business will be attracted by this solution as long as the pricing/capex ist affordable for them. Which means from a business perspective the following: VMware delivers the software (vSphere, vRealize and the EVO-engine) and the vendor delivers the hardware & support. The business-management (#beersherpa) guy inside of me says…. perfect… everyone stays at its core competencies and bundle the power together to bring a much better solution for the customer (One contact point for support, a completely integrated and supported virtualization stack, shortest implementation times).

I believe for the big x86 vendors this solution is just a next step in becoming a commodity. Isn’t the whole software-defined datacenter thing about decoupling software from hardware, creating/using a smart VMware controlled control plane and a commodity data plane which is responsible for the concrete data processing based on the control plane logic? We don’t or will not care anymore if the hardware (switch, storage, computing nodes) is HP, Cisco, Juniper, IBM, etc. We will care about the control plane.

With EVO:RAIL it will get even tougher for the hardware vendors to differentiate from each other and the competition in the end can only be won by the price (in the small/medium sized market). I want to add that I missed the chance in the EVO:RAIL demo room to have a discussion about this topic from a vendor perspective (damn you VEEAM party 😉 ), so if you have done anything similar or have own opinions please comment on this post or contact me directly.

The use cases of EVO:RAIL can vary (Management Clusters, DMZ, VDI, small production environments) a lot and I believe that this is a product is a pretty good solution which will be triggered from a bottom-up perspective within the companies (I am referring to my bottom-up / top-down approach of bringing innovation in companies at the NSX post (link)). Administrators will love to reduce the setup time of a complete vSphere environment.

Especially for VDI solutions I can imagine a brilliant use case for the EVO:RAIL, which means next step… VMware please bundle the VMware Horizon View licence into EVO:RAIL and integrate the View setup into the Evo- engine :-).

Useful links around EVO:RAIL:

IMO: #VMworld 2014 recap on VMware NSX (part 1)

It’s really long ago that I have put any content on this blog, but the amount of discussions during VMworld Europe this year have lead to the situation that I somehow need to get out my thinkings/opinion (IMO) on all this new trending VMware topics. Feel free to confront me with my statements and I would love to defend or readjust them. (That’s how knowledge expansion works, does it?!)

While writing the several parts I have realized that it was suddenly much more content that I had in mind the first place, so I separated the articles in several parts. All articles are reflecting my personal opinion (IMO) and are differing a little from the other posts we have published so far on

IMO: #VMworld 2014 recap on VMware NSX (part 1)

IMO: #VMworld 2014 recap VMware EVO:RAIL (part 2)

IMO: #VMworld 2014 recap VMware vCloud Air (part 3)

IMO: #VMworld 2014 recap vSAN and vVol (part 4)

IMO: #VMworld 2014 recap Automation & Orchestration (part 5)

VMware NSX

NSX is the newest technology by VMware trying to enable the software-defined network (and be a part of the software-defined datacenter). I put a lot effort on NSX over the last days and must admit: this is a really cool concept and solution. We create a logical switch within and across all of our datacenter. You can define rule based networks (who can communicate with whom (DMZ, Multi-Tier Application) and have it integrated inside of the VMkernel (e.g. IP-traffic routed inside the ESXi instead of touching the physical devices).

Pat Gelsinger described it very well during his keynote. “The datacenter today is like o a hard boiled egg – hard on the outside, soft on inside”. NSX will enable to deliver security mechanisms within the virtualized datacenter as well integrated in the VMkernel of the ESXi.

NSX will offer us a great flexibility managed in a central point (NSX Manager) via an UI or API which can be used by orchestration engines likes vCO.

From a technological perspective this is definitely awesome, but will we see a similar development of NSX like we have seen with the x-86 virtualization products? IMO I don’t think so on a short- to mid-term.

The advantages of NSX will come to play in very large environments with high flexibility and security requirements (financial services, IT-provider, e.g.) which means I don’t see a mass market currently out there in the next years. This does not mean it won’t be a financial benefit for VMware (good things never come for free), but only a few of us will be confronted with a NSX installation or customer who are going to implement it.

The second thing I see is that those large enterprises will get faced with organizational challenges when implementing NSX. From my experiences and chats I had during VMworld, large enterprises typically have different organization units for Network and Virtualizations. Technologies like NSX will have a huge impact on the guys from the network team and from my personal feeling (I know a lot of network guys and had chats around those topics) I doubt that the network guys do want this product out of their own conviction.

This lead to the fact that with the implementation of a software-defined network an organizational transformation in the companies will be mandatory. Network and Virtualization (Storage and Programmers of course as well) team would need to re-organized as a single…(yes I hate buzzwords, but I think this describes it best) software-defined datacenter unit.

This means that the (software-defined) evolution inside the datacenter needs to be top-down driven by the management, which might lead to a high resistance in current organization and time-intensive process-changes (Network processes matured a lot during all the years). VMware will need to convince their customer on a much higher (organizational) level, than probably for vSAN/EVO:Rail which are IMO products wanted by the admins.

That should not mean I don’t believe in NSX. I believe that this is a great technology, but we should be aware of that the transformation to a software-defined network is not only a technical thing we are implementing and which will be automatically adopted by the network admins (which would be something like a bottom-up innovation). An adoption on the technical and organizational level will be crucial for the success of NSX.

I wish VMware good luck on this task, since I would love to get involved in some NSX projects in 2015.

Useful links around NSX:

Loading VMware Appliances into OpenStack

Today, I want to talk about the challenges of loading VMware appliances into OpenStack Glance and give you a recipe of how to do it.

I migrated my lab to OpenStack but need to be able to test the latest VMware products in order to keep myself up to speed. As VMware provides more and more of its software as virtual appliances in OVF or OVA format it makes sense to have them in Glance for provisioning on OpenStack.

The following procedure is illustrated at the example of vCAC Appliance 6.0:


Challenge 1: Format Conversion

If you get to download the appliance as OVF you are already one step ahead. OVF is simply an XML-based configuration and does not include any information required to run the VM on OpenStack.

OVAs on the other hand need to be unpacked first. Luckily, OVA is nothing but TAR:

$ file VMware-vCAC-Appliance-
VMware-vCAC-Appliance- POSIX tar archive (GNU)

So we continue extracting the archive:

$ tar xvf ../VMware-vCAC-Appliance-

The .ovf, .mf and .cert files can be deleted right away. We will not need those anymore. After that the VMDK files must be converted to QCOW2 or RAW:

$ qemu-img convert -O qcow2 VMware-vCAC-Appliance- VMware-vCAC-Appliance-
$ qemu-img convert -O qcow2 VMware-vCAC-Appliance- VMware-vCAC-Appliance-


Challenge 2: Multi-Disk Virtual Machines

Unfortunately, OpenStack does not support images existing of multiple disks. Bad luck that VMware has the habit of distributing their appliances with a system disk and a separate data disk (*system.vmdk and *-data.vmdk). To still load this appliance into Glance we need to merge the disks into a single one:

First, we use guestfish to get some information on the filesystems inside the disk images:

$ guestfish

Welcome to guestfish, the guest filesystem shell for
editing virtual machine filesystems and disk images.

Type: 'help' for help on commands
'man' to read the manual
'quit' to quit the shell

> add VMware-vCAC-Appliance-
> add VMware-vCAC-Appliance-
> run
> list-filesystems
/dev/sda1: ext3
/dev/sda2: swap
/dev/sdb1: ext3
/dev/sdb2: ext3

It is safe to assume that the EXT3 on /dev/sda1 is the root file system, so we mount it and have a look into /etc/fstab to see where the other filesystems should be mounted:

> mount /dev/sda1 /
> cat /etc/fstab
/dev/sda2 swap swap defaults 0 0
/dev/sda1 / ext3 defaults 1 1
proc /proc proc defaults 0 0
sysfs /sys sysfs noauto 0 0
debugfs /sys/kernel/debug debugfs noauto 0 0
devpts /dev/pts devpts mode=0620,gid=5 0 0
/dev/sdb1 /storage/log ext3 rw,nosuid,nodev,exec,auto,nouser,async 0 1
/dev/sdb2 /storage/db ext3 rw,nosuid,nodev,exec,auto,nouser,async 0 1


Next, as we want to get rid of sdb all together, we remove the entries in /etc/fstab

> vi /etc/fstab

and mount /dev/sdb1 to /mnt. This way we can copy the contents over to /storage/log:

> mount /dev/sdb1 /mnt
> cp_a /mnt/core /storage/log/
> cp_a /mnt/vmware /storage/log/

Of course, the same has to happen for /dev/sdb2:

> umount /mnt
> mount /dev/sdb2 /mnt
> cp_a /mnt/pgdata /storage/db/

And we are done! Please not it is important to use cp_a instead of cp_r as it will preserve permissions and ownership.

> exit


Challenge 3: Disk Space

Well, so far so good! But the image that originally served as the system disk now has to hold all the data as well. Therefore, we need more space! So we have to resize the disk image, the partition(s) and filesystem(s). Here is the easiest way I’ve found:

$ qemu-img create -f qcow2 newdisk.qcow2 50G
$ virt-resize --expand /dev/sda1 VMware-vCAC-Appliance- newdisk.qcow2
Examining VMware-vCAC-Appliance- ...

Summary of changes:

/dev/sda1: This partition will be resized from 12.0G to 47.0G. The
filesystem ext3 on /dev/sda1 will be expanded using the 'resize2fs'

/dev/sda2: This partition will be left alone.

Setting up initial partition table on newdisk.qcow2 ...
Copying /dev/sda1 ...
Copying /dev/sda2 ...
Expanding /dev/sda1 using the 'resize2fs' method ...

Resize operation completed with no errors. Before deleting the old
disk, carefully check that the resized disk boots and works correctly.


What’s happening here? First, we create a new empty disk of the desired size with “qemu-img create”. After that, virt-resize copies data from the original file to the new one resizing partitions on the fly. Next, the filesystems are resized using resize2fs.

The image can now be uploaded into Glance. Please make sure to add properties that set the SCSI controller chipset properly. For example, IDE is going to work. The reason for this is the name of disks: using VirtIO we will get /dev/vda1 and would have to adjust those name e.g. in /etc/fstab, too. I have only had luck with ide so far:

glance image-update --property hw_disk_bus=ide 724ad050-a636-4c98-8ae5-9ff58973c84c

Have fun!

© 2019 v(e)Xpertise

Theme by Anders NorénUp ↑