Cluster Manager, Server Manager & Hyper-V Console not starting

This week I had a very strange issue with a Hyper-V Cluster managed by Virtual Machine Manager.

Completely randomly different cluster nodes failed and I weren’t able to start failover cluster manager on one of the cluster nodes.  On the infected node it self, I wasn’t able to open the hyper-v manager or server manager.

After a lot of research I found a solution from the windows server core team which pointed me to the solution.

Unable to launch Cluster Failover Manager on any node of a 2012/2012R2 Cluster

When Failover Cluster Manager is opened to manage a Cluster, it will contact all the nodes and retrieve Cluster configuration information using WMI calls. If any one of the nodes in the Cluster does not have the cluster namespace “root\mscluster” in WMI, Failover Cluster Manager will fail and give one of the below errors:

clip_image002

Or,

Unfortunately, it does not give any indication of which node is missing the WMI namespace.  One of the ways you can check to see which one has it missing is to run the below command on each node of the Cluster.

It can be a bit tedious and time consuming if you have quite a few nodes, say like 64 of them.  The below script can be run on one of the nodes that will connect to all the other nodes and check to see if the namespace is present.  If it is, it will succeed.  If the namespace does not exist, it will fail.

—————–

 
—————–

In the below example, you can see that one of the nodes failed.

To correct the problem, you would need to run the below from an administrative command prompt on the “failed” node(s).

cd c:\windows\system32\wbem
mofcomp.exe cluswmi.mof

Once the Cluster WMI has been added back, you can successfully open Failover Cluster Management.  There is no restart of the machine or the Cluster Service needed.

Quote: Microsoft Ask the Core Team Blog

In my case I wasn’t able to fix it so easy because the server vendor implemented the WMI Provider directly in his BMC via Agent (for the interested ones Fujitsu). during the process of recompiling the WMI for the Cluster the whole Server Network interfaces and BMC fail.

so my fix:

  1. shutdown the server
  2. make it powerless
  3. start it
  4. check cluster (everything fine)
  5. uninstall the (fucking) agent

Since than it worked.

How do you get your System Center Virtual Machine Manager really highly available

Sometimes when I’m invited to visit a customer to “optimize their high available virtual machine manager”, I normally see following configuration.

VMMHA01

 

When I ask why they say it is high available, they normally tell me that they can move the machine from one host to another. Normally i ask now “And what happens when you need to patch the SQL DB, VMM or Windows Server or the storage fail?”

Here comes the point where most people realize that high availability means other things than moving services from A to B.

So now let us think what we need to get our VMM Server high available.

On the VMM Site we need following parts:

  • two VMM Management Servers running in a Cluster
  • two Database Servers running in a Cluster
  • two Fileserver running in a Cluster as Library server
  • two Hyper-V Hosts for VM Placement
  • two Storages with Storage Replication

VMMHA02

 

When it comes to a very big Hyper-V and VMM Environment, I would suggest to run you Management Systems in a separated Hyper-V Cluster. That helps you to keep your VM workload running even when you need to take down your fabric cluster in maintenance mode.

VMMHA03

How to dump firewall rules into file with powershell

Hi everybody,

again a short note for my self to not search again in my offline notes. 😉

How to dump Firewall rules with Powershell 3.0 and newer in a file “oneliner”.

Inbound rules:

 

Outbound rules:

 

How to fix non compliant NICs with no logical network connectivity in VMM

Hi everybody,

today again a blogpost out of my heading “I need to remind my self because I love to forget this”.

I want to show you how you need to configure Network Adapters which are not used in a virtual switch for Hyper-V. That network adapters could be used for example with Storage like iSCSI, SMB3 or other things.

At first you will get a warning from Virtual Machine Manager that your host is not compliant.

2015-10-09_17-37-52

Like you see in my example below.

2015-10-09_12-15-23

 

Now there are two options to get the NICs compliant. The first way, which we don’t want to use with Storage Adapters is, to move them into a virtual switch.

The second one, which I prefer when working with storage adapter, is ti define the connection within the hardware profile of the server. For that you go into the properties of your Hyper-V Hosts in VMM and select hardware. Scroll down until you reached the network adapter you want to configure. Now you should see the logical network connectivity.

2015-10-09_17-39-46

There you check the logical networks, your host has connection with and press ok.

2015-10-09_12-16-44

So that’s all. Now your Networkadapter is shown as compliant in VMM.

2015-10-09_17-38-42

How to fix VMM error 20553

Hi,

today I want you to provide you some GPO Templates, which could help you to fix following error in VMM:

Error (20553)
The Windows Remote Management (WinRM) client on the VMM server cannot process the request. A computer policy does not allow the delegation of the user credentials to the target computer **.

WinRM: URL: ** , Verb: [ENUMERATE], Resource: [http://schemas.microsoft.com/wbem/wsman/1/wmi/root/cimv2/Win32_ComputerSystemProduct], Filter: []

Unknown error (0x803381a3)

Recommended Action
Use gpedit.msc and look at the following policy: Computer Configuration -> Administrative Templates -> System -> Credentials Delegation -> Allow Delegating Fresh Credentials. Verify that it is enabled and configured with an SPN appropriate for the target computer. For example, for a target computer name myserver.domain.com, the SPN can be one of the following: WSMAN/myserver.domain.com OR WSMAN/*.domain.com OR WSMAN/*

Thanks to Radhika Gupta for his blog on TechNet which gave me the final solution 🙂

In my case I needed to create two GPOs.

The first deployed on the Hyper-V Hosts to enable WinRM with CreedSSP

Computer Configuration\Administrative template\Windows Components\Windows Remote Management (WinRM)\WinRM Service\[Allow CredSSP authentication] = true

The first deployed on the VMM Hosts to enable WinRM with CreedSSP and Credentials Delegation

Computer Configuration\Administrative template\Windows Components\Windows Remote Management (WinRM)\WinRM Service\[Allow CredSSP authentication] = true

Computer Configuration\Administrative Templates\System\Credentials Delegation\[AllowFreshCredentials ] = “WSMAN/*”