Short checklist what you should do after you configured a Microsoft Failover Cluster

Hi everybody,

today I will provide a short checklist what I do after I configured a Microsoft Failover Cluster.

I need to say, the blogpost is inspired by some consultants who think they are so gifted with fucking awesomeness that they can install a (mal)functioning Hyper-V Cluster incl. System Center Virtual Machine Manager with all Components and Software Defined Network in only 6 hours in whole and even don’t know what a VLAN or IP Subnet is.

So than let us start.

We are now on the point that you successfully installed your failover cluster.

2015-08-30_12-15-17

  1. You need to configure the Cluster Quorum and Witness for your cluster. I would suggest you to use the same witness typ like the storage you use. So if you use a SMB File based storage you should use a fileshare witness or even with Server 2016 an Azure Witness. If you use a block storage, you should use a disk witness on the storage your hosting you LUNs with. Mixing up different types of storage and witness in a cluster could sometimes a bit troublemaking. Best Practice is to use disk witness if possible. When you are using fileshare witness never open a fileshare on a Host within you cluster or a virtual machine which is running on the cluster. The could properly result in some issues or even a split brain issue during maintenance or failure scenarios.
  2. After you configured the quorum, you should configure the communication of your cluster heartbeat. Therefor you can use the following small script.
  3. Configure the firewalls of you cluster nodes. NO NOT DISABLE THE FIREWALL, configure it as it is needed for your service. The reason why you shouldn’t disable the firewall is that at first you lose a security layer and open gates for attacks within the network. The second is that some windows services and applications may not function right with disabled firewall.
  4. Afterwards you need to configure the Active Directory Organizational Unit delegation so that the cluster service can create and change objects within the active directory. That is needed to create cluster aware update or new cluster roles. Delegation of Cluster Machine Accounts with Active Directory
  5. If you need  or wish to configure Kerberos constrained delegation, now is the point to do so for your cluster.
  6. Configure cluster aware updating for cluster. Starting with Cluster-Aware Updating: Self-Updating
  7. Configure your backup
  8. Make Failover tests for all cluster nodes, cluster roles and services and test you backup and the recovery
  9. Last but not DOCUMENTATION. Document what you have done, so that also your coworkers can see how awesome you are 😉

I hope that helps you a bit in your daily work.

Cheers,

Flo

Rolling Cluster Update with Windows Server 2016 TP3 – short notes & first tries

Hi everybody,

the following post is just a short one out of my learnings during my tests with rolling cluster upgrade.

In the first place, I think many of you already noticed the new failover cluster feature. It enables you to migrate clusters deployed on Windows Server 2012 R2 to Windows Server 2016 without building a new cluster and migrating the cluster roles to it. Currently there is only a validation for clusters running Hyper-V and Scale out Fileserver but as soon as I have some more time I will also try to Upgrade some of my Virtual Machine Manager and Fileserver Clusters and report back to you.

The way how you migrate the cluster is already very well documented on technet.

For those of you who are familiar with Active Directory Migrations, the way a Failover Cluster is Upgrade looks very familiar. At first you have three phases like shown in the figure below.

Source: Microsoft TechNet

Preparations before you start with the migration.

  1. Check if your Servers are compatible with Windows Server 2016. Run the new build only on supported environments.
  2. Ensure that you have always enough compute resources during the whole time upgrade process. Normally you run a cluster with a minimum of n+1 cluster nodes. During the cluster upgrade, I would suggest to add another node to the cluster and run with a minimum of n+2 nodes. That would prevent you from any resource shortages during the upgrade.

In the first Phase with nativ Windows Server 2012 R2 you have the following tasks to perform:

  1. Run Cluster Aware Update on your Cluster and Update it to the lates patchstate
  2. Backup your Cluster Database and Cluster Configuration
  3. Install the first 2016 node, add the server role and failover cluster feature and features like MPIO (if needed). Please note inplace upgrades of nodes are not supported, so please reinstall the nodes.

Source: Microsoft TechNet

In the second Phase, you will run in cluster mixed mode:

Please notice that the mixed mode is only supported for 4 weeks and you should get out of it as soon as possible. Anyway, you should take your time to check if the new hosts and the cluster runs stable. As soon as you are on Windows Server 2016 native mode there is no way back.

  1. Add the first 2016 node to the cluster
  2. when the node is added  properly and runs fine, migrate to cluster role over to the new role
  3. if the migration runs fine and for example the VMs are working, set the first Windows Server 2012 R2 node in maintenance mode and drain the roles. After that evict the Windows Server 2012 R2 node
  4. Now you can install the second node and redo the steps 1. to 3. until you have removed all Windows Server 2012 R2 nodes

At this point as, long as you still have one Windows Server 2012 R2 node left in the cluster you can go back if anything goes wrong.

Source: Microsoft TechNet

At the end, you have a native Windows Server 2016 cluster node running in functional level Windows Server 2012 R2. Like an active directory with Windows Server 2012 R2 and running on forest function and domain level Windows Server 200 R2 before you raised the level.

Source: Microsoft TechNet

Now we enter the third stage, here we need to raise the Cluster Function Level. For that we need to run a powershell command.

So please open the PowerShell Commandline on one of your new cluster nodes as administrator.

 

Afterwards you can start your backup again and restart the cluster aware update service.

Source: Microsoft TechNet

 

Now the last point, housekeeping. That means, update the virtual machine versions of you VMs and install the new version of the virtual machine management tools or what ever need to be done for the cluster roles.

So that’s all from my site today. I will write a much more detailed post, as soon as Windows Server 2016 reaches RTM.

 

Updating permission setting for folder ‘ ‘ failed when I install a high available SQL Instance on cluster shared volume

Hey everybody,

the following blog is more to remind my self on a mistake I do very often. 😉

When I install a SQL Failover Cluster and a High Available Instance on a cluster shared volume, I get the error “Updating permission setting for folder ‘ ‘ failed”.

IMG-20150725-WA0002

There are different posts to solve the issue more or less complex.

SQL Installation Error updating permisson setting for folder

SQL Server 2008 installation will fail if the setup account does not have certain user rights

Permission error installing Failover Cluster instance

In my case the solution was pretty easy. I mostly forget to create a sub directory for SQL Databases and Files on the cluster shared volume. So as example:

False:

C.\ClusterStorage\SQL-Backup\ <- will give you the error

Right:

C:\ClusterStorage\SQL-Backup\Files\ <- will work fine

So easy solution for the error, create a subfolder and install use that path during installation.

Renominated for Microsoft TechConnect Community (former CLIP)

Today I got a great message from Florian Endres (Community Program Manager @ Microsoft) that my Windows Server User Group Berlin and I in person were renominated and transitioned for Microsoft TechConnect. A Commnity Program that values local communities, community contributors and leaders for their feedbacks and work that they do with their local communities and members.

I’m very proud to now be in the second year Clip/TechConnect member and I’m happy that people see some value within my work and hobby in the community. 🙂

tc-badge-blue-2

 

Removed Cluster Node still shows it’s self as a cluster member

Hi guy’s,

one thing some of you maybe notice from time to time. When you evict a node from a cluster it can happen that the cluster node it self says it belongs still to a cluster and your not able to force it into a new one or use the node as independent server.

2015-08-08_20-15-37

 

The reason for that is quite simple. There are some points which are configured in a AD Computer Account and DNS for a Cluster Node. Sometimes it happens, that not all attributes are deleted during evicting the node. Most likely it is the following attribute.

2015-08-08_20-14-47

 

So now there are three way’s to solve the issue:

1. Remove the the failover clutser feature from your node, reboot and reinstall it if needed. That fixes the issue in 80% of all cases (in my personal experience) .

2015-08-08_20-33-10

 

2. Remove the cluster node from active directory, delete the computer objekt and rejoin the node. That work in 100% of all cases because you have a totally new computer object and GUID with no old stuff in.

2015-08-08_20-33-52

3. Or for the guy’s and girls who love some pain. Search your AD Computer Attributes and DNS for all cluster entries where the fault node is still in and edit the entries. I wouldn’t suggest it because it is very risky and takes very long time.