In my previous three articles I have gone through how you can configure a set of Virtual machines with Windows 2003 x64 Enterprise Edition as a Majority Node Set cluster. I have discussed some issues that I have come across and how I worked around them.

In part 4 of this article I would like to discuss how I installed Exchange server on each of my VMWARE nodes as a CCR cluster.

Firstly – what is CCR?

CCR stands for Cluster Continuous Replication, and is a new feature in Exchange 2007 (Enterprise) that increases the availability of your Exchange servers. It is based on asynchronous log shipping and replaying whilst utilising Windows 2003 MNS Clustering.

The key benefits of CCR are:

• Unlike conventional clustering, there are no specific hardware requirements for your nodes / storage
• The requirement of a central “shared storage” array is removed (as it makes use of the Windows 2003 MNS and log shipping)
• Due to the above, in typical clustering scenarios the shared disk is the single point of failure, whereas using CCR removes this requirement
• As MNS Clusters can be installed in geographically dispersed locations (as long as they are in the same subnet) you can have nodes in separate locations (covering Disaster Recovery and Business continuity issues)

When a passive node is added into an Exchange 2007 CCR cluster, each configured storage group and associated databases are copied from the Primary Node to the passive node which is called “seeding”. After the initial copy logs are replicated and replayed into the passive database continuously.

As well as providing a highly available solution for businesses, CCR is very useful when applying updates to Clustered Mailbox servers as you can make use of the Exchange Management Shell command “move-ClusteredMailboxServer” which locates services to the passive node then the administrator can apply patches and service packs safely.

One point of note worthy of mention here is, that typically in a normal Exchange 2003 cluster, if an administrator wished to move resources from one node to the other, they would either use the Cluster Administrator “move group” option or from the command line use the “cluster.exe group “Exchange EVS Group” /move” command – this is NOT recommended in Exchange 2007 CCR environments – you are advised to use the “move-clusteredMailboxServer” cmdlet which checks the status of data replication, and if the passive node is ok to take over from the primary.

There are so many reasons why CCR clustering is a exciting addition to Exchange 2007 – for a more in-depth overview please have a look here: http://technet.microsoft.com/en-us/library/c5f5da15-f593-40c1-838d-e6123adb5e10.aspx

Installing Exchange 2007 a CCR IN VMWARE;

Ok, if you have just joined us – please cast you eyes over parts one, two and three – its not essential, but it might help you get an understanding as to what I have been doing and where we are. From here on I am going to go through Installing Exchange 2007 in my test lab, and explain some of the problems that I have encountered.

Before we begin, I have a confession to make, in order to produce these articles, I run through the setup and document the configuration and the problems, therefore – some of the screen shots that you may see contain names that have changed from the end product – this should not effect any of you trying to replicate what I am doing – and I have tried to keep things consistent throughout.

Before we begin a couple of tips:

Again a slight confession – I made a few mistakes before I got to the end and had a working installation, but, even though I had made errors they were exceptionally valuable from a learning curve perspective – the following are some of my installation “Bloopers” feel would be good to share…

Problem 1: In one of my first attempts to get CCR working, I configured the VMWARE guests with shared disks (doh!) – my excuse was that it had been a long night, and typically I have been used to shared resource clustering – needless to say my Exchange server nor my cluster worked very well (corrupted Exchange databases, wouldn’t fail over and other stuff that goes on and on).

What made this mistake interesting (apart from the fact that it was really stupid) was that during the installation I received the following error:

Indicating that the shared resource could not start – after I had figured out what had gone wrong (e.g. that there was contention between the nodes for the data due to the fact that I was using shared disks – and both nodes were talking to the disks and database files at the same time) I decided to un-install Exchange and reconfigure the clustering service.

Going through the process of “Add Remove Programs->Microsoft Exchange->Remove” on both nodes (Passive First) the removal of Exchange seemed to go well, however when I went to the Cluster Administrator to evict the nodes from the cluster (as I needed to reconfigure the cluster service as a proper Majority Node Set) I received the following error “An error occurred attempting the read properties for the “Microsoft Exchange Database Instance”:

I selected “Yes to All” which allowed me to open up the cluster administrator, but indicated that the clustering resources that had been added to my machines had not been removed properly. Conscious that this could create issues further down the line – and also I suspected that if you are to use normal clustering with Exchange 2007 you may get the same error after an un-installation I used the following to remove the references from the cluster service:

I opened up the Windows command prompt and typed in the following commands – pressing enter after each:

cluster resourcetype “Microsoft Exchange Database Instance” /Delete /Type

cluster resourcetype “Microsoft Exchange Information Store” /Delete /Type

cluster resourcetype “Microsoft Exchange System Attendant” /Delete /Type

The above gave me full access back to the cluster administrator I then evicted the passive node from the cluster, and then the primary node.

I then went away reconfigured my VMWARE guest machines to have local storage and then installed the MNS clustering and tried to install Exchange again – it was at this point I ran into my second problem:

Problem 2:  When running Exchange setup during the analysis phase it gave me an error and would not proceed – I checked the event log and found the following entry Source: MSEXCHANGESETUP Event ID: 1002:

This puzzled me – I had removed Exchange, recreated the cluster – why was this happening?

After some quick research I found that Exchange 2007 Setup now stores its progress in the registry (see here) – so I navigated to:

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Exchange (see below)

Right clicked on the “Exchange” parent Key and deleted it – and then re-ran setup which then worked as expected, which leads us to the main point of the article – installing Exchange CCR functionality on the nodes that we have created.

NOTE: The registry key needs to be deleted on both nodes

Installing Exchange;

You may remember that back in part 1 – I copied all of the Exchange binaries to the B: drive of my Exchange servers (for faster execution) so, where drive path are mentioned they will be from b:\ExchangeMedia

Firstly I logged into the node which was to become the primary node of my cluster (x64EXND01) and then went to START->RUN and typed in B:\ExchangeMedia\Setup.com which after a few seconds brought up the following dialog box:

As you will see from above – as I had already installed the prerequisites in part 1, all I needed to do was click on the option entitled “Step 4: Install Microsoft Exchange”.

When I clicked on this option the setup program presented me with the following dialog box:

Here I clicked on the “Next” button – located at the bottom of the screen – doing this presented me with the license agreement screen:

Here I clicked on the “I accept the terms in the licensing agreement” and then clicked “Next”:

Here I was asked if I wished to send error data to Microsoft, normally I am glad to tick yes, however as this is a lab and not connected to the Internet I picked “No” – and then clicked on the “Next” button:

From the screen above I ensured that the “Custom Exchange Server Installation” was selected – I accepted the default Binary installation path and then clicked on the “Next” button:

On the screen above (which allows me to choose the roles that I wish to be installed on my first node) I selected (ticked) the “Active Clustered Mailbox” role – you will notice that this then removes the ability for you to install any other role on the server, and will elect to install the Exchange Management tools by default – when I was happy with the above I clicked on the “Next” button.

Here I am asked about the type of cluster that I would like to configure – here I picked the CCR option, rather than the Single Copy Cluster (just for information the single copy cluster can be considered as the more traditional Exchange Cluster Model that we are used to for example it makes use of shared storage). I then provided my Exchange CCR cluster with a name (by entering it in the section entitled “Clustered Mailbox Server Name”) and provided it with a FREE IP address (in the section entitled Clustered Mailbox Server IP Address) – finally I choose the location for the Database files – this was entered in the section entitled “Specify the path for the Clustered Mailbox Server Database Files” – now remember not to do what I did and specify a Shared Storage location – this is CCR clustering and I am a idiot.

The Exchange setup went away and analyzed my Hardware and domain infrastructure and came back with a warning – this was to let me know that I did not have any Hub or CAS servers in my Organisation – which is ok, as at this stage I didn’t – so I clicked “Next”.

This then began to install Exchange 2007 on my node – as per above (and after about 20 minutes) everything completed ok.

I clicked on the “Finish” button and was returned to the main Exchange setup screen, which I closed.

I then logged onto the passive node in my cluster and ran through the the same process as per above, with the only difference being when at the server role selection screen I selected “Passive Mailbox Role” – see below;

When the Exchange setup program had completed I was presented with the following dialog:

Post Installation Checks;

Ok now it would appear that I had a working Exchange CCR cluster – however, I needed to prove that certain operations where working correctly – these were as follows:

1. Inspect the system and application event logs for errors
2. Ensure that the CCR functionality was working correctly
3. License the cluster

Inspect the system and application event logs for errors

Much to my amazement I could only see two errors in the Application event log Event ID: 9317 which contained the following information:

Event Type: Error
Event Source: MSExchangeSA
Event Category: General
Event ID: 9317
Date: <date>
Time: <time>
User: N/A
Computer: <computername>
Description:
Failed to register Service Principal Name for exchangeRFR; error code was c0072098.

Event Type: Error
Event Source: MSExchangeSA
Event Category: General
Event ID: 9317
Date: <date>
Time: <time>
User: N/A
Computer: <computername>
Description:
Failed to register Service Principal Name for exchangeMDB; error code was c0072098.

After some quick research (KB935676) – I found that using the following PowerShell cmdlets resolved the issue:

add-ADPermission -identity “cn=ccr-x64-ex,cn=computers,dc=infrastructure,dc=local” -user “x64exnd01$” -AccessRights WriteProperty -Properties “Validated-SPN”

I ran the above command on both nodes and then used the following Exchange Management Shell commands to take the Exchange CCR cluster offline: stop-clusteredmailboxserver -identity ccr-x64-ex -StopReason “Permissions Update”

The above command will take all of the associated databases, and services including the system attendant offline – which, implements the permissions changes that we have just configured – I used the start-ClusteredMailBoxServer -identity ccr-x64–ex to bring the cluster back online – however I could have also used the move-clusteredMailboxServer -identity:”CN=CCR-x64-EX,CN=Servers,CN=Exchange Administrative Group (FYDIBOHF23SPDLT),CN=Administrative Groups,CN=Infrastructure,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=infrastructure,DC=local” -target:x64exnd02 -moveComment:”Permissions Update” to achieve the same results, although I did not wish to move the role to the other node.

The following is an example output from the stop command:

One point of note here is that the preferred method to closing down your CCR cluster is to use the stop-clusteredMailboxServer command.

Ensure that the CCR functionality was working correctly

Here I wished to check to ensure that the fail-over functionality of the cluster was working between nodes – in order to do this I needed to use the following command:

move-clusteredMailboxServer -identity:”CN=CCR-x64-EX,CN=Servers,CN=Exchange Administrative Group (FYDIBOHF23SPDLT),CN=Administrative Groups,CN=Infrastructure,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=infrastructure,DC=local” -target:x64exnd02 -moveComment:”Test Move”

QUICK TIP: I have found that when you add more than one Exchange 2007 server into your Organisation the Exchange Management Shell requires you to use the Distinguished names of your Exchange servers, rather than NETBIOS names – this can take some time to type in the management shell, so what I have done is place a text file on the desktop of each node with the DN of my clustered Exchange server contained within it, I then type the EMS command into the file and then paste it into the ESM command Window.

I then used the following command to check for the results of the move:

get-clusteredMailboxServerStatus -identity:”CN=CCR-x64-EX,CN=Servers,CN=Exchange Administrative Group (FYDIBOHF23SPDLT),CN=Administrative Groups,CN=Infrastructure,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=infrastructure,DC=local”

Which gave me the following output:

If you cannot read the output the following is a text example:

[PS] C:\>get-clusteredMailboxServerStatus -Identity:ccr-x64-ex

Identity                   : CCR-x64-EX
ClusteredMailboxServerName : CCR-X64-EX.infrastructure.local
State                      : Online
OperationalMachines  : {X64EXND01 <Quorum Owner>, X64EXND02 <Active>}
FailedResources        : {}
IsValid                    : True
ObjectState                : Unchanged

As you can see from the above the x64EXND02 node now owns the CCR cluster – you can also check the CCR replication status by looking at the database location on each of the nodes – see below:

License the cluster

It had now come for me to license the cluster – in order to do this I used the following EMS command:

set-exchangeserver -Identity ‘CN=CCR-x64-EX,CN=Servers,CN=Exchange Administrative Group (FYDIBOHF23SPDLT),CN=Administrative Groups,CN=Infrastructure,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=infrastructure,DC=local’ -ProductKey ‘xxxxx-xxxxx-xxxxx-xxxxx-xxxxx’

Ok, at this stage we now have a working two node CCR cluster running Exchange 2007, I hope that you have all enjoyed this series – I will be expanding on this over the next few weeks by adding in HUB, CAS and Edge roles and providing an update to my feelings on my personal migration which may have changed due to the work that I have been doing here.

I will also be providing some tools that I have used which help me construct a LAB environment – which I hope that will help you all along.