Syed Jahanzaib Personal Blog to Share Knowledge !

April 1, 2013

Lotus Domino Cluster / High Availability

Filed under: IBM Related — Tags: , , — Syed Jahanzaib / Pinochio~:) @ 4:22 PM

After several crashes of our production Lotus Domino Server dueto some application corruption, we finally decided to upgrade it with latest version and also decided to add Cluster server for fail over support. Clustering is an excellent topology which really works well specifically in domino environment, and is also used for high availability / load balancing high load. Following is an small howto reference guide which I created for future reference. Using this guide you can also create cluster server for your domino server. It really helped me in many situation when database corruption occurred at production server or if i have to plan any upgrade and downtime is required. Notes client switch over to cluster is almost seamless and most of the time, client even don’t notice it that he is on cluster server🙂 ~


Some Best Practices for clustering that I have learned so far:

[Some points in below paragraph have been taken from Lotusphere 2011 – How to build a better cluster – By Gabriella Davis – Paul Mooney]

– Clustered Servers should not use the same disk storage array for example SAN/NAS, Its because if somehow there is some problem or failure with the SAN/NAS, secondary should continue its services using its local storage or alternate storage.

– Clustered Servers should not use the same power source, Its because if somehow the power source of primary server fails, secondary server services should be available via alternate power source.

– They should not be in the same building and never in the same room, Its because if there is some physical disturbance like Fire, Earthquake or others, secondary server should keep continue serving its mail service.

– Should have enough CPU power and memory. „It’s common that clustered servers need more power and more disk resources than un clustered servers.

– Few minutes or an hour spent on testing could save you countless hours in deployment

– Ignoring errors or alerts that appear during replica creation or cluster setup will come back to haunt you later

– Always use latest Version of Domino/Notes clients possible with latest available Feature Pack (FP) , Currently 8.5.3 with FP3 is good. Also failover in notes client is good, fast and almost transparent to users.

Ok lets start  . . .

I assume that you already have working domino server in place. first thing first, create server ID that will be use when secondary domino server installation is initiated at cluster server.

Current Scenario:

Primary Mail Server: D1
Secondary Mail Server: D2
Domain Name: syed

Create Server ID for Secondary Server

–  Open Domino Administrator Client,
–  Goto Configuration ,
–  On right side panel, click on Registration > Server ,
–  In Server Name, type your Secondary Server name,
–  In Domain name, type your domain name
–  In Location for storing server ID, click on in file, and select your destination folder. (Save it any folder that you can access later from secondary server to copy this ID file)
–  Click on Green Icon of tick, and then click on REGISTER

As showed in the images below . . .

Now move to your secondary cluster server,


At your Secondary Server, Initiate Domino Server Setup, Click NEXT to Continue . . .


Click on Setup an additional serverd2-2

Click on Browse and select the D2 server id that we created earlier at production server. (You can copy the id file from the production server to this cluster server)d2-3





Type in your Production server name , so this secondary server will know from where to pull its replication data.d2-7




After clicking on Setup it will start configuring its initial setup.

After it finishes, double click on the DOMINO SERVER icon on your desktop to launch the server.

After initiating, it will start replicating names.nsf

Now its time to add this secondary server into cluster group at production server configuration.

Add Secondary Server into Cluster Group at Primary Server

– Open Domino Administrator Client,
– Goto Configuration ,
– Goto All Server Document, Here you will see both server names, Primary and Secondary
– Select both servers and click on Add to cluster

As showed in the images below . . .

A popup will ask you if you want to continue, Click on YES
It will ask you for cluster name with option of “Create new cluster“, click OK to continue

As showed in the images below . . .

Now enter your cluster name and click on OK OK
It will ask you if you want to create it immediately, click YES

As showed in the images below . . .


Wait few minutes so that it creates cluster.
Now at your Primary Server domino Console. Issue following command to start replication of Primary Domino Databases to Secondary Server. (not mail boxes)

replicate D2

As showed in the image below . . .


After its replication finishes, its time to create replicas of your mail files to secondary server. For this purpose you have to first assign necessary permissions at secondary server.

Assigning Permissions for Create Replica

– At your Primary server,
–  Open Domino Administrator Client,
–  Goto Configuration ,
–  Goto All Server Document, Here you will see both server names, Primary and Secondary
–  Double click on Secondary Server.

As showed in the image below . . .

Goto Security,
In Server Access – Who Can section, Add your Admin ID and Primary Server name in following Sections.
  – Create database & templates
  – Create new replicas

As showed in the image below . . .


Click on Save & Close

Create Replicas of Users Mail Files

To create mail replica copies of user mail files from primary to secondary, First you have to create mail folder in secondary server, because when domino first install, it donot create mail folder.

– At your Primary server,
– Open Domino Administrator Client,
– Goto File / Open Server , Select your secondary server, if its not appearing in the list, then click on Other and it will show you both servers, click on secondary server. You can also type in your secondary server name with its domain, for example D2/syed .Now you will be connected to your secondary server.
– Goto Files , Right click on Mail Parent folder of Domino, and create New Folder name mail (In my case I installed domino at D:\Lotus\domino\data

After the folder is created, Switch back to your primary server.
– Goto Files section,
– Select your desired mail file, (For test select single mail file, later you can select all or multiple mail files at a time to replicate), On right side panel, Click on Databases > Create Replica(s)

As showed in the image below . . .


A new window will popup with some options, First click on Show me only cluster members” so that your cluster servers should appear in the list. Now select secondary server, and click on ADD,
Then select all three options of
– Copy Access Control List
– Create full text indexing for searching
– Exchange unread marks on replication.

As showed in the image below . . .

Now click Ok to continue.

Depends on the mail file size , It will few minutes or longer to complete the task. After it creates replica, the user mail file will appear on secondary server under mail folder.

TIP: You can force immeidate replication by using following command at console on both servers.

tell adminp p all

and monitor the console for information or errors.

. . .  TESTING  . . .

Before going further, some important tips and information on Clients and Clustering

[Some points in below paragraph have been taken from Lotusphere 2011 – How to build a better cluster – By Gabriella Davis – Paul Mooney]

The Notes client is “Cluster Aware

● When it communicates with a Domino server that is in a cluster, that server updates a file on the client with its cluster name and its cluster mates
─ this makes the client aware that the server it wants is in a cluster and means even if the server is down, the client has built in information about where to go next
─ if you add a domino server to a cluster and rapidly shut down the original server, the notes clients will never get a chance to “learn” about the cluster
● If a primary server fails, the Notes client will search the other cluster mates it knows about for a replica of the database it wants

─ Searches cluster mates alphabetically,  not by availability
– this is very important in your planning
─ Opens database on the first available cluster mate it finds


● The cluster.ncf is a file in the data directory on the client
● It contains details of all clustered servers you have connected to from that client, along with their cluster mates
– there can be several clusters in your cluster.ncf
● If you delete the cluster.ncf the Notes client will forget all about any clusters it was aware of, but will relearn the first time you reconnect to a clustered server

Notes 8.5.2 – Improved Client Clustering/Repilcas

● Client Transparency
● In the 8.5.2 or above client, the ability to failover to a cluster mate without the user being notified or stopped in their work has been improved
● During the following activities, if the primary server goes down, the user will be invisibly transferred to one of the cluster mates
─ Creating a new entry for PIM views (Mail message, Calendar entry, or To Do)
─ Opening a document in an application
─ Opening a different folder or view
─ Replying or forwarding with attachment
─ Creating a new mail message while multiple documents are selected
─ Deleting a single or multiple document
─ Sending a message
─ Switching back to Primary server when it becomes available

Ok Enough theory, its time to hit the road.

Configure your Lotus Notes Client and send an test message to any user or yourself.
You will see in logs that it immediately replicate/duplicate mail from your primary server to secondary server.

As showed in the image below . . .


You can also verify it by directly open user mail file from secondary server using admin client😀🙂

Testing from the client  side . . .

To test fail over switch at client end, temporarily disable Primary server LAN ,
At client end Open Notes client , when it will fail to find primary server , it will auto switch over to secondary server.
Also Try to send an test message to yourself for test purpose while the primary server is down for test purpose.

As showed in the image below . . . (client have switched over to D2 i.e our secondary clustered server)

At secondary server console , you will see notification of user failing over from primary to secondary.

As showed in the image below . . .




Now enable LAN on Primary server that we temporarily disabled for test purposes.
After few minutes, Secondary server will auto replicate data to primary server , so that primary server can update itself with the changes that occurred while it was offline.

As showed in the image below . . . (Replication in progress from D2 secondary server to D1 primary server)


& at your client end, you will be switch back to primary server where you can see your email that you sent from the secondary server while the primary was down.






Triggering Failover for CLUSTER

[Some points in below paragraph have been taken from Lotusphere 2011 – How to build a better cluster – By Gabriella Davis – Paul Mooney]

You can force all users to fail to another cluster server
─ Useful for Disaster Recovery testing
─ Useful for server maintenance work
─ Useful for migrations

● On the server console type the following command

Set Config Server_Restricted=
1 Prevent any new sessions opened on this server until restart

● Administrators bypass this restriction if using the Domino Administrator client

2 Prevent any new sessions opened on this server until setting changed

● Administrators bypass this restriction if using the Domino Administrator client

● Setting is persistent even through server reboots

0 Feature disabled

Also issue drop all command to force disconnection of all connected clients. („By the way, iMap and POP3 users will still have access to there Mail).

You can also set the maximum number of concurrent NRPC users allowed to connect to a server in NOTES.INI using following variable


„Set variable to a number determined in planning stage

Set variable using console command Or use NOTES.INI tab in server configuration document

Set config Server_MaxUsers = desired maximum number of active concurrent users, „Additional users will fail over to the other members of  the cluster



Creating Replication Document for backup purposes

Its a good idea to always have a connection document replicating between server cluster members because it replicates stubs too. Also if something got missed from routine replication, this forced replication will make sure everything is in synch.


Centralized Management is GOOD🙂 by having single Configuration Document for all servers

Its a good idea to centralize manage all servers in cluster group.

Open Admin Client, Goto Configuration tab, here you will see two configuration docs for primary and secondary.
Double click on primary server configuration doc to take its properties,
As showed in the image below . . .

1- separate-config-for-servers


On Basic tab scree, select check on “Use these settings as the default settings for all servers:
Then click on Save & Close.
As showed in the image below . . .

2- make one



Now you will see * [All  Servers] doc
As showed in the image below . . .

3- after config

Now rename the second doc with the secondary server name with any other name so secondary server will not take data from it, all configuration settings should be taken from centralized document * [All  Servers]

As showed in the image below . . .

4- final

WAN Connectivity Tips for Primary & Secondary Server

There are several scenarios that wan connectivity can be maintained. For example A live can be configured on Mail server WAN interface , but I will not recommend to put mail server on danger by directly attaching it to the internet, better to configure live ip on any of your router/firewall and then configure port forwarding of smtp/pop3 traffic to your server.
In this scenario If you have only single live IP , then you have to manually change the IP of primary server to secondary in case your primary goes down and you want to receive mails on your secondary server. Lot of discussion can be made here, but no time for it at a moment🙂

Scenario # 1

You have public ip pool of /30. For example and Both ips are configured on Router and router do port forwarding for mail traffic. First configure to forward mail traffic to primary server and to forward mail traffic to secondary server. Now Request your web server hosting company to set secondary MX record to (It is required so when no response is received from primary server smtp/pop3 , then it should try to send traffic to secondary MX which is

Now to test , quit primary server, wait few minutes, and then send an test email from the internet to your domain, it should arrive in your secondary server, and when the primary will be available , it will replicate its data with one another.




  1. SYED JAHANZAIB – Thanks for sharing Lotus Articles it is really very helpful……..

    Comment by Sumit — April 2, 2013 @ 9:58 AM

  2. Hi SYED
    Please I would like to ask some help concerning Butch evans Qos script
    I dont know if you already tried it ? I’m asking because I bought it a month ago but
    struggling to set it without real success and unfortunately all my calls and mails to get help from Wispgear result of no responses (nice way to sell and leave customers ….)
    right now I’m planning to set it on both hotspot and also pppoe server any of them running on RB1100
    I think my big deal with this script is metarouter… anyway any help will be welcome

    Bests regards

    Comment by maxwellnina — April 16, 2013 @ 4:13 AM

    • I have not used Butch’s scripts as they are commercial scripts and available on payment only.
      You can get the best support only from the creator as he knows how and for what purpose this script is really created for.
      I can’t comment on it with out seeing them and the without knowing the goal required.

      Comment by Syed Jahanzaib / Pinochio~:) — April 16, 2013 @ 8:48 AM

  3. Assalamu alaykum,
    I found your blog well designed and very informative

    Mustaque Ahmed

    Comment by Mustaque Ahmed — July 1, 2013 @ 6:20 PM

  4. HI ,
    Is it possible to use the common user database file for 2 cluster servers. For ex. My home server is a and b is cluster of a. if both of the server points the mail file path at same location? It is also considered as cluster??

    Comment by Kuber Shrestha — July 14, 2013 @ 6:02 PM

  5. I do have domino cluster (A and B servers) configured but recent failure of server A it did not work as expected!. All incoming mails from outside domain bounced back with error home server A cannot reach…I had to reconfigure all lotus clients to use (location) server B,,, I really suprised where is the clustering aware of lotus notes clients?

    Comment by sbayasa — October 14, 2013 @ 9:55 AM

    • You have to make sure that your hostname is able to reach other on notes level.
      I have proper clsutering configured, and at my router level too. so when the Server A goes down, all emials automatically goes to server B

      Comment by Syed Jahanzaib / Pinochio~:) — October 14, 2013 @ 12:40 PM

  6. this is very helpful for beginners like me. thanks a lot dude. God Bless

    Comment by Mart — January 26, 2016 @ 1:04 PM

RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Create a free website or blog at

%d bloggers like this: