Effects of Losing an Operations Master (FSMO) Role Holder in Relation to an Active Directory Forest and/or Domain

Introduction

THE PAGER just went off.  It’s 3am.  You’ve been asleep for two hours after just finishing a brutal online battle where you saved the world from an encroaching Evil.  So what is this pager thingy and why does it insist of beckoning for your attention in a loud, obnoxious fashion?

You look through sleepy eyes at the display.  Server down.  Ouch.  And with your organization shooting for 99.999% uptime to support 24/7 manufacturing operations, you are subliminally aware that the clock is ticking.  Thinking through the 3am fog, let’s see – this server was destined to be decommissioned as part of your schedule for next week.  All of the file shares and printers were relocated to another server last Fri.  It was performing no critical network services…  And, then you remember.  The server Decomm wasn’t complete yet because that server is the RID Master for your Active Directory domain and that role still needed to be transferred.

So .. the question becomes, is this a call you place to your Manager giving her a heads-up about your awareness of the server down, but assuring her that it can wait until morning without causing any inconveniences or is this something which requires immediate attention?

What if the downed server wasn’t the RID Master, but rather the domain’s PDC Emulator and your organization is still running a Mixed-Mode environment (thank you Sales and Marketing for not upgrading their Windows NT 4.0 Server Domain Controllers)?  How does this impact the need for expediency to resolve the server down situation?

Lost, but not Down

Schema Master (Forest Role)

All Domain Controllers contain a copy of the Active Directory Schema.  This Schema is essentially a template or listing of the various Active Directory object types and available attributes present within a given forest.  This template is used to refresh the Active Directory database where the actual objects are stored.

The loss of the Schema Master role holder in an Active Directory puts the forest into a state of stasis so no extensions (addition of object types and/or attributes) to the Schema can be made.  This would impair activities such upgrading an Active Directory domain from Windows Server 2000 to Windows Server 2003, installing Microsoft Exchange, and/or adding new attributes to an object.  All things considered, as this sort of activity does not happen on a daily basis, a forest could survive the loss of this role holder and continue with minimal inconvenience in most cases.

Domain Naming Master (Forest Role)

If the Domain Naming Master role holder is lost, domains won’t be able to be added or removed from the Active Directory forest.  DCPROMO is also affected, meaning that servers can neither be promoted nor demoted.

Though the loss of this role holder impacts some more common operations performed within an Active Directory forest and its contained domains, it is still doesn’t create highly visible issues within your environment.

RID Master (Domain Role)

If the Domain Controller performing as the RID Master goes down or becomes inaccessible, Windows 2000 and above domain controllers will have no place to acquire new RID pool assignments.  As this function is only called upon sporadically, unless you are adding security principals in bulk, this outage may not become apparent for some time.

A more noticeable occurrence may be the failure of the movetree.exe command to function properly as it relies upon the RID Master present in the domain that the object is coming from to actually perform the move.

Infrastructure Master (Domain Role)

In the event that the Infrastructure Master role holder is lost, the ramifications will vary based upon whether the forest is in itself a single domain, or if it contains multiple domains.  If everything within your Active Directory forest is contained within a single domain, the Infrastructure Master really doesn’t have anything to do as there are no cross-domain references to be maintained.

In a forest with multiple domains, the Infrastructure Master role holder plays a more vital role by maintaining cross-domain references (ie users from Domain A are members of a group in Domain B).  In Domain B, these references (the users from Domain A) are called phantoms.  If the Infrastructure Master for Domain B were to go down, changes to phantom objects would no longer be maintained and communicated to the Domain Controllers in Domain B.  So if one of the user objects in Domain A changes it’s UPN, that change will not be maintained across Domain B.  Now the kicker here, any server in Domain B that is a Global Catalog Server will be automatically maintained due to the intercommunications of the GC processes forest-wide.  This would make for an intermittent issue as some servers would have stale phantom references and others would be up-to-date.

PDC Emulator (Domain Role)

Okay, in short, the loss of any Domain Controller performing an Operations Master (FSMO) function will not be the end of your environment; but each does have a potential for impact given a sufficient window of absence.  In usual circumstances, however, the most crippling role to lose is the PDC Emulator.

In a mixed-mode environment, when the PDC Emulator goes down, you lose the bridgehead server for Windows NT 4.0 networks involved in trust relationships with that Active Directory domain.  You also lose any down-level updating of the SAM for your Windows NT 4.0 Backup Domain Controllers, therefore Active Directory account changes such as password changes, login name changes (somebody marries/divorces), etc are never communicated.

Regardless of the mode or functional-level of the domain, you stand the risk of losing time-synch within the domain (assuming that all of the hosts in the environment are configured in accordance with best practice).  Essentially, the PDC Emulator in the forest is to be synchronized to an outside, commonly recognized time source.  The PDC Emulators in the parent domains should look to the forest PDC Emulator for time synch.  The PDC Emulator in a child domain will look to its parent and so on and so forth.  The other hosts in each domain should, in turn, look to their authenticating Domain Controller for their time sync.  With component broken, it has the domino effect .. the more complex and distributed the domain/forest, the greater the potential of Kerberos failures as the clocks fall apart from one-another on the Domain Controllers or clients which were directly dependent upon the defunct PDC Emulator.

Any Domain sitting above Mixed Mode would still also be susceptible to password changes not being communicated across the domain in a timely fashion.  Let’s explore this…

Say a user at Site C calls the Help Desk requesting a password reset.  The Help Desk person (located in Site A, where the PDC Emulator is) goes to reset the user’s password.  This person realizes that the server he normally changes passwords is down, so he resets the user’s password on another Domain Controller located at Site A.  Once performed, the Help Desk person instructs the user to try to logon.  The user tries, and after a bit of delay, his logon still fails.  Why?

Simple.  The server on which the password change was made has never communicated it beyond the confines of Site A, and more importantly, this change has never been communicated to the Active Directory domain’s PDC Emulator.  Since the PDC Emulator is down, this change was never communicated to the recognized “final authority” in that domain for password authentication issues.  The Domain Controllers in Site A will be communicated this change rather quickly, but due to the timing of Intersite Replication, the bridgehead server for Site C won’t become aware of this change for some amount of time.

Now, when the Domain Controller in Site C is presented with the user’s new credentials, it compares the password (presented by the user) to the password previously known for that user (stored in its copy of the Directory).  The two passwords won’t match.  So the next step for the Domain Controller in Site C is to contact the domain’s PDC Emulator located in Site A for determination.  After some amount of waiting due to the lack of response from the PDC Emulator, the Domain Controller in Site C has no recourse but to deny the user’s authentication attempt.

Finding Your Operations Masters Role Holders

In a prior paper, I showed you how to identify the Active Directory Operations Masters through the available GUIs.  There is a much easier way.

Quick FSMO Role Holder Identification

Install the Support Tools for your OS (Windows 2000 and up).  This suite of tools includes a command-line utility called NETDOM.  Open a command prompt and execute the NETDOM command in accordance with the following syntax:

NETDOM QUERY /DOMAIN:domain FSMO

This will return the FSMO role holders for the indicated domain.  This will also indicate the Schema Master and Domain Naming Master role holders for the forest.  Viola – all information in one simple query!

You are now empowered with information.  You have identified the players, you know what they do, and you know what happens if they go down .. so .. where do you go from here?  Let’s talk about architecture, placement, of the Operations Masters role holders and Microsoft-recommended best practices.

Operations Masters Placement

Design by default, let’s review this.  First-up Domain Controller in an Active Directory Forest will hold all of the roles (including the Global Catalog).  If you know you are designing a forest which will have multiple Domains, you will see a stumbling-block fast.  The Global Catalog and the Infrastructure Master role should not exist on the same server unless your Active Directory environment will only be comprised of a single domain.

That said, bring-up a second Domain controller for the forest and transfer the PDC Emulator, Infrastructure Master, and RID Master roles to that new DC.  Leave the Schema Master, the Domain Naming Master, and the Global Catalog on the initial Domain Controller.

For redundancy, it would be a good idea to bring-up a third Domain Controller to serve as a secondary Global Catalog server for that site.  If this is done, the RID Master and the PDC Emulator roles could be moved to this server if desired.  Keep in mind, however, the PDC Emulator role holder will probably be the busiest of all role holders as many servers will query it and request processes to be performed.  Ensure the hardware platform is reasonable for the task.

Microsoft recommends having the RID Master and the PDC Emulator roles located on the same Domain Controller.  This is primarily because of two reasons:

  1. The RID Master is a relatively low-traffic, low-overhead
  2. Having the RID Master and the PDC Emulator on the same Domain Controller enhances the performance of the creation of objects at the request of NT 4 BDCs when working in a mixed-mode environment.

Any domains created within the forest should take a similar approach to the one shown above; however the Schema Master and Domain Naming Master roles will not be addressed on a domain-by-domain basis as they are Forest-wide roles.  Once addressed at that top-level, they are set. The placement of the Domain-specific role holders should still follow the pattern indicated above.  It is also recommended that those role holders be located at a site within the Domain which provides the best bandwidth and most reliable intersite connection.

One final suggestion, at remote sites it may be a good idea to setup two Domain Controllers for redundancy.  Aside from the inherent benefits, each should be configured as a Global Catalog server.  This is especially viable in Windows Server 2003 environments due to the enhancements for granularity in its replication of objects and attributes.  Windows Server 2003 will utilize less bandwidth during the Global Catalog synchronizations and yield better performance for forest-wide object look-ups initiated from users at the remote site.

How Do We Transfer Roles?

The Domain and Forest roles can be transferred via GUIs or via the command-line utility NTDSUTIL.  For the following activities, we are going to use only the command-line as it is simpler from the stand-point that all operations, transfer or seizure, for all roles can be performed from this one utility.  For these activities, we will assume a basic familiarity of both the Windows and the Command-line environments exist.

1)      Open a command prompt

2)      Type ntdsutil

 

3)      At the ntdsutil: prompt, type roles

4)      At the fsmo maintenance: prompt, type connections

5)      At the server connections: prompt, type connect to servername

Note:  The servername must be a Domain Controller within the domain in which you are working.  This will be the destination (or recipient) for the FSMO/Operations Master role to be transferred.  NetBIOS names usually suffice.  If the user you are currently logged in as does not have credentials recognized on the destination Domain Controller, may be prompted.  In most cases, Domain Admin credentials are appropriate.

Binding to dcb2k3 …

Connected to dcb2k3 using credentials of locally logged on user.

6)      At the server connections: prompt, type q

7)      At the fsmo maintenance: prompt, type your desired action

Action

Description

transfer domain naming master Transfers the Domain Naming Master role to the connected server
transfer infrastructure master Transfers the Infrastructure Master role to the connected server
transfer pdc Transfers the PDC Emulator role to the connected server
transfer rid master Transfers the RID Master role to the connected server
transfer schema master Transfers the Schema Master role to the connected server

After the transfer process has completed…

8)      At the fsmo maintenance: prompt, type q

9)      At the ntdsutil: prompt, type q

This will exit the NTDS Utility and return you to the command prompt

How Do I Seize a Role?

Okay .. Let’s say disaster has struck and you have lost a Domain Controller which was an Operations Master.  For whatever reason, you need to regain the functionality provided by that role holder immediately.  This is why we love NTDS Utility.  The process we will use is very similar to that of transferring an Operations Master role.

1)      Open a command prompt

2)      Type ntdsutil

3)      At the ntdsutil: prompt, type roles

4)      At the fsmo maintenance: prompt, type connections

5)      At the server connections: prompt, type connect to servername

Note:  The servername must be a Domain Controller within the domain in which you are working.  This will be the destination (or recipient) for the FSMO/Operations Master role to be transferred.  NetBIOS names usually suffice.  If the user you are currently logged in as does not have credentials recognized on the destination Domain Controller, may be prompted.  In most cases, Domain Admin credentials are appropriate.

Binding to dcb2k3 …

Connected to dcb2k3 using credentials of locally logged on user.

6)      At the server connections: prompt, type q

7)      At the fsmo maintenance: prompt, type your desired action

Action

Description

seize domain naming master Assigns the Domain Naming Master role to the connected server
seize infrastructure master Assigns the Infrastructure Master role to the connected server
seize pdc Assigns the PDC Emulator role to the connected server
seize rid master Assigns the RID Master role to the connected server
seize schema master Assigns the Schema Master role to the connected server

After the seize process has completed…

8)      At the fsmo maintenance: prompt, type q

9)      At the ntdsutil: prompt, type q

This will exit the NTDS Utility and return you to the command prompt

Seizing a role has more ramifications in an environment than a more graceful transfer does.  In fact, a transfer is a clean hand-off.  A seizing operation assumes that the state of the Operations Master role present on the DC to which the role has been assigned was intact and up-to-date. There can be inherent fall-out from this assumption.  Anytime a role is seized, be prepared to watch for fallout .. aberrant events in the System Event log.  Monitor this for a while and be prepared to deal.  Seizing a role is really a last effort, not to be undertaken lightly.  If the role holder can be brought back-up, OS drive array (system state) intact, within an acceptable amount of time it would be best to pursue that course of action.

Conclusion

Divide and distribute, redundancy, live to fight another day – those are the thoughts I would like to leave you with.  Of course, from an Active Directory perspective redundancy applies to the Global Catalog servers, not the Operations Master roles.  Design your enterprise Active Directory structure to mitigate risk.  Harden your servers and use well-connected sites as much as possible, especially between the primary data centers for each domain.  Distribute the Operations Master roles amongst the Domain Controllers in that location providing that location is the best based on the following criteria:

  • Bandwidth
  • Power
  • Cooling
  • Security
  • IT Support

Know your environment; know what is at risk with each server and Domain Controller.  Doing so will help you rest, even on the nights when you get the 3am pager alerts.

Bibliography

  1. “Understanding and Identifying the Operations Masters within Your Active Directory Environment”, Rick Gregson, Convergent Technologies, http://www.convergenttechonline.com/documents/understanding%20and%20identifying%20the%20operations%20masters%20within%20your%20active%20directory%20environment.htm
  2. “Using Ntdsutil.exe to transfer or seize FSMO roles to a domain controller”, Article #255504, Microsoft Support Knowledge Base
  3.  “FSMO placement and optimization on Active Directory domain controllers”, Article #223346, Microsoft Support Knowledge Base
  4. “Flexible Single Master Operation Transfer and Seizure Process”, Article #223787, Microsoft Support Knowledge Base
  5. “Initial synchronization requirements for Windows 2000 Server and Windows Server 2003 operations master role holders”, Article #305476, Microsoft Support Knowledge Base