I am trying to learn how the failover clustering features works with Windows 2008 R2 as I am going to be using as part of SQL Server 2012 always on.
I have been able to find information on how to set it up, and what it does. However, I am having trouble finding a nice technical document on how it actually works in detail (For example with keepalived there are docs that explain how often are heartbeat packets sent, what they look like etc).
I know this is a pretty broad question, but I would like to have a good understanding of mechanisms of this feature.
Being that it's a proprietary product I don't think you're going to binary-level analysis of the protocol or even flow diagrams, but I think you can glean some helpful information from:
The Design and Architecture of the Microsoft Cluster Service - This paper, co-authored by one of my CompSci heroes Jim Gray (RIP, Jim), published in the 1998 IEEE Proceedings of FTCS, describes the basic design of the Microsoft Cluster Server product as it was in Windows NT 4.0, which the later products all evolved from. There's even a flow diagram here, though it's probably wildly out of date with respect to the current generations of the product.
Network failure detection and recovery in a two-node Windows Server 2000 cluster - This article, while dated, does describe the low level details of the state machine used to determine network availability of nodes in a cluster.
[MS-CMRP]: Failover Cluster: Management API (ClusAPI) Protocol Specification - This document makes statements about how the product works internally thought its purpose is not specifically documenting the intra-cluster communication protocols and flows.
This functionality has gotten more configurable in later versions of the product. Searching for the name of the command-line management tool,
cluster.exe, and the word "heartbeat" targeted against Microsoft's site seems to give a lot of potentially good results (one of the first hits I found has a paragraph containing the phrase "...here each node has an interface on each cluster network, there are N * (N - 1) unicast heartbeats sent per network every 1.2 seconds...", which sounds just like the kind of things you're looking for-- albeit describing Windows Server 2003.)And, of course, if you want to see more of the low levels yourself set it up and throw a sniffer at it.