Replication
InternalsFitting Everything Together
2.8, Refactored
● Architecture as of 2.8
● Unit testable; more, and faster, cpp tests
● Many changes (heartbeats, locking, future)
● Interop with 2.6
● Larger replica sets
Large Blocks
● Topology Manager (state machine)
● Replication Coordinator (repl facade)
● Applier (replicate/apply oplog)
● Executor (network, heartbeats, serialization)
● Commands (re-config, init, status, etc)
● External (writes, storage, query, commands)
Blocks
Applier
Topology Manager
Replication
Coordinator
Oplog
CFG
CM
Ds
Write
s
Qu
ery
Executo
r
Blocks
Applier
Topology Manager
Replication
Coordinator
Oplog
CFG
CM
Ds
Write
s
Qu
ery
Executo
r
Topology
● Maintains Authoritative Stateo Heartbeat, ping, member state
o Roles and transitions
● Contains Decision Logic
● Unit Testable
● Serial AccessTopology Manager
CFG
Examples
● updateConfig
● prepare*Response for commands
● getSyncSource, *
● setFollowerMode (state)
● processHeartbeat
● prepareHeartbeatResponse
PrepareHeartbeatResponseStatus TopologyCoordinatorImpl::prepareHeartbeatResponse(...) {
// Check error conditions, then set response fields …
response->setElectable(!_getMyUnelectableReason(...));
response->setHbMsg(_getHbmsg(...));
response->setTime(...);
response->setOpTime(lastOpApplied);
if (!_syncSource) {
response->setSyncingTo(_syncSource); }
… topology_coordinator_impl.cpp:628
Failover Scenario
Heart
beats P
S
S
Health Check (rsHB)Active Primary
Failover Scenario
Heart
beats P
S
S
Active PrimaryP
Failed Primary
Failover Scenario
Heart
beats Failed
P
S
Health Check (rsHB)
Blocks
Applier
Topology Manager
Replication
Coordinator
Oplog
CFG
CM
Ds
Write
s
Qu
ery
Executo
r
Replications Coordinator
● Interface to other subsystems
● Uses executor to scheduleo Commands
o Elections, Initiate, Reconfig
o Role/State Changes
● Unit Testableo With help, requires mocking out bridge for
subsystems
Replication
Coordinator
Blocks
ApplierReplication
Coordinator
OplogC
MD
s
Write
s
Qu
ery
Executo
r
Topology ManagerCFG
Examples
● process*Response for commands
● awaitReplication* (for writes or migration)
● isReplEnabled
● canAcceptWrites*
Accepting writesstatic bool checkIsMasterForDatabase(const std::string& db, ...) {
if (!getReplicationCoordinator()->canAcceptWritesForDatabase(db)){
errorDetail->setErrCode(ErrorCodes::NotMaster);
errorDetail->setErrMessage("Not primary while writing to " + ns);
return false;
}
return true;
}
Blocks
Applier
Topology Manager
Replication
Coordinator
Oplog
CFG
CM
Ds
Write
s
Qu
ery
Executo
r
Applier
● Reads from *upstream* oplog
● Applier operations transformations
● Mostly unchanged since 2.4
● Includes UpdatePosition commands
Applier
Read + Apply Decoupled
● Background oplog reader thread (net)
● Pool of oplog applier threads (by collection)
Repl Source
Applier
Pool
Buffer
DB4
DB3
DB1 DB2
Local Oplog
Network
Replication Operations
oplog entry (fields):
o = update, o2 = query
{ "ns" : "test.tags",
"op" : "u", "v" : 2, "ts": ...,
"o2" : { "_id" : 1 },
"o" : { "$set" : { "tags.4" : "e" } } }
Blocks
Applier
Topology Manager
Replication
Coordinator
Oplog
CFG
CM
Ds
Write
s
Qu
ery
Executo
r
Executor
● Serializes access to Topology state
● Serializes global state changes wrt db writes
● Processes network requests in IO pool
● Supports event/signal notification
Write Request
● Sent by user
● Interpreted by command subsystem
● Checked by replication coordinator
● Executed
● Idempotent entry recorded in oplog
● ~ Replicated
● ~ Possibly verified during user write request
Write Request
ApplierReplication
Coordinator
OplogC
MD
s
Write
s
Qu
ery
Executo
r
Topology ManagerCFG
● Topology Manager (state machine)
● Replication Coordinator (repl facade)
● Applier (replicate/apply oplog)
● Executor (network, heartbeats, serialization)
● Commands (re-config, init, status, etc)
● External (writes, storage, query, commands)
Thanks
Questions?
Recommended
