Download presentation
Presentation is loading. Please wait.
1
Life of a Sharded Write by Randolph Tan
2
Sharding Concepts
3
Sharding concepts Shard0 Shard1 Shard2
4
Sharding concepts [minKey, -200) Shard0 Shard1 Shard2
5
Sharding concepts [minKey, -200) [-200, -100) Shard0 Shard1 Shard2
6
Sharding concepts Shard0 Shard1 Shard2 [minKey, -200) [-200, -100)
[-100, 0) Shard0 Shard1 Shard2
7
Sharding concepts Shard0 Shard1 Shard2 [minKey, -200) [-200, -100)
[0, 100) [-200, -100) [-100, 0) Shard0 Shard1 Shard2
8
Sharding concepts Shard0 Shard1 Shard2 [100, 200) [minKey, -200)
[0, 100) [100, 200) [-200, -100) [-100, 0) Shard0 Shard1 Shard2
9
Sharding concepts Shard0 Shard1 Shard2 [100, 200) [minKey, -200)
[0, 100) [200, maxKey) [100, 200) [-200, -100) [-100, 0) Shard0 Shard1 Shard2
10
Routing table min max owner MinKey -200 shard0 -100 shard2 100 200
100 200 shard1 MaxKey
11
Sharded Cluster Architecture
shards mongos/ router sh0 Client App sh2 sh1 config servers
12
Sharded Single Doc Write
db.foo.insert({ x: 1 }); sh0 shard0 mongos sh1 shard1 config servers
13
Sharded Single Doc Write
Get routing table mongos sh1 shard1 config servers
14
Routing table db.foo.insert({ x: 1 }); min max owner MinKey -200
shard0 -100 shard2 100 200 shard1 MaxKey
15
Sharded Single Doc Write
insert({ x: 1 }); shard0 mongos sh1 shard1 config servers
16
Chunk Migration Shard0 Shard1 Shard2 [100, 200) [minKey, -200)
[0, 100) [200, maxKey) [100, 200) [-200, -100) [-100, 0) Shard0 Shard1 Shard2
17
Chunk Migration Shard0 Shard1 Shard2 [100, 200) [minKey, -200)
[0, 100) [200, maxKey) [100, 200) [0, 100) [-200, -100) [-100, 0) Shard0 Shard1 Shard2
18
Sharding Version Protocol
min max owner 1 MinKey -200 shard0 2 -100 shard2 3 4 100 5 200 shard1 6 MaxKey
19
Sharding Version Protocol
nextVersion = max(routingTableVersions) + 1
20
Chunk Migration Shard0 Shard1 Shard2 [100, 200) [minKey, -200)
[0, 100) [200, maxKey) [100, 200) [0, 100) [-200, -100) [-100, 0) Shard0 Shard1 Shard2
21
Sharding Version Protocol
min max owner 1 MinKey -200 shard0 2 -100 shard2 3 4 -> 7 100 shard0 -> shard1 5 200 shard1 6 MaxKey
22
Sharded Single Doc Write
db.foo.insert({ x: 1 }); shard0 w/ routing table v7 sh0 mongos shard1 w/ routing table v7 sh1 w/ routing table v6 config servers w/ routing table v7
23
Sharding Version Protocol
min max owner 1 MinKey -200 shard0 2 -100 shard2 3 4 100 5 200 shard1 6 MaxKey
24
Sharded Single Doc Write
w/ routing table v7 insert({ x: 1 }); @ v6 sh0 mongos shard1 w/ routing table v7 sh1 w/ routing table v6 config servers w/ routing table v7
25
Sharded Single Doc Write
v6 != v7 shard0 w/ routing table v7 sh0 mongos shard1 w/ routing table v7 sh1 w/ routing table v6 config servers w/ routing table v7
26
Sharded Single Doc Write
w/ routing table v7 sh0 update routing table mongos shard1 w/ routing table v7 sh1 w/ routing table v6 config servers w/ routing table v7
27
Sharding Version Protocol
min max owner 1 MinKey -200 shard0 2 -100 shard2 3 7 100 shard1 5 200 6 MaxKey
28
Sharded Single Doc Write
w/ routing table v7 sh0 insert({ x: 1 }); @v7 mongos shard1 w/ routing table v7 sh1 w/ routing table v7 config servers w/ routing table v7
29
Mirrored Config Servers
mongos
30
Mirrored Config Servers
update x mongos
31
Mirrored Config Servers
1 update x mongos
32
Mirrored Config Servers
update x mongos 2
33
Mirrored Config Servers
update x mongos 3
34
Mirrored Config Servers
update x mongos
35
Mirrored Config Servers
mongos
36
Mirrored Config Servers
mongos
37
CSRS (Config Server as Replica Sets)
Config servers mongos
38
Mirrored Config vs CSRS Recap
Lost write inconsistency
39
Mirrored Config vs CSRS Recap
Lost write inconsistency Single source of truth
40
Mirrored Config vs CSRS Recap
Lost write inconsistency Single source of truth Read only when special server dies
41
Mirrored Config vs CSRS Recap
Lost write inconsistency Single source of truth Read only when special server dies New primary gets elected
42
Mirrored Config vs CSRS Recap
Lost write inconsistency Single source of truth Read only when special server dies New primary gets elected Complicated distributed lock
43
Mirrored Config vs CSRS Recap
Lost write inconsistency Single source of truth Read only when special server dies New primary gets elected Complicated distributed lock Single server maintains lock
44
CSRS Challenges Possible to read data that may roll back Data on secondaries may be too stale
45
Rollback Example rTable v6 [0, 100) -> sh0 [-100, 0) -> sh2 P
46
Rollback Example rTable v7 [0, 100) -> sh0 [-100, 0) -> sh2 sh1
47
Rollback Example rTable v7 [0, 100) -> sh0 [-100, 0) -> sh1 P
48
Rollback Example rTable v7 [0, 100) -> sh0 [-100, 0) -> sh1
49
Rollback Example rTable v7 [0, 100) -> sh0 [-100, 0) -> sh1
50
Rollback Example rTable v7 [0, 100) -> sh0 [-100, 0) -> sh1
51
Rollback Example rTable v7 [0, 100) -> sh0 [-100, 0) -> sh1 R
52
Rollback Example rTable v6 [0, 100) -> sh0 [-100, 0) -> sh1 sh2
53
Rollback Example rTable v6 [0, 100) -> sh0 [-100, 0) -> sh2 S
54
Rollback Example rTable v7 [0, 100) -> sh1 [-100, 0) -> sh2 S
55
Rollback Example rTable v7 [0, 100) -> sh0 [-100, 0) -> sh1
56
New Feature db.runCommand({ find: ‘foo’, filter: { x: 1 }, readConcern: { level: ‘majority’ } });
57
Rollback Example Commited view rTable v6 [0, 100) -> sh0
58
CSRS Challenges Possible to read data that may roll back readConcern majority Data on secondaries may be too stale
59
Stale Reads Example v6 != v7 sh0 rTable v7 mongos rTable v6 sh1
Resume practice here rTable v6
60
Stale Reads Example sh0 rTable v7 mongos rTable v6 sh1 update routing
Already up to date??? rTable v7 rTable v6
61
New Feature (internal use only)
db.runCommand({ find: ‘foo’, filter: { x: 1 }, readConcern: { level: ‘majority’, afterOpTime: <opTime> }});
62
Read after OpTime Example
rTable v6, opTime t9 sh0 rTable v7 mongos rTable v6 sh1 rTable v7 rTable v7 rTable v6
63
Read after OpTime Example
rTable v7, opTime t11 sh0 rTable v7 mongos rTable v6 sh1 rTable v7 rTable v7 rTable v6
64
Read after OpTime Example
sh0 rTable v7 mongos sh1 rTable v7 Wait until t >= t11; update routing table rTable v7 rTable v6
65
Read after OpTime Example
sh0 rTable v7 mongos sh1 rTable v7 Wait until t >= t11; update routing table rTable v7 rTable v7
66
Sharding Version Protocol
min max owner 1 MinKey -200 shard0 2 -100 shard2 3 7 100 shard1 5 200 6 MaxKey
67
Read after OpTime Example
sh0 rTable v7 insert({ x: 1 }); mongos sh1 rTable v7 rTable v7 rTable v7
68
CSRS Challenges Possible to read data that may roll back readConcern majority Data on secondaries may be too stale readConcern afterOpTime
69
New in v3.2 Config servers can be replica sets (required in v3.4) New feature: readConcern
70
Future Possibilities afterOpTime functionality outside of internal code
71
Further reading read concern replica set rollback sharding concepts
72
Questions?
73
International Offices
Market Size $36 Billion International Offices 15 Partners 1,000+ Global Employees 575+ Downloads Worldwide 15,000,000+ Make a GIANT Impact
Similar presentations
