Embed Size (px)
Citation preview
Maciej Maciejewski
HARDWARE
2
Persistent memory
3
How does it differ from DRAM
4
How fast is it?
5
OPERATING SYSTEM
6
ACPI NFIT
7
E820
8
What can we do with it?
9
10
PersistentMemory
UserSpace
KernelSpace
Application
Load/Store
MMUMappings
NVDIMM Driver
temporary file
StandardFile API
PM-awareFile System
NVML
Memory
12
Memory - libvmmalloc
13
libvmem
jemalloc
Application
PM
KernelSpace
UserSpace
malloc
vmmalloc
mmap
DRAM
vmem_pool_mallocvmem_pool_create
constructor
temporary file
Memory - libvmem
14
libvmem
jemalloc (3.6.0)
Application
Persistent Memory
KernelSpace
UserSpace
libc(or other)
vmem_malloc
vmem
fallocate/mmap
malloc
DRAM
mmap/sbrk
vmem_create
vmem_pool_mallocmalloc
vmem_pool_create
Memory - memkind
15
Block storage
16
17
Byte persistency
18
Why persistency is an issue?
19
CPU
L1
L2
L3
iMC
NVDIMM
CPU
L1
L2
iMC
NVDIMM
CLFLUSHCLFLUSHOPTCLWB
SFENCE
PCOMMIT
SFENCE
20
Flushing to Persistence
open(…);
mmap(…);
strcpy(pmem, „Hello");
msync(pmem, 6, MS_SYNC);
pmem_persist(pmem, 6);
strcpy(pmem, „Hello, World!");
pmem_persist(pmem, 14);
Crossing the 8-Byte Store
Result?
1. „\0\0\0\0\0\0\0\0\0\0...”
2. „Hello, W\0\0\0\0\0\0...”
3. „\0\0\0\0\0\0\0\0orld!\0”
4. „Hello, \0\0\0\0\0\0\0\0”
5. „Hello, World!\0”crash
nvml Persistent Libraries
libpmem – Basic persistency handling
libpmemblk – Block access to pmem
libpmemlog - Log file on pmem (append-mostly)
libpmemobj - Transactional Object Store on pmem
21
libpmemobj
Object API
Transactional API
22
Applications
23
pm_invaders
24
25
Redis key/value store
26
31
0
5
10
15
20
25
30
32 64 128 256 512 1024 2048 4096 8192
seco
nd
s
object size
Startup time
RDB
AOF
PM
32
0
1000
2000
3000
4000
5000
6000
7000
8000
32 64 128 256 512 1024 2048 4096 8192
DR
AM
all
oca
tio
ns
[MB
]
objects size
DRAM usage
AOF
RDB
PM
33
0
20000
40000
60000
80000
100000
120000
36 43 50 57 64 71 78 85 92 99 106 113 120 127 134 141
op
era
tio
ns
/ se
c
% of OS memory
Running out of DRAM
No Persist
34
0
20000
40000
60000
80000
100000
120000
36 43 50 57 64 71 78 85 92 99 106 113 120 127 134 141
op
era
tio
ns
/ se
c
% of OS memory
Running out of DRAM
No Persist
RDB
35
0
20000
40000
60000
80000
100000
120000
36 43 50 57 64 71 78 85 92 99 106 113 120 127 134 141
op
era
tio
ns
/ se
c
% of OS memory
Running out of DRAM
No Persist
RDB
AOF
36
0
20000
40000
60000
80000
100000
120000
36 43 50 57 64 71 78 85 92 99 106 113 120 127 134 141
op
era
tio
ns
/ se
c
% of OS memory
Running out of DRAM
No Persist
RDB
PM
AOF
37
0
10000
20000
30000
40000
50000
60000
70000
80000
32 64 128 256 512 1024 2048 4096 8192
op
era
tio
ns
/ se
c
objects size
CPU usage
PM 1X
AOF 1X
38
0
10000
20000
30000
40000
50000
60000
70000
80000
32 64 128 256 512 1024 2048 4096 8192
op
era
tio
ns
/ se
c
objects size
CPU usage
PM 1X
PM 4X
AOF 1X
AOF 4X
39
0
10000
20000
30000
40000
50000
60000
70000
80000
32 64 128 256 512 1024 2048 4096 8192
op
era
tio
ns
/ se
c
objects size
CPU usage
PM 1X
PM 4X
PM 6X
PM 10X
AOF 1X
AOF 4X
AOF 10X
Moving even more data
0
100
200
300
400
500
600
Time [s] Memory [MB]
GMRES Sparse Matrix solver
Original Matrix data in PM All in PM
40
Modifying app engine
for (int i=0; i<NUMBER_OF_ITERATIONS; i++) {result = calculateThis(i, result);
}
41
i_pm = 0; //at first runtime of app only
...
...
for (i_pm; i_pm<NUMBER_OF_ITERATIONS; i_pm++) {TX_BEGIN(pool) {
result_pm = calculateThis(i_pm, result_pm);} TX_END
} i_pm = 0;
Possibilities
Choices
Even more fun ...
42
