View
222
Download
0
Category
Tags:
Preview:
Citation preview
CONFIDENTIAL MATERIAL / RESTRICTED ACCESSCONFIDENTIAL MATERIAL / RESTRICTED ACCESS
Shinya Kitajima, Tetsuya Uchiumi, Shinji Kikuchi and Yasuhide MatsumotoSystem Management Lab., System Software Laboratories, FUJITSU LABORATORIES LTD.
Automatic Server Role Identification for Cloud Infrastructure Construction
Copyright 2013 FUJITSU LABORATORIES LIMITED
CONFIDENTIAL MATERIAL / RESTRICTED ACCESSCONFIDENTIAL MATERIAL / RESTRICTED ACCESS
Contents
Background Misconfigurations in settings How to decide the same role servers?
Our method Four rules
Evaluation Accuracy rate Impact of four rules
Conclusion
Copyright 2013 FUJITSU LABORATORIES LIMITED2
CONFIDENTIAL MATERIAL / RESTRICTED ACCESSCONFIDENTIAL MATERIAL / RESTRICTED ACCESS
Contents
Background Misconfigurations in settings How to decide the same role servers?
Our method Four rules
Evaluation Accuracy rate Impact of four rules
Conclusion
Copyright 2013 FUJITSU LABORATORIES LIMITED3
CONFIDENTIAL MATERIAL / RESTRICTED ACCESSCONFIDENTIAL MATERIAL / RESTRICTED ACCESS
Background
Public cloud Resources and infrastructure are put together.
• [Resources] : CPU, Memory, Disk space...• [Infrastructure] : Server, Switch, Network...
Users rent the virtual resources.
Copyright 2013 FUJITSU LABORATORIES LIMITED
VirtualResources
Need to install more resources and infrastructure!!
4
CONFIDENTIAL MATERIAL / RESTRICTED ACCESSCONFIDENTIAL MATERIAL / RESTRICTED ACCESS
Installation of new infrastructure
Copy the configuration settings. Can reduce construction costs.
Copyright 2013 FUJITSU LABORATORIES LIMITED
Existing infrastructure New infrastructure
Settings
Operations manager
ModifyNetworksettings
Have to modify network settings
5
CONFIDENTIAL MATERIAL / RESTRICTED ACCESSCONFIDENTIAL MATERIAL / RESTRICTED ACCESS
Misconfigurations
Sometimes misconfigurations occur. Servers cannot communicate with each other. The new infrastructure does not work properly.
Copyright 2013 FUJITSU LABORATORIES LIMITED
Existing infrastructure New infrastructure
Settings
Operations manager
ModifyNetworksettings
Host name: manager2IP address: 192.168.4.25DNS server: 192.168.44.5Gateway: 192.168.0.240
Host name: manager1IP address: 192.168.0.25DNS server: 192.168.0.5Gateway: 192.168.0.240
Mistyping
Forgot to change
6
CONFIDENTIAL MATERIAL / RESTRICTED ACCESSCONFIDENTIAL MATERIAL / RESTRICTED ACCESS
Misconfiguration detection
Our approach Detect the differences between the communication logs.
• New infrastructure is copy of existing infrastructure.• Communication pattern should also be the same.• [Communication log]
• Source IP address, source port, destination IP address, destination port• Captured by tcpdump
Copyright 2013 FUJITSU LABORATORIES LIMITED
Server AServer B Server C Server A’
Server B’ Server C’Existing infrastructure New infrastructure
Setting of Server B’ is wrong?
7
CONFIDENTIAL MATERIAL / RESTRICTED ACCESSCONFIDENTIAL MATERIAL / RESTRICTED ACCESS
Two types of servers
Management servers Organize cloud computing services
• User information• Dom0 information• Storage information• Network information• Charge information
Dom0 servers Lent their resources as virtual resources to users
• e.g. CPU, memory, storage
Settings of management servers are different from each other.
Copyright 2013 FUJITSU LABORATORIES LIMITED
Dom0 servers
Management servers
Focus only on management servers
8
CONFIDENTIAL MATERIAL / RESTRICTED ACCESSCONFIDENTIAL MATERIAL / RESTRICTED ACCESS
Goal of our research
Determine pairs of servers to compare the communication log.
Copyright 2013 FUJITSU LABORATORIES LIMITED
Existing infrastructure New infrastructure
Can not compare the communication logs.
?
9
CONFIDENTIAL MATERIAL / RESTRICTED ACCESSCONFIDENTIAL MATERIAL / RESTRICTED ACCESS
Goal of our research
Determine pairs of servers to compare the communication log.
Copyright 2013 FUJITSU LABORATORIES LIMITED
Existing infrastructure New infrastructure
Can compare the communication logs.
10
CONFIDENTIAL MATERIAL / RESTRICTED ACCESSCONFIDENTIAL MATERIAL / RESTRICTED ACCESS
Goal of our research
Determine pairs of servers to compare the communication log.
Copyright 2013 FUJITSU LABORATORIES LIMITED
Existing infrastructure New infrastructure
DNS
DNS
MailMail
NTP
NTP yum
yumCMDB
CMDB
Same role server = Same communication logs
11
CONFIDENTIAL MATERIAL / RESTRICTED ACCESSCONFIDENTIAL MATERIAL / RESTRICTED ACCESS
Applying Scene
Copyright 2013 FUJITSU LABORATORIES LIMITED
Plan Construc-tion
Function Test
Operation Test Operation
Construction phase Operation phase
Scene to apply our method
Function test after construction Another, function test after update or improvement
12
CONFIDENTIAL MATERIAL / RESTRICTED ACCESSCONFIDENTIAL MATERIAL / RESTRICTED ACCESS
Motivation
You may think ...
However, A data center continues to change.
• The actual structure of data center changes from a plan gradually.• New function, new machine, fix problems, etc.
Constructers change a plan.• They often do not install a new data center according to plan.
• Misconfigurations, unreasonable plan, etc.
Copyright 2013 FUJITSU LABORATORIES LIMITED
If there is a plan, it is not necessary to determine
the same role server by using technique.
A plan is only a plan.
13
CONFIDENTIAL MATERIAL / RESTRICTED ACCESSCONFIDENTIAL MATERIAL / RESTRICTED ACCESS
Difficulty of this problem
It is difficult to know servers’ role from their appearances.
The configurations of servers in both data center is not completely the same.
Copyright 2013 FUJITSU LABORATORIES LIMITED
Very similar
Very similar
What?
Which?
14
CONFIDENTIAL MATERIAL / RESTRICTED ACCESSCONFIDENTIAL MATERIAL / RESTRICTED ACCESS
Automatic identification
To use communication logs is easy and make sense. We can also detect misconfigurations from communication logs. Communication logs influenced by misconfigurations.
• Consider the differences between the communication logs.
Copyright 2013 FUJITSU LABORATORIES LIMITED
Communicationlogs
List of the samerole servers
Detectmisconfigurations
15
CONFIDENTIAL MATERIAL / RESTRICTED ACCESSCONFIDENTIAL MATERIAL / RESTRICTED ACCESS
Contents
Background Misconfigurations in settings How to decide the same role servers?
Our method Four rules
Evaluation Accuracy rate Impact of four rules
Conclusion
Copyright 2013 FUJITSU LABORATORIES LIMITED16
CONFIDENTIAL MATERIAL / RESTRICTED ACCESSCONFIDENTIAL MATERIAL / RESTRICTED ACCESS
Compare communication logs
Summary of our method
Copyright 2013 FUJITSU LABORATORIES LIMITED
Assumption: configurations are almost the same.
Compare communication logs
Can observe almost the same communication logs.
UniquePortRule
Corre-spondingSources
Rule
RemainingUnique
PortRule
CommonPortsRule
Communication logs
Our method(Four rules)
IdentificationExisting data center
New data center17
CONFIDENTIAL MATERIAL / RESTRICTED ACCESSCONFIDENTIAL MATERIAL / RESTRICTED ACCESS
Existing data center
Basic idea of our method
Same role server have the same listening ports
Copyright 2013 FUJITSU LABORATORIES LIMITED
Send packets
NTP server192.168.1.3
Listening port for NTPPort number : 123
New data center
Send packets
NTP server192.168.5.3
IP 192.168.1.13.53746 > 192.168.1.3.123
Listening port for NTPPort number : 123
NTP client192.168.1.13
NTP client192.168.5.13
IP 192.168.5.13.52131 > 192.168.5.3.123
Communication log Communication log
If the listening ports are the same,we can assume that those server have the
same role.We call these servers as the corresponding servers.
18
CONFIDENTIAL MATERIAL / RESTRICTED ACCESSCONFIDENTIAL MATERIAL / RESTRICTED ACCESS
Rule 1 : Unique port rule
Focus on the unique listening port. Used by only one pair of servers.
• These pairs are the corresponding servers.
Copyright 2013 FUJITSU LABORATORIES LIMITED
Existing data center New data center
254438080
252952
90049004
258080 252952
9004
: Listening port number
254438080
251238080
9004
254438080
258080
19
CONFIDENTIAL MATERIAL / RESTRICTED ACCESSCONFIDENTIAL MATERIAL / RESTRICTED ACCESS
Rule 1 : Unique port rule
Focus on the unique listening port. Used by only one pair of servers.
• These pairs are the corresponding servers.
Copyright 2013 FUJITSU LABORATORIES LIMITED
Existing data center New data center
254438080
252952
9004
258080 252952
9004
: Listening port number
9004
254438080
251238080
9004
254438080
258080
Unique listening port
20
CONFIDENTIAL MATERIAL / RESTRICTED ACCESSCONFIDENTIAL MATERIAL / RESTRICTED ACCESS
Rule 1 : Unique port rule
Focus on the unique listening port. Used by only one pair of servers.
• These pairs are the corresponding servers.
Copyright 2013 FUJITSU LABORATORIES LIMITED
Existing data center New data center
254438080
252952
9004
258080 252952
9004
: Corresponding servers
: Listening port number
9004
254438080
251238080
9004
254438080
258080
Corresponding servers
21
CONFIDENTIAL MATERIAL / RESTRICTED ACCESSCONFIDENTIAL MATERIAL / RESTRICTED ACCESS
Rule 1 : Unique port rule
Focus on the unique listening port. Used by only one pair of servers.
• These pairs are the corresponding servers.
Copyright 2013 FUJITSU LABORATORIES LIMITED
Existing data center New data center
254438080
252952
9004
258080 252952
9004
: Corresponding servers
: Listening port number
9004
254438080
251238080
9004
254438080
258080
Corresponding serversCorresponding servers
22
CONFIDENTIAL MATERIAL / RESTRICTED ACCESSCONFIDENTIAL MATERIAL / RESTRICTED ACCESS
Rule 2 : Corresponding sources rule
Focus on the servers where the source servers are the corresponding servers. These pairs are the corresponding servers.
Copyright 2013 FUJITSU LABORATORIES LIMITED
Existing data center New data center
254438080
252952
9004
258080 252952
9004
: Corresponding servers
: Listening port number
9004
254438080
251238080
9004
254438080
258080
23
CONFIDENTIAL MATERIAL / RESTRICTED ACCESSCONFIDENTIAL MATERIAL / RESTRICTED ACCESS
Rule 2 : Corresponding sources rule
Focus on the servers where the source servers are the corresponding servers. These pairs are the corresponding servers.
Copyright 2013 FUJITSU LABORATORIES LIMITED
Existing data center New data center
254438080
252952
9004
258080 252952
9004
: Corresponding servers
: Listening port number
9004
254438080
251238080
9004
254438080
258080
Corresponding serversSource servers are the corresponding servers
24
CONFIDENTIAL MATERIAL / RESTRICTED ACCESSCONFIDENTIAL MATERIAL / RESTRICTED ACCESS
Rule 2 : Corresponding sources rule
Focus on the servers where the source servers are the corresponding servers. These pairs are the corresponding servers.
Copyright 2013 FUJITSU LABORATORIES LIMITED
Existing data center New data center
254438080
252952
9004
258080 252952
9004
: Corresponding servers
: Listening port number
9004
254438080
251238080
9004
254438080
258080
Corresponding serversCorresponding servers
25
CONFIDENTIAL MATERIAL / RESTRICTED ACCESSCONFIDENTIAL MATERIAL / RESTRICTED ACCESS
Rule 3 : Remaining unique port rule
Focus on the unique listening port again. Ignore the listening ports used by the corresponding servers. These pairs are the corresponding servers.
Copyright 2013 FUJITSU LABORATORIES LIMITED
Existing data center New data center
254438080
252952
9004
258080 252952
90049004
254438080
251238080
9004
254438080
258080
: Corresponding servers
: Listening port number
26
CONFIDENTIAL MATERIAL / RESTRICTED ACCESSCONFIDENTIAL MATERIAL / RESTRICTED ACCESS
Rule 3 : Remaining unique port rule
Focus on the unique listening port again. Ignore the listening ports used by the corresponding server. These pairs are the corresponding servers.
Copyright 2013 FUJITSU LABORATORIES LIMITED
Existing data center New data center
254438080
252952
9004
258080 252952
9004
: Corresponding servers
: Listening port number
9004
254438080
251238080
9004
254438080
258080
27
CONFIDENTIAL MATERIAL / RESTRICTED ACCESSCONFIDENTIAL MATERIAL / RESTRICTED ACCESS
Rule 3 : Remaining unique port rule
Focus on the unique listening port again. Ignore the listening ports used by the corresponding server. These pairs are the corresponding servers.
Copyright 2013 FUJITSU LABORATORIES LIMITED
Existing data center New data center
254438080
252952
9004
258080 252952
9004
: Corresponding servers
: Listening port number
9004
254438080
251238080
9004
254438080
258080
Corresponding serversUnique listening port
28
CONFIDENTIAL MATERIAL / RESTRICTED ACCESSCONFIDENTIAL MATERIAL / RESTRICTED ACCESS
Rule 3 : Remaining unique port rule
Focus on the unique listening port again. Ignore the listening ports used by the corresponding server. These pairs are the corresponding servers.
Copyright 2013 FUJITSU LABORATORIES LIMITED
Existing data center New data center
254438080
252952
9004
258080 252952
9004
: Corresponding servers
: Listening port number
9004
254438080
251238080
9004
254438080
258080
Corresponding serversCorresponding serversCorresponding serversCorresponding servers
29
CONFIDENTIAL MATERIAL / RESTRICTED ACCESSCONFIDENTIAL MATERIAL / RESTRICTED ACCESS
Rule 4 : Common ports rule
Focus on the servers where the source servers are the corresponding servers again. Several servers are candidates for the corresponding servers. The servers that have most common listening ports as the
corresponding servers.
Copyright 2013 FUJITSU LABORATORIES LIMITED
Existing data center New data center
254438080
252952
9004
258080 252952
9004
: Corresponding servers
: Listening port number
9004
254438080
251238080
9004
254438080
258080
30
CONFIDENTIAL MATERIAL / RESTRICTED ACCESSCONFIDENTIAL MATERIAL / RESTRICTED ACCESS
Rule 4 : Common ports rule
Focus on the servers where the source servers are the corresponding servers again. Several servers are candidates for the corresponding servers. The servers that have most common listening ports as the
corresponding servers.
Copyright 2013 FUJITSU LABORATORIES LIMITED
Existing data center New data center
254438080
252952
9004
258080 252952
9004
: Corresponding servers
: Listening port number
9004
254438080
251238080
9004
254438080
258080
Corresponding serversCorresponding serversCorresponding serversSource servers are the corresponding servers
31
CONFIDENTIAL MATERIAL / RESTRICTED ACCESSCONFIDENTIAL MATERIAL / RESTRICTED ACCESS
Rule 4 : Common ports rule
Focus on the servers where the source servers are the corresponding servers again. Several servers are candidates for the corresponding servers. The servers that have most common ports as the corresponding servers.
Coincident rate : Co Represents the degree of similarity of the listening ports.
• : The number of common listening ports• : The number of listening ports of server a• : The number of listening ports of server b
Copyright 2013 FUJITSU LABORATORIES LIMITED32
CONFIDENTIAL MATERIAL / RESTRICTED ACCESSCONFIDENTIAL MATERIAL / RESTRICTED ACCESS
Rule 4 : Common ports rule
Focus on the servers where the source servers are the corresponding servers again. Several servers are candidates for the corresponding servers. The servers that have most common listening ports as the
corresponding servers.
Copyright 2013 FUJITSU LABORATORIES LIMITED
Existing data center New data center
254438080
252952
9004
258080 252952
9004
: Corresponding servers
: Listening port number
9004
254438080
251238080
9004
254438080
2580801
0.830.670.83
33
CONFIDENTIAL MATERIAL / RESTRICTED ACCESSCONFIDENTIAL MATERIAL / RESTRICTED ACCESS
Rule 4 : Common ports rule
Focus on the servers where the source servers are the corresponding servers again. Several servers are candidates for the corresponding servers. The servers that have most common listening ports as the
corresponding servers.
Copyright 2013 FUJITSU LABORATORIES LIMITED
Existing data center New data center
254438080
252952
9004
258080 252952
9004
: Corresponding servers
: Listening port number
9004
254438080
251238080
9004
254438080
258080
Corresponding serversCorresponding servers
34
CONFIDENTIAL MATERIAL / RESTRICTED ACCESSCONFIDENTIAL MATERIAL / RESTRICTED ACCESS
Identification of remaining servers
Apply the corresponding source rule again.
Copyright 2013 FUJITSU LABORATORIES LIMITED
Existing data center New data center
254438080
252952
9004
258080 252952
9004
: Corresponding servers
: Listening port number
9004
254438080
251238080
9004
254438080
258080
35
CONFIDENTIAL MATERIAL / RESTRICTED ACCESSCONFIDENTIAL MATERIAL / RESTRICTED ACCESS
Identification of remaining servers
Apply the corresponding source rule again.
Copyright 2013 FUJITSU LABORATORIES LIMITED
Existing data center New data center
254438080
252952
9004
258080 252952
9004
: Corresponding servers
: Listening port number
9004
254438080
251238080
9004
254438080
258080
Corresponding serversCorresponding servers
36
CONFIDENTIAL MATERIAL / RESTRICTED ACCESSCONFIDENTIAL MATERIAL / RESTRICTED ACCESS
Identification of remaining servers
Apply the remaining unique port rule again.
Copyright 2013 FUJITSU LABORATORIES LIMITED
Existing data center New data center
254438080
252952
9004
258080 252952
9004
: Corresponding servers
: Listening port number
9004
254438080
251238080
9004
254438080
258080
37
CONFIDENTIAL MATERIAL / RESTRICTED ACCESSCONFIDENTIAL MATERIAL / RESTRICTED ACCESS
Identification of remaining servers
Apply the remaining unique port rule again. Finally identify all servers.
Copyright 2013 FUJITSU LABORATORIES LIMITED
Existing data center New data center
254438080
252952
9004
258080 252952
9004
: Corresponding servers
: Listening port number
9004
254438080
251238080
9004
254438080
258080
Corresponding serversCorresponding serversCorresponding serversCorresponding servers
38
CONFIDENTIAL MATERIAL / RESTRICTED ACCESSCONFIDENTIAL MATERIAL / RESTRICTED ACCESS
Contents
Background Misconfigurations in settings How to decide the same role servers?
Our method Four rules
Evaluation Accuracy rate Impact of four rules
Conclusion
Copyright 2013 FUJITSU LABORATORIES LIMITED39
CONFIDENTIAL MATERIAL / RESTRICTED ACCESSCONFIDENTIAL MATERIAL / RESTRICTED ACCESS
Evaluation environment
Two small experimental cloud data centers Actual data center in our laboratory
• Management servers : 39• Dom0 servers
• Ignore the communication logs
Recorded period• One and a half day• Enough to obtain almost all types of communication logs
Copyright 2013 FUJITSU LABORATORIES LIMITED
Existing data center New data center
Almost the sameconfiguration
40
CONFIDENTIAL MATERIAL / RESTRICTED ACCESSCONFIDENTIAL MATERIAL / RESTRICTED ACCESS
Evaluation contents
Accuracy rate : Ac How precisely our method can estimate the corresponding servers.
• : The set of correct pairs manually identified• : The set of estimated pairs identified by our method
Comparing methods
Copyright 2013 FUJITSU LABORATORIES LIMITED
Unique port rule ✔ ✔ ✔ ✔
Corresponding sources rule ✔ ✔ ✔
Remaining unique port rule ✔ ✔
Common ports rule ✔
41
CONFIDENTIAL MATERIAL / RESTRICTED ACCESSCONFIDENTIAL MATERIAL / RESTRICTED ACCESS
Evaluation result
Results of accuracy rate
Copyright 2013 FUJITSU LABORATORIES LIMITED
Contribution to accuracy rate
Repeatedly Applied
Contain wrong answer
Unique port rule Middle
Corresponding sources rule Small
Remaining unique port rule High ✔
Common ports rule Middle ✔ ✔
42
CONFIDENTIAL MATERIAL / RESTRICTED ACCESSCONFIDENTIAL MATERIAL / RESTRICTED ACCESS
Conclusion
Automatically identifies servers that have the same role. By comparing the communication logs. The accuracy rate is 94.1%.
[Future works] Deal with the following cases
• The number of servers is different.• The components working on servers is different.
Propose a new misconfigurations detection method.• By comparing communication logs.• Use the corresponding servers list according to our method.
Copyright 2013 FUJITSU LABORATORIES LIMITED43
CONFIDENTIAL MATERIAL / RESTRICTED ACCESSCONFIDENTIAL MATERIAL / RESTRICTED ACCESS Copyright 2013 FUJITSU LABORATORIES LIMITED44
Recommended