View
1.528
Download
0
Category
Tags:
Preview:
Citation preview
© jinyh@sjtu
OpenStack@NIC.SJTU
Yaohui Jin (jinyh@sjtu.edu.cn)
Network & Information Center
Shanghai Jiao Tong University
© jinyh@sjtu
About Me and Team
Professor, Deputy Director of NIC.SJTU
Email: jinyh@sjtu.edu.cn
My research interests: Data Center Network, Big Data Analysis, Converged Broadband Network
Team: Engineers: Xuan Luo (Ph. D), Jianwen Wei (M. Eng.), Qiang Sun
(M. Eng.)
Ph.D Students: Jianxiong Tang, Xiaming Chen, Pengfei Zhang, Siwei Qiang
Master Students: Wei Ye, Xin Yang, Xiujie Feng, Xiaosheng Zuo, Zhaohui Zhang
Interns: Hongbo Fan, and other 10+ undergraduate
2
© jinyh@sjtu
Agenda
Hardware configuration
Performance monitoring and measurement
Potential applications
3
© jinyh@sjtu
OpenStack Architecture
courtesy of Dell
4
© jinyh@sjtu
Our Testbed: Sept. 2011
5
© jinyh@sjtu
Testbed Photo
6
© jinyh@sjtu
Server Details
Name Vendor Configuration Purpose
Nova-controller SuperCloud-R6210-S2
2 *E5620/48GB RAM/2*1TB SATA (Raid 1)/GE
Nova-api, Nova-scheduler, Nova-objectstore, RabbitMQ, MySQL, euca2ools, Dashboard, VNC server, Ganglia
Nova-network SuperCloud-R6210-S2
2 *E5620/48GB RAM/2 *1TB SATA (Raid 1)/2 *10GE
Nova-network
Nova-volume IBM x3650 IBM DS3512 + EXP3512
2 *E5620/48GB RAM/2 *146GB SAS (Raid 1)/2 *10GE + 96TB SATA (Raid 10)
Nova-volume
Nova-compute IBM dx360 M3 2 *E5650/96GB RAM/2 *146GB SAS (Raid 1)/2 *10GE
Nova-compute
Glance Dell R610 2 * E5620/8GB RAM/2 * 146GB SAS (Raid 1) /2 *10GE + 320GB SSD
Glance-api, Glance-registry, Image Store, puppet server
Proxy node SuperCloud-R6210-S2
2 * E5620/48GB RAM/2 * 1TB SATA (Raid 1)/2 *10GE
Swauth, Proxy server
Storage node SuperCloud-RE436
2 * E5620/48GB RAM/2 *146GB SAS (Raid 1) /10GE+ 34 * 2TB SATA desktop disks
Account server, Container server, Object server
7
© jinyh@sjtu
Network Details
Data Center Network: 10 GE Switch (BNT&H3C) in 2 domains
Control and Manage: GE Switch (DCRS)
10GE connect to campus network
Fat tree topology; L3: VRRP; L2: LACP+VLAG+MSTP
Security control: SSH, NAT, ACL, VLAN
NIC: Intel X520-DA2; Chelsio T420E-CR
L2-L7 Network tester: IXIA XM2
L2-L3 Network impairment emulator: Apposite Netropy 10G
8
© jinyh@sjtu
Nova Network Traffic
9
courtesy of Vishvananda Ishaya
© jinyh@sjtu
Swift Details
10
Raw storage capacity: 400T Bytes
Storage node configuration: No Raid, JBOD, 3 Replicas, 6 Zones
Hardware cost: ~ 1000 RMB/TB (Raw, including servers and switches)
Collaboration with StorBridge and SkyCloud Shanghai
© jinyh@sjtu
Nova Cluster Monitoring
Monitor by Ganglia
11
© jinyh@sjtu
VM Provisioning Time
VM: Windows7; Image Size: 20GB
12
© jinyh@sjtu
NOVA I/O Throughput
Tested by ATTO
13
© jinyh@sjtu
VM Network Throughput
Co-Located in a single physical machine (CSM)Distributed in multiple physical machines (DMM)
Connected by a single switch (CSS)Connected by multiple switches (CMS)
14
© jinyh@sjtu
Swift Testing
Scalability Adding a storage node to an existing zone
Adding a storage node as a new zone
No influence on the functions of swift
Reliability Disk failure/recovery
Storage node failure/recovery
Fault duration: 10 min & 1 hour
No influence on the functions of swift
Performance testing (ongoing) Throughput
Response time
Concurrency
15
Collaboration with Intel Shanghai
© jinyh@sjtu
Nova Potential Applications
Infrastructure as a service (either private or public)
VM management for DevOps in IT service department
Big data analysis and tools, e.g. noSQL and Map/Reduce
Elastic provisioning of web service, particularly for burst requests of web 2.0 or mobile applications
Next generation high performance computing, virtual cluster provisioning with middleware
16
© jinyh@sjtu
Syslog Analysis
17
RAW mirrored traffic into DPI: ~6Gbits/s
syslog into MongoDB: ~4MBytes/s ( 12000records/s )MongoDB increases ~400GBytes/day
© jinyh@sjtu
MongoDB Components
• Actual data• Needs RAM + Disk IO
• Stores sharding configuration • Stores small amounts of data• Infrequently queried/updated
by MongoS
• Stateless router• Typically run on App Servers
• Can run as Arbiter• No data• Just votes to elect primary
courtesy of 10gen
18
© jinyh@sjtu
MongoDB Dataset Provisioning and Primary Results
Cluster in OpenStack:
1 conf server (2CPU + 8GB MEM + 100GB HDD)
1 mongos (2CPU + 8GB MEM + 10GB HDD)
4 mongod (2CPU + 24GB MEM + 2TB HDD)
NO replication
Both volume size and compute nodes can be dynamically changed
No service interruption, no significant performance degradation when data increases
Primary Performance (To be significantly improved)
Aggregate 10min traffic (~7M records)
• MongoDB Map/Reduce takes less than 4 minutes
Query “time + 5 tuples” in 900M records
• MongoDB returns results in 10 seconds
Target (hopefully not dream )
Query dataset of 30 days, less than 1 sec.
19
© jinyh@sjtu
Swift Potential Applications
Similar to Amazon S3, so there are many potential applications, such as: Dropbox, Slideshare, Netflix,…
Sector related, such as medicine, education, media, …
Korean Telecom Commercial Deployment (CloudScaling)
Lack of monitoring; no quota restriction; less auto deployment
Our testing:
20
© jinyh@sjtu
Acknowledgement
Network & Information Center; State Key Lab of Optical Communication
Intel; IBM/BNT; H3C; Dell; Skycloud; Storbridge; IXIA; Apposite; Netronome; Chelsio; Fusion-IO
OpenStack Community
21
Recommended