Embed Size (px)
Citation preview
Systems in System Groupon the Web
Jinyu Zhang Feb. 2008
To Resume…..
• Cross layers Architecture for Internet based Distributed system, Overlay in other words.
• Security should always be emphasized• Very large and famous System is invaluable• DHT is still hot• In order to publish a paper In System areas, Topic is
not as crucial as in other fields. Solve a realistic problem and solve it well are more important!
• They can do architecture, but we can not!• MIT is my favorite style, fundamental and full-fledged,
all deliverable!
MIT
• Name:Parallel&Distributed Operating System Group• Core Persons:
– M. Frans Kaashoek(Student of Andy Tanenbaum)» Phd. Thesis: Group communication in distributed
computer systems– Robert Morris
• Active Projects: currently Focus around DHT– CHORD >>> IRIS: The ultimate P2P network-to provide a
novel decentralized infrastructure– Click: Modular software router, a new software architecture
for building flexible and configurable routers– SFS: Self-Certifying File System– ‘C (dynamic c code generation) and Prolac(specially for
protocol implementation): Programming Language(Both completed)
– Exokernel: Operating System(completed)
MIT(more)
• Name: Networks and Mobile Systems• Core persons:
– Hari Balakrishnan, John Guttag , Dina Katabi
• Active Projects: Belongs to IRIS• SFR, Semantic-Free Referencing
– Take place existing DNS+URL reference• DOA, Delegation-Oriented Architecture
– Senders and receivers direct traffic through intermediaries with a delegation primitive
• XCP, take place TCP when product of bandwidth and latency is high
• Completed Projects:• RON, to improve the reliability and resilience of packet routing in
the wide-area Internet• DNS analysis, analysis of DNS performance and caching
– Sigcomm 2001, Inforcomm 2003 !!!!• Securing SSH, known_hosts file
Stanford
• Name: Distributed Systems Group• Core Persons: David Cheriton• Active Projects:
– Defense against Distributed Denial of Service Attacks(2005)
– Extreme Scale Cluster Architecture (2005)– Feedback Based Inter-domain Routing (2005)– History-based Anti-spam (2005)– KIWI Project - Towards Self-Managed Wireless
LANs (2005)– TRIAD - Translating Relaying Internet Architecture
integrating Active Directories (Content routing 2001)
UC Berkeley
• Name:Reliable Adaptive Distributed Systems Lab• Core Persons:
– Armando Fox Michael Jordan, Anthony Joseph, Randy Katz, David Patterson(President of ACM 2004-2006), Scott Shenker and Ion Stoica
• Active Project– Trying to Figure Out How to Put a Google In Every Data
Center.• Portals of The WallStreet Journal Feb 20, 2008
– To enable one person to invent and run the next revolution IT service, operationally expression a new business as a multi-million-user service over the course of a long weekend!!!
UC Berkeley
• Develop: to leverage highly sophisticated existing building blocks. HousingMaps.com=Craiglist+Google Map API DHT Middleware
• Access: For large scale and heavy usage systems, to apply machine learning and statistical learning theory(SML)."Statistical Debugging". without special-case checks. Relative lack of observation and control points makes it difficult to access the behavior of wide-area network. To deploy a new type of network element iBox
• To deploy iBox relay on A delegation and indirection mechanisms that allow both the sender and the receiver to specify explicitly iBoxes along the data path
• Operate: Middleware. Besides easing development by allowing the creation of reusable components, Middleware serves to separate the operational concerns of monitoring and recovery from the mainline application logic
UC Berkeley
– The X-Trace Network Tracing Framework• A network diagnostic tool designed to provide users and
network operators with better visibility into increasingly complex Internet applications. It does this by annotating network requests with metadata that can be used to reconstruct requests
– Network Annotations• Lack of some form of signalling between different nodes along
the path, as well as across different layers of the network stack
– D-Trigger: A general framework for efficient online detection• Distributed monitoring and anomaly detection systems
– Monitoring the Operators of Internet Services• Use various kinds of tools to record and learn how operators
resolved any of the problems in the past.
UC Berkeley– Internet in a Box
• using a multiboard FPGA based system, called RAMP (the acronym for Research Accelerator for Multiple Processors), to build a reconfigurable testbed to accelerate the development of distribute systems
– Applying control theory in flow control of software system– Replay Debugging for Distributed Applications– OCALA: An Architecture for Supporting Legacy Applications over Overlays
• To imposing an Overlay Convergence (OC) layer below the transport layer in the IP stack to leverage functionality(Caching, Load balance or Content Routing) of various of Overlay without any modifications to the existing applications and operating systems
– A Policy-aware Switching Layer for Data Centers– Adaptive Power Management– Workload characterization and generation– DONA: Data Oriented Network Architecture
• explores a clean-slate data-centric approach to Internet architecture• Data Handlers (DHs), which combine the functions of name resolution and data
caching. Collectively, DHs assume the responsibility for routing clients' requests to nearby copies of the data.
– Learning in Security Sensitive Environments• To study the security issue of machine learning mechanism
– Monitoring Hadoop using X-Trace
CMU• Name: Computer System Research
– CALCM(Computer Architecture Group)– Database Group– Parallel Data Lab– CMCL(Computing Media and Communication Lab)
• emphasis on the interaction of networks and applications– How can the network manage its resources so that it best
supports the application? (Application-aware networks)– How can we construct applications so that they can adjust their
resource demands in response to changes in the network(Network-aware Application)
• Understanding Network-Friendliness of Peer-to-Peer Application
– Mobile Computing Lab• AFS• Coda• Odyssey• Coda and Odyssey are building blocks of Aura , a new flagship
project focusing on distraction-free ubiquitous computing
清华
SOSP2007
• Web meets operating systems(3)– AjaxScope: a platform for remotely monitoring the client-side
behavior of web 2.0 applications• AjaxScope is a proxy that performs on-the-fly parsing and
instrumentation of JavaScript code as it is sent to users' browsers
• Byzantine fault tolerance(3)• Concurrency(2)• Software robustness(3)
– /*icomment: bugs or bad comments?*/
• Distributed systems(4)– Sinfonia: a new paradigm for building scalable distributed
systems.• Distributed Memory vs Messaging• a novel minitransaction primitive that enables efficient and
consistent access to data, while hiding the complexities that arise from concurrency and failures
SOSP2007
– PeerReview: practical accountability for distributed systems.• PeerReview ensures that Byzantine faults whose effects are
observed by a correct node are eventually detected and irrefutably linked to a faulty node. At the same time, PeerReview ensures that a correct node can always defend itself against false accusations
– Attested append-only memory: making adversaries stick to their word.
• A2M practically improve Byzantine tolerance from 1/3 to 1/2
– Dynamo: amazon's highly available key-value store• on top of an infrastructure of tens of thousands of servers and
network components located in many datacenters around the world
• To achieve this level of availability, Dynamo sacrifices consistency under certain failure scenarios. It makes extensive use of object versioning and application-assisted conflict resolution
SOSP2007
• System maintenance(2)– Staged deployment in mirage, an integrated software
upgrade testing and distribution system.– AutoBash: improving configuration management with
operating system causality analysis.
• Energy(2)– Integrating concurrency control and energy management in
device drivers.– VirtualPower: coordinated power management in virtualized
enterprise systems.
• Storage(3)– DejaView: a personal virtual computer recorder.– Improving file system reliability with I/O shepherding.– Generalized file system dependencies.
• Operating system security(3)
OSDI2006
• Distributed System Infrastructure– Bigtable: A Distributed Storage System for Structured
Data （ Awarded Best Paper)– HQ Replication: A Hybrid Quorum Protocol for Byzantine Fault
Tolerance
• Distributed Systems of Little Things– EnsemBlue: Integrating Distributed Storage and Consumer
Electronics(Distributed File System)– Persistent Personal Names for Globally Connected Mobile
Devices(sensornet architecture)– Making Information Flow Explicit in HiStar(Security)– Splitting Interfaces: Making Trust Between Applications and
Operating Systems Configurable. Proxos takes a novel approach to allowing applications to specify their trust in an operating system(Security)
– Connection Handoff Policies for TCP Offload Network Interfaces(Performance)
OSDI2006
• Distributed Storage and Locking– Ceph: A Scalable, High-Performance Distributed File
System• special-purpose data distribution function called CRUSH [29]
assigns objects to storage devices. calculate (rather than look up) the name and location of objects comprising a file's contents,
– Distributed Directory Service in the Farsite File System(Directory from centralized-distributed)
– The Chubby Lock Service for Loosely-Coupled Distributed Systems
• Chubby provides an interface much like a distributed file system with advisory locks, but the design emphasis is on availability and reliability, as opposed to high performance.
OSDI2006
• Large Distributed Systems– Experiences Building PlanetLab.– iPlane: An Information Plane for Distributed
Services• Emerging large-scale distributed services depend on
richer information about the state of the Internet• To provide a common measurement infrastructure for
distributed applications.
– Fidelity and Yield in a Volcano Monitoring Sensor Network
