Upload
dice
View
78.055
Download
1
Embed Size (px)
DESCRIPTION
Presentation by Johan Mjönes & Joakim Bodin at Stockholm GTUG meeting at DICE
Citation preview
BATTLELOG//Building scalable web sites with tight game integrationJohan Mjönes & Joakim Bodin
What is battlelog?
› Battlefield 10 year anniversary soon› Battlelog becoming the community hangout› Integrate game launching for PC players
BATTLELOG
Features
› Friend-centric› Easy communication› Current stats and history› Activity log› PC Menu› News / Forums
BATTLELOG
SCALING
TECHNOLOGY
TECHNOLOGYESN PLANET
TECHNOLOGYESN PLANET OVERVIEW
Languages
› Developing Battlelog means developing in Python, Java, C++› The Battlelog Web is written primarily using Python› Plugin, game components in C++› Python allows for rapid web development (e.g. remote access console)
TECHNOLOGY
TECHNOLOGY
Horizontal (sharding)
1-7 7-14 ...
Battlelog
User DBs(slices and shards)
......
Vertical
Battlelog
User Lab Feature
Different processes
Partitioning
Persistence & Index
› MySQL› No joins› Used as indexed KVS
› Apache Solr› Full text search for forums› Other uses in the future› Fast!
TECHNOLOGY
Server to client push
› Uses ESN Beaconpush› Allows delivery of messages from web server to client browser.› Uses long polling or websockets (or flash)
› AV / anti-malware software + push = › Blocks all websockets› Injects Javascript into the DOM› Etc...
TECHNOLOGY
Caching with Memcached
› Great framework support› Service methods cached via annotations
› Invalidation using method signature instead of key
› Allows populating cache directly after modification› Dog pile prevention
TECHNOLOGY
EA Services
› Nucleus› Users
› No user info in Battlelog DB! Success!
› Personas› Soldiers
› Entitlements› Licenses
› Blaze› Game Servers reports to Blaze› Events to Battlelog
› Game Server information› Asynchronous responses (e.g. matchmaking)
› REST› Stats
TECHNOLOGY
Nucleus Details
TECHNOLOGY
Nucleus
Accounts
Entitlements
Web Backend
Web
Soldiers
Licenses
HTTP
TECHNOLOGYBlaze Details
› Uses Web Access Layer (WAL)› WAL client generated from TDF
› TDF is a API definition language
› Blaze events (XML over HTTP)
Blaze
DB
Web BackendWeb Frontend
Blaze
Game Servers
WAL
Real-time Events
Thrift
Events
AJAXWeb Browser
BATTLELOG FEATURES
Friends
› Started out normalized› Unmanagable amount of rows› Ended up as one blob per user
› Packed user ids
› A lot easier to cache properly
BATTLELOG FEATURES
Server Browser
› Custom search server (Java)› Custom query language, minimal message overhead› Fast update, fast search
BATTLELOG FEATURES
Joining a game
BATTLELOG FEATURES
BATTLELOG FEATURES
Joining a game: plugin details
BATTLELOG FEATURES
Web Browser
Game ClientPluginGame UI JS
Pipe
Battle Reports
› Game report from Blaze (HTTP XML event)› Parse and divide per player
› One report per player via internal message queue› Look for unlocks, rankups, medals, awards etc
› Add to feed, send real time updates
› Invalidate player stats caches
› Compile Battlereport (& pre-cache it)› Send notification to involved (logged in) users
BATTLELOG FEATURES
LOAD TESTING
Background
› Load tested using Locust› Open Source (MIT License)› https://github.com/cgbystrom/locust› Built by ESN (& others)
› Battlelog tested with 2.7 million PSU with 36 million players› That’s 40,000 requests per second› The tests was successful
LOAD TESTING
Pre-pass
› Planet dev bar› Timing› Queries
› No point in load testing if there are apparent issues with a single user
LOAD TESTING
Realistic load testing
› Some testing tools simply hammer certain URLs› Locust allows for realistic usage scenarios
› Many players go away for 20 minutes while playing, receiving notifications
› Updated tests using actual usage data from alpha trial / open beta
› Coordinated testing› Same user spans
› Overpopulate tables to simulate fragmented MySQL indices
LOAD TESTING
What did we find?
› Bugs› Bottlenecks
› Never assume where your bottlenecks are› Test to find out
› Network related› Saturated network, not CPU› Keep the traffic within the rack!
› Software related› Best practices be damned, cheat if neccessary› Lazy loading data in controllers› Client side rendering
› Surface + History API + JSON =
LOAD TESTING
GAME + WEB DEVELOPMENT
Being a part of the pipeline
› Data driven games› Battlelog has its own pipeline
› Written in Python› Imports data from the game pipeline
› Weapons, vehicles, levels, dog tags, etc› Assets include images, text, data
› Resolving unlocks and dependencies› Generates python structs and javascript structs
› Generates many helper maps. Some might only be used once.
› DLC friendly!› Result of many iterations
GAME + WEB DEVELOPMENT
User Interface› Separate UI from consoles / in game menu
› Having the same wouldn’t make sense, different UX› Adapt to different devices
› Web features› Easily accessible, e.g. Sharing› Extendability with browser extensions, user styles etc› Access on any platform (play on console, have Battlelog on iPad)
› Easy to iterate over UI changes
GAME + WEB DEVELOPMENT
THE FUTURE
Battlelog will evolve
› Agile: Easy to update› Big things:
› DLC› Public APIs› Other things we can’t talk about
› Small things› Improvements based on community feedback
THE FUTURE
YOUR TEAM WON
QUESTIONS?
Johan Mjönes@nollbit
Joakim Bodin@jbripley
The Battlelog team is hiring!
http://dice.se/[email protected]
Technical DirectorDevelopment DirectorBackendFrontend