Apache Geode - The First Six Months

The First Six Months

(incubating)

About

Anthony Baker (@metatype)

Engineering leader for Pivotal GemFire, committer on Apache Geode

How we transformed a hard-core commercial engineering team into an open source community-driven

powerhouse*

* forward-looking statement

What the government doesn’t want you to know about open source

6 reasons to be addicted to open source

7 things Lady Gaga has in common with open source

8 unbelievable things you never knew about open source

Why you should give up sex and devote your life to open source

11 ways investing in open source can make you a millionaire

http://www.contentrow.com/tools/link-bait-title-generator

Alternative titles?

(just kidding)

Topics• Introducing Geode

• Preparing for incubation

• Getting started

• Adopting The Apache Way

• Growing a community

Introducing Geode

Apache Geode is…

"…an in-memory, distributed database

with strong consistency

built to support low latency

transactional applications

at extreme scale.”

2004 2008 2014

•  Massive increase in data volumes

•  Falling margins per transaction

•  Increasing cost of IT maintenance

•  Need for elasticity in systems

•  Financial Services Providers (every major Wall Street bank)

•  Department of Defense

•  Real Time response needs •  Time to market constraints •  Need for flexible data

models across enterprise •  Distributed development •  Persistence + In-memory

•  Global data visibility needs •  Fast Ingest needs for data •  Need to allow devices to

hook into enterprise data •  Always on

•  Largest travel Portal •  Airlines •  Trade clearing •  Online gambling

•  Largest Telcos •  Large mfrers •  Largest Payroll processor •  Auto insurance giants •  Largest rail systems on

earth

China RailwayCorporation

5,700 train stations 4.5 million tickets per day

20 million daily users 1.4 billion page views per day

40,000 visits per second

*http://pivotal.io/big-data/pivotal-gemfire

Indian Railways

7,000 stations 72,000 miles of track

23 million passengers daily 120,000 concurrent users

10,000 transactions per minute

World: ~7,349,000,000

~36% of the world population

Population: 1,251,695,6161,401,586,609

China RailwayCorporation

Indian Railways

Application patterns• Caching for speed and scale using read-through,

write-through, and write-behind

• OLTP system of record with in-memory for speed, on disk for durability

• Parallel compute grid

• Real-time analytics

Preparing for Incubation

Some context

• 1M+ LOC, 1000’s of customers running business critical systems in production

• More than 12 years in development

• Large, multi-geo engineering team

• Established development practices

Why OSS? Why ASF?• Open source is fundamentally changing software

buying patterns

• Customers avoid vendor lock-in and get transparency, co-development of features

• It’s the community that matters

• ASF provides a framework for open source

Find a champion

(Roman)

and mentors too!

A rose by any other name…

Legal & biz stuff

• Source code clearance

• License compatibility, attribution

• Commercial features?

Cleaning up the source

• Get rid of internal dependencies

• Make sure the build is easy and fast

• Remove embarrassing source comments :-)

• Make testing easy

Writing the proposal

• Use a prior project as a template (lots of options)

• Identify background, rational, status, comparables, risks

• Who are the committers?

Submitting the proposal

• Is your source available? Not necessary, but helpful!

• Discussion will occur on general@incubator.apache.org

• Wait for consensus and vote :-)

Getting Started

Setting up infrastructure

Review Board

https://issues.apache.org/jira/browse/INFRA-9536

Getting social

http://what’s important• Brief product description

• Community coordinates

• Email lists (dev, user), JIRA, wiki

• Getting started (one page)

• Building and running the product

• How to obtain the source (no binaries before a release)

• Roadmap, things to work on

Adopting The Apache Way

https://en.wikipedia.org/wiki/The_Scream#/media/File:The_Scream.jpg

http://theapacheway.com

“The Apache Way is sort of like Zen. It's something that's difficult to explain, has many

interpretations, and the best way to learn it is to do it.”

–Shane Curcuru, ASF

“Community, Merit, and Openness, backed up by Pragmatism and Charity.”

The Apache Way

• Community over code

• If it didn’t happen on the mailing list, it didn’t happen

• Rough consensus and working code: do-ocracy

• Decision model: +1, 0, -1 (also lazy)

Let it go!(sing along with me)

Apache stuff

• License headers (duh)

• Packages: org.apache.*

• Version: *-incubating

• Incubation reports

Policies• RTC or CTR?

• Branching (gitflow)

• Versioning (semver)

• Role of mentors (advisors or participants)

• Commit messages (GEODE-XXX: …)

• Design reviews

Committing>  git  flow  feature  start  GEODE-­‐nn  

>  rbt  post  -­‐-­‐repository  geode  

(wait  for  +1)  

>  git  checkout  develop  >  git  pull  -­‐-­‐rebase  >  git  flow  feature  finish  -­‐-­‐no-­‐ff  GEODE-­‐nn  

>  git  push

Pull Requestscontributor:  

fork,  clone,  <write  code>,  create  PR,  wait  for  review  

committer:  

>  git  remote  add  github  https://github.com/apache/incubator-­‐geode  

>  git  fetch  github  pull/n/head:feature/GEODE-­‐nn  >  git  checkout  feature/GEODE-­‐nn  

>  git  checkout  develop  >  git  pull  -­‐-­‐rebase  >  git  flow  feature  finish  -­‐-­‐no-­‐ff  GEODE-­‐nn  

>  git  push

Releasing

Growing a Community

Engage!https://hu.wikipedia.org/wiki/Jean-Luc_Picard#/media/File:Jean-Luc_Picard_2.jpg

Project roles• Contributor

• Lurking on mailing lists :-)

• Asking questions

• Answering questions

• Submitting bugs

• Submitting patches

• Committer

• PMC

Include your customers• Increase their opportunity to interact directly with

committers

• Do they want to contribute existing tooling?

• Questions they will ask:

• Are you still committed to the product?

• Why should I pay?

Community activities• ApacheCon! (and other conferences)

• OSCon, SpringOne2GX, QCon, …

• Local meetup groups

• Portland, Toronto, Palo Alto, San Francisco, London, Cork, Pune

• Virtual meetups (Geode Clubhouse) 2x / month

• Blogs, twitter, hackathon

Virtual meetups• Two formats

• Technical deep-dive on a specific subject

• Open mike, like a “standup” where participants get to bring up topics for discussion

• Note: any decisions must still be published on the dev list!

Be responsive

• More responsive and interactive communities have better engagement and retention

• On a temporary basis, find community activists to ensure questions, PR’s, and bugs are addressed quickly

• Soon this behavior becomes automatic

“Open source is where ecosystems are built.”

Ecosystem

YCSB

Metrics

• Committers: > 60

• Commits: 194 on /develop

• JIRA’s: 219 (open), 164 (resolved)

• Emails: 1658 (dev), 212 (user)

What’s next? TLP of course!

https://issues.apache.org/jira/browse/GEODE-1

Thank you!

http://geode.incubator.apache.org dev@geode.incubator.apache.org

@ApacheGeode