MercadoLibre (NASDAQ: MELI) is eBay’s Latin American partner, the largest e-commerce ecosystem in Latin America, and comScore’s 8th largest e-retailer in the world. A Fortune “Fastest-Growing Company” for three years, 2013 showed net revenues and gross merchandise volume up by 50%, total payment volume up 66%, and registered users up 22% to 99.5 million. Headquartered in Argentina, they operate across thirteen countries, and allow people to list items, sell, buy, pay, and collect payments online. They also provide payment solutions, digital advertising, and web stores.

As a digital business, years of growth began to impact the original architecture, and analysts forecasted more growth in the future. In 2010, they began transforming the original, monolithic e-commerce system into a next-generation, decoupled architecture to provide independence to internal teams while scaling applications, data, and data centers. They also expanded the core API layer which provides access to users, items, orders, and more.

CHALLENGE Two things drove Mercado Libre to change—speed to market and scaling past 20 million requests per minute and 4GB of bandwidth per second.

User Experience Suffers from Slow Release Cycles and Intertwined Development Applications must scale to achieve solid user experience and customer satisfaction. Matias Waisgold, Technical Lead for the core API, explained, “Over time, our services and applications became entangled. This caused problems. Often, one team’s release impacted other team’s code. Our release frequency slowed, and we couldn’t improve user experience fast enough. So, we set a vision—an enterprise service bus with decentralized, decoupled applications that support an agile process.”

Revenue at Risk with Traditional Databases, Tight Coupling, and Single Data Centers Mercado Libre’s application generates revenue. So, scale and uptime are critical. “As our user-base grew, we needed a multi-

AT-A-GLANCE

Challenges

• Improving time to market and user experience

• Scaling past 20 million requests per minute and 4GB bandwidth per second

• Reliably supporting revenue while removing traditional database bottlenecks

• Connecting micro-services with an ESB architecture

Solution

• RabbitMQ

• Redis

• Groovy on Grails

Key Benefits

• Improved development agility, time to market, and user experience

• Scaling data stores and applications across languages with an ESB

• Open source solutions and support communities

CASE STUDY

Mercado Libre IMPROVING USER EXPERIENCE, DEVELOPMENT AGILITY, AND SCALING TRADITIONAL DBS

OVERVIEW

“ Our ESB, built largely with RabbitMQ, Groovy on Grails, and Redis, has a big impact on development and operations. We can scale individual apps and data services instead of wrestling with one big monolithic structure where we would be bound by the slowest part. Asynchronous and independent components also allow our teams to develop and deploy independently with less governance and coordination. Together, these three tools help us get to market faster, improve our competitiveness, make a better user experience, and ultimately impact revenue.”

—Matias Waisgold, Technical Lead, Mercado Libre

pivotal.io

Pivotal is a registered trademark or trademark of Pivotal Software, Inc. in the United States and other countries. All other trademarks used herein are the property of their respective owners. © Copyright 2014 Pivotal Software, Inc. All rights reserved. Published in the USA. PVTL-CS-367-06/14

At Pivotal our mission is to enable customers to build a new class of applications, leveraging big and fast data, and do all of this with the power of cloud independence. Uniting selected technology, people and programs from EMC and VMware, the following products and services are now part of Pivotal: Greenplum, Cloud Foundry, Spring, GemFire and other products from the VMware vFabric Suite, Cetas and Pivotal Labs.

CASE STUDY MERCADO LIBRE

Pivotal 3495 Deer Creek Road Palo Alto, CA 94304 pivotal.io

datacenter architecture to avoid failures and help scale. We knew our Oracle database wouldn’t support the volume of reads and writes, and both users and usage were growing across geographies. We designed a hybrid cloud with reads/writes in one location and reads from other data centers. We also needed a cache layer to take load off the database. Lastly, we wanted to support a distributed computing model that allowed us to scale specific services or components,” said Waisgold.

SOLUTIONDelivering Faster Releases and Scaling with RabbitMQ, Redis, and Groovy Largely an open source shop, the Mercado Libre team built a roadmap. At the application layer, they broke the application into micro-services and transformed each into an open, API-based platform connected by an enterprise service bus (ESB).

At the data layer, they designed the main data center to support reads and writes with 100% data consistency. The team relaxed data consistency rules for reads in the other data centers and used Oracle’s, Java-based Golden Gate technology to replicate across data centers and feed changes to RabbitMQ within a single data center. RabbitMQ messages to a refetching service written in Groovy on Grails that updates a Memcached cluster and cost effectively scales the RDBMS. The refetchers also publish to a system called Big Queue that uses Redis for in-memory storage. Big Queue stores events as separate topics within each datacenter. These are consumed asynchronously by other apps in parallel. It supports an HTTP-based API, is highly available, and consistent within a data center, ultimately delivering the ESB messages to dozens of services.

By taking this approach, components independently scale to address bottlenecks. Each application now operates asynchronously and uses the ESB to listen for relevant events from other apps like a new listing or an update to an item. For example, data changes pass through the bus to notify and update Elastic search, and the developer APIs for items updates both Oracle and RabbitMQ directly, leading to cache updates and ESB events. The decentralized architecture allows each business unit and development team to operate with autonomy. As long as shared APIs are respected, developers can deploy specific app updates or scale components as needed. Users of the site still see a seamless, single experience made up of dozens of apps each controlled by a separate team.

BUSINESS BENEFITSImproved User Experience, Time to Market, and Development Agility “RabbitMQ is certainly very easy to understand, use, and implement. However, our ESB, built largely with RabbitMQ, Groovy on Grails, and Redis, has a bigger impact on development and operations teams. We can scale individual apps and data services instead of wrestling with one big monolithic structure where we would be bound by the slowest part. Asynchronous and independent components also allow our teams to develop and deploy independently with less governance and coordination. Groovy helps us accelerate the time we spend coding and testing. Redis stores key-values of big data in memory for the real-time use cases across our ESB. Together, these three tools help us get to market faster, improve our competitiveness, make a better user experience, and ultimately impact revenue,” said Waisgold.

Scaling Data Stores and Applications across Languages Waisgold explained, “RabbitMQ is lightweight, supports the open standard AMQP protocol, and helps to scale our existing RDBMS by feeding the cache and Big Queue through Groovy, ultimately fanning out to other apps. Across 18 queues that are fed by Oracle or external APIs, RabbitMQ processes spikes up to 1000 messages per second and 2000 acknowledgements per second on a 4 machine cluster with 16 GB of RAM and 8 cores each. We see 0.188-second transactions writing to RabbitMQ, which is super fast. With external developer APIs for items, we write to RabbitMQ directly from our webservers without degrading performance. RabbitMQ gives us a simple, reliable way to scale on pools of shared infrastructure or components and makes it easy for any app to talk to Java, Groovy on Grails, Ruby, or virtually any other language we might use. Similarly, the Redis data store works with all modern development languages, and this gives us tons of flexibility for downstream apps.”

CONCLUSIONWith RabbitMQ, Redis, and Groovy on Grails, Mercado Libre has a set of open source capabilities and a surrounding community to help them improve development lifecycles and scale both applications and data services, ultimately improving the customer experience. As they look to future growth, they have a solid foundation with a cloud-based architecture they can rely on.

LEARN MORETo learn more about our products, services and solutions, visit us at pivotal.io.