Upload
evans-ye
View
399
Download
2
Tags:
Embed Size (px)
Citation preview
BigTopVM and Docker
ProvisionerEvans Ye
2015.2.17
Apache Bigtop is a project for…
• Packaging• Packages Hadoop ecosystem
• Testing• integration/interoperability testing framework
• Virtualization• vagrant/docker based hadoop provisioning
• Deployment• puppet recipes for included components
What does vagrant-puppet do?
1) Create a set of VMs by vagrant box
2/17/2015
Vagrant Box(Centos, Ubuntu,…)
VirtualboxVMs
2) Install puppet agents
2/17/2015
3) Deploy Hadoop by bigtop puppet (masterless)
2/17/2015
puppet apply -d--modulepath="bigtop-deploy/puppet/modules:/etc/puppet/modules" \bigtop-deploy/puppet/manifests/site.pp
puppet apply…
puppet apply…
puppet apply…
One click auto deployment
2/17/2015
$ cd bigtop-deploy/vm/vagrant-puppet-vm$ vagrant up
And the docker-puppet?
Similar to vagrant-puppetbut spin up a cluster on top of Linux containers
2/17/2015
$ cd bigtop-deploy/vm/vagrant-puppet-docker$ ./docker-hadoop.sh --build-image --create 3
Some prerequisites (see each README.md)vagrant-puppet docker-puppet
2/17/2015 Confidential | Copyright 2013 TrendMicro Inc. 2
Use cases
• for BigTop• to do bigtop integration tests
• to test bigtop puppet recipes
• integrates packaging, deployment, and testing all into one package(demonstrate the BigTop life cycle)
• for Contributors & Users• run round trip test locally (from source code to a testing cluster)
• easy-to-get Hadoop cluster environment
2/17/2015 Confidential | Copyright 2013 TrendMicro Inc. 2
Execute BigTop smoke-tests
• vagrant-puppet• README.md
• vagrantconfig.yaml• set run_smoke_tests: true
• Vagrantfile
• docker-puppet• ./docker-hadoop.sh --smoke-tests
2/17/2015 Confidential | Copyright 2013 TrendMicro Inc. 2