View
225
Download
5
Category
Preview:
Citation preview
Adventures with Laravel and Kubernetesat superbalist.com
William Stewart zatech zoidbergwill basically everywhere
1 / 28
OverviewDocker
What is Docker?Why do we use it?
KubernetesWhat is Kubernetes?Why do we use it?
Some extra toolsA quick demoWar stories
LoggingSignal propagationLong running processesSchema changes
3 / 28
How does it help?Language agnostic build artifact
FROM composer:alpineCOPY composer.json composer.lock /appRUN composer install --ansi --no-interaction --no-scripts --no-dev --no-autoloaderCOPY . /appRUN composer install
or
FROM node:alpineCOPY package.json /appRUN npm installCOPY . /app
6 / 28
How does it help?No more "works on my machine", because they're the same images locally and on production
Smaller shippable artifacts than full machine images, and easier to independently scale components
A lot of useful services already in the Docker registry:
MySQLRedisMemcachedElastic SearchRabbitMQ
7 / 28
How does it help?- Container hosting- Con�g changes- Supervision- Monitoring- Rolling deployments- Networking / Service discovery- and more...
9 / 28
Automate the boring stu�f“Automation is a force multiplier, not a panacea”
Value of automationConsistencyExtensibilityMTTR (Mean Time To Repair)Faster non-repair actionsTime savings
Dan Luu's Notes on Google's Site Reliability Engineering book
10 / 28
How does it help?Faster/smarter builds than a .tar.gzReliable rolling updates (safer than a symlink swap)Similar build/deploy scripts for all languages and servicesAutoscaling (based on CPU usage)Fine-grained healthcheckingAccurate resource monitoring and better usage
12 / 28
Helps us to easily move towards SOA,Easily start new projects in any language, that can rely on the same service discovery (using DNS)
Discovery via DNS (curl http://elasticsearch/) or automatically set Kubernetes environmentvariables (env('ELASTICSEARCH_SERVICE_HOST')), that balances load over a collection of Elastic Searchclients, all automatically updated by Kubernetes.
A huge collection of base Docker�les on hub.docker.com and quay.io
13 / 28
Avoid "works on my machine" issuesAccurately replicate most of our production environment locally (other than a mocked Google Loadbalancer)
Using:
minikube, which behaves like a single-node Kubernetes clusterdocker, moving towards rocker, to better utilise global Composer/NPM cacheNFS, to allow faster feedback when developingsome bash glue
14 / 28
Easily implement monitoring, logging, and LetsEncryptSimply implement metrics monitoring and LetsEncrypt using Kubernetes annotations:
annotations: prometheus.io/port: "9150" prometheus.io/scrape: "true"
apiVersion: extensions/v1beta1kind: Ingressmetadata: annotations: kubernetes.io/ingress.class: gce kubernetes.io/tls-acme: "true" name: general-ingressspec: rules: - host: tasks.zoidbergwill.com http: paths: - backend: serviceName: tasks-app servicePort: 80 path: / tls: - hosts: - tasks.zoidbergwill.com secretName: tasks-tls
15 / 28
Demo toolsMinikube
Rocker
Kube-lego
16 / 28
LoggingLogging to a �le worked great on compute engine with �uentd
Gcloud Stackdriver Logging is pretty amazing, but with container engine it wants stdout
± ag log tasks/config/app.php112: | Logging Configuration115: | Here you may configure the log settings for your application. Out of116: | the box, Laravel uses the Monolog PHP logging library. This gives117: | you a variety of powerful log handlers / formatters to utilize.119: | Available Settings: "single", "daily", "syslog", "errorlog"123: 'log' => env('APP_LOG', 'single'),125: 'log_level' => env('APP_LOG_LEVEL', 'debug'),212: 'Log' => Illuminate\Support\Facades\Log::class,
19 / 28
Logging Monolog/Laravel optionssingle/daily �le:
doesn't go to stdout and get consumed by fluentd-cloud-logging
syslog:
another dependency, and we'd need to install syslog and tail it, or talk to an external syslog daemon
errorlog:
php-fpm logs have a pre�x you can't get rid ofBubbling them to the nginx container also has a gross pre�x and means you can't separate the logs for each container
20 / 28
Upstream php images also do some gross logging stu�fFROM php:7-fpm
[global]error_log = /proc/self/fd/2
[www]access.log = /proc/self/fd/2
Access logs are junk for Laravel:
:: - 07/Feb/2017:13:57:57 +0000 "GET /index.php" 200 (x 1337)
error_log can be pretty crap too:
[07-Feb-2015 11:46:44] WARNING: [pool www] child 32465 said into stderr: "NOTICE: PHP message: Array"[07-Feb-2015 11:46:44] WARNING: [pool www] child 32465 said into stderr: "("[07-Feb-2015 11:46:44] WARNING: [pool www] child 32465 said into stderr: " [6] => 363"[07-Feb-2015 11:46:44] WARNING: [pool www] child 32465 said into stderr: ")"[07-Feb-2015 11:46:44] WARNING: [pool www] child 32465 said into stderr: ""
21 / 28
Going full circleLet's try single again:
> cat entrypoint.sh# Tail a log file, and background that processtail -f /var/www/html/storage/logs/laravel.log &# Now run the more important piecephp-fpm7
22 / 28
Signal propagationLinux relies on PID 1 doing important stuff, that an init system does
Most Docker entrypoints use exec to become the main process, but this kills anything running in the script.
Shells (like bash) don't propagate signals, like SIGTERM or SIGQUIT, which messes with graceful shutting down.
When a pod dies:
PID 1 in the container gets a SIGTERM and given a grace period (default: 30s) to shutdown by defaultOnce the grace period has expired then a SIGKILL signal is sent
dumb-init and lifecycle hooks to the rescue!
dumb-init wraps your ENTRYPOINT shell script and makes sure signals propagate to child processes
lifecycle: preStop: exec: # SIGTERM triggers a quick exit; gracefully terminate instead command: ["/usr/sbin/nginx","-s","quit"] # or php artisan queue:restart
23 / 28
CronIf you run plain old cron it runs processes in a separate env, so we dump our Kubernetes environment for phpdotenv toread:
# Cron is a bastard that has sub processes with different env vars.env > .envsed -i -e "s/=\(.*\)$/=\"\1\"/g" .env
24 / 28
Long running processesThe max "grace period" for a pod is 5 minutes.
Easy steps in the right direction:
Break tasks up as small as possiblePut most tasks on queuesUse Kubernetes "Jobs" when forced to
25 / 28
Superbalist.com and Takealot.com are hiring!!!
Sourcehttps://github.com/zoidbergwill/kubernetes-php-examples
Slides: zoidergwill.github.io/presentations/2017/kubernetes-php/
Links on the next slide!
27 / 28
LinksO'Reilly's Site Reliability Engineering: How Google Runs Production Systems Amazon
Dan Luu's notes are good too
Borg, Omega, and Kubernetes Lessons learned from three container-management systems over a decade. Essay
kubernetes-dashboard
Cloud Native Computing Foundation
Kubernetes-Anywhere: An of�cial Kubernetes repo with some documentation on running Kubernetes with Dockerfor Mac beta
minikube: The of�cial Go binary for running a simpler local cluster.
awesome-kubernetes list on GitHub, cuz it has some neat things.
Docker and the PID 1 reaping problem
Containers really are the future
28 / 28
Recommended