Upload
sam-keen
View
120
Download
0
Tags:
Embed Size (px)
Citation preview
REQUEST TRACING
PROMOTING TRANSPARENCY & TRUTHFULNESS
Hello!I AM SAM KEENpolyglot software developer
You can find me at:github.com/samkeenlinkedin.com/in/[email protected]/athletes/560497
REQUEST TRACING
What is it?
ASSOCIATING A GLOBALLY UNIQUE IDENTIFIER TO ALL REQUESTS OF A GIVEN SERVICE
COMMON OCCURRENCE IN THE WILDPOST https://graph.facebook.com/v2.2/…Response headers…x-fb-rev → 1821576
x-fb-trace-id → E1+VUthAHKh
x-fb-debug → NXN2ONVBqC8lLJVzgjknii86r5AJbmLGS5zX227q+jmLI8sm+IIuAo8jETQ3bmH82w9SmjfAQktXTXpZJ7rtuA==
GREAT, LET’S SEE THE CODE!
ACTUALLY, I’D LIKE TO APPROACH THE SUBJECT IN A DIFFERENT WAY
BEFORE, WHAT, THERE IS WHY
Development teams have a limited bandwidthBefore launching an endeavor, estimate the value it will add to the company and use that to prioritize the work, or possibly rule it out
MEASURE VALUE
How will the work help you obtain your goals
OK, GOALS, WHERE DO WE GET THOSE?
COMPLEXITY
Software is one of the most complex of human endeavors
To organize, prioritize goals, We need a plan
STRATEGIC GOALS
These are the highest level, inward facing drivers for a company
ACQUIRE ONLINE SHOE SALES MARKET LEADER
PRINCIPLES
Rules in place to help you remain aligned with Strategic Goals
SYSTEMS ARE BUILT WITH CONSISTENT, DOCUMENTED INTEGRATION POINTS
^ supports goal
PRACTICES
Often technology specific guidance to ensure Principles are being upheld
ALL SERVICES ARE DOCUMENTED WITH API BLUEPRINT
^ supports principle
GOALS <PRINCIPLES <PRACTICES
Strategic Goals
+ Acquire online shoe sales market leader
Principles
+ Systems are built with consistent and documented integration points
Practices
+ All services are documented with API Blueprint
GOALS <PRINCIPLES <PRACTICES
Strategic Goals
+ Acquire online shoe sale market leader
+ Enable rapid innovation
Principles
+ Systems are built with consistent and documented integration points
+ Ensure systems are transparent and discoverable
Practices
+ All services are documented with API Blueprint
+ All application logs leverage trace tokens
PRINCIPLE: SYSTEM TRANSPARENCYwhat do we mean by that
NOT A NEW CONCEPT
The Art of Linux ProgrammingChapter 6 : Transparency
SOFTWARE SYSTEM TRANSPARENCYGOOGLED
- A strength of OS software is that the src code is available to all so the NSA cannot hide back doors in it.
- DNS is a transparent system
A TRANSPARENT SOFTWARE SYSTEM
- has a low learning curve, low barrier to entry
- values simplicity over complexity- has high discoverability- is highly accessible
TRANSPARENCY SUPPORTING PRACTICES
- gathering performance metrics- All devs have access to all repos- Any given system should be capable of
running on a dev's machine (README)- centralized logging- Verbose modes of operation (-v). Have
verbosity off by default but don't make it inaccessible
WTF, I THOUGHT THIS WAS A TALK ON TRACING?WHEN DO WE GET TO THE TRACING?
PRACTICE TO SUPPORT TRANSPARENCY PRINCIPLE
Request Tracing
LOG ALL THE THINGS
disk is cheapcpu is cheap
logs can be valuable for more than stack traces
THE STEPS
- per request create some sort of UUID token
- override your logger to add said token to all log statements
- add said token to all responses (in case of HTTP, use header)
ALTERNATIVES
- Separate handler to its own log file- a TRACE level in the logger, just below or
above DEBUG
DEMO
CLI
aws --profile test logs filter-log-events \
--log-group-name export-demo \
--filter-pattern 559819cf1381f468e
SCALING
The are many know strategies for scaling log aggregationAlso many for log analysis
For Instance:Real-time Processing of Log Data with Subscriptions [http://goo.gl/Oo1m73]
PRINCIPLE: TRUTHFULNESS
How truthful is our documentation
How do you create / maintain truthful documentation
TRUTHFUL DOCUMENTATIONTHE REALITYStatic docs are at best: Truth’ish
As soon as they are pened, they they start a steady migration towards lies and half-truths
Entropy always wins
OFTEN THE CASEDOCS → MISINFORMATIONplanning stage, we document the system we plan to build. valuable exercise, spawns collaboration and feedbackThen we build the system. Then the system ALWAYS changes.No one goes back and updates the docs
REDUCE PURE STATIC DOCS
- Generated docs can help (javadoc, etc)- Real time metrics : better to look at what
the app is actually doing, than rely on a perf test from 3 months agoContinual Non-functional testing anyone?
- All docs should be wiki style, including diagrams!
DOCUMENTATION BEST PRACTICE
Generate as much documentation as possible from the actual operation of the System and/or the source code of the System
PRACTICE TO SUPPORT TRUTHFUL DOCUMENTATION PRINCIPLE
Request Rendering
DEMO
THANKS!Any questions?
You can find me at:@samkeen / github.com/samkeenSlides:
CREDITS
Special thanks to all the people who made and released these awesome resources for free:○ Presentation template by SlidesCarnival○ Photographs by Unsplash