Upload
lora
View
33
Download
0
Tags:
Embed Size (px)
DESCRIPTION
VisualizeME. Kia Manoochehri. Motivation. Data Analysis Learn about AWS Struggle to do so Data Visualization An exercise in scalability. Motivation. Do something different. Introduction. Original intent: Gather Windows Event Log files Determine when an error would occur - PowerPoint PPT Presentation
Citation preview
Slide 1
VisualizeMEKia ManoochehriMotivationData Analysis
Learn about AWSStruggle to do so
Data Visualization
An exercise in scalabilityMotivationDo something different.
IntroductionOriginal intent:Gather Windows Event Log filesDetermine when an error would occurVisualize this somehowIntroductionProblems:Gather Windows Event Log filesNo access to them outside of my ownDetermine when an error would occurEasy if I had access to a monumental amount of dataVisualize this somehowComplicated because *what* would I visualize?IntroductionActual design:A Log File (data) Analyzer of a different system
Ran through AWS to exploit parallelization of the log filesExercise in scalability
Visualize this data in a meaningful way
Side note about the DataCame from another project I previously worked on
.txt files of varying size
Total Volume of data = 6gbPotential ResultsShow that using AWS can relieve problemsMore data?Errors?Runtime?
Have a cool visualization tool!Data in an easy to read way.Tasty Preview of the Result
Tasty Preview of the Result
OrganizationDesign DecisionsMATLABMany prosMany many cons
AWSWe werent given any other option
Project itselfProblems EncounteredMATLABAble to use the Parallel Computing ToolboxCosts $$$ - Need licenses for the cloud
Solution?Be dirtyProblems EncounteredAWSCame into this class with 0 cloud experience and knowledge
Solution?Spent more time outside of class learning and reading about the cloud and running an application on AWS than inside of class.Problems EncounteredProject itselfMultiple aspects to this projectAnalyzer (coding)Running it on AWS (design, choices, and 0 exp)Visualizing the data (coding and design)
On the surface:Not an impressive use of AWSGood lesson on Scalability aspect of Cloud ServicesGreat lesson on Trust/Security of dataImplementation
ImplementationMATLAB on EC2
Implementation
ImplementationDevelopment and Testing:My home desktop and laptopGIT makes this easy
After Tobys presentation on AWS early in the semester I chose to isolate my development and testing environment (from the cloud).
PerformanceScalability:Time and effort saving
Fundamentally, space saving>6gb (6144 MB) of .txt files compressed/converted to 229MB (wow)
Visually pleasing?Lessons LearnedAWSApplication of some of the topics we discussedFirst hand account of security issues and reluctance to use the Cloud
Dont over commit
I was already addicted to Caffeine.
DEMO?Demo Fail???
Demo Fail???
Demo Fail???
Demo Fail???