Slide 1

VisualizeMEKia ManoochehriMotivationData Analysis

Learn about AWSStruggle to do so

Data Visualization

An exercise in scalabilityMotivationDo something different.

IntroductionOriginal intent:Gather Windows Event Log filesDetermine when an error would occurVisualize this somehowIntroductionProblems:Gather Windows Event Log filesNo access to them outside of my ownDetermine when an error would occurEasy if I had access to a monumental amount of dataVisualize this somehowComplicated because *what* would I visualize?IntroductionActual design:A Log File (data) Analyzer of a different system

Ran through AWS to exploit parallelization of the log filesExercise in scalability

Visualize this data in a meaningful way

Side note about the DataCame from another project I previously worked on

.txt files of varying size

Total Volume of data = 6gbPotential ResultsShow that using AWS can relieve problemsMore data?Errors?Runtime?

Have a cool visualization tool!Data in an easy to read way.Tasty Preview of the Result

Tasty Preview of the Result

OrganizationDesign DecisionsMATLABMany prosMany many cons

AWSWe werent given any other option

Project itselfProblems EncounteredMATLABAble to use the Parallel Computing ToolboxCosts $$$ - Need licenses for the cloud

Solution?Be dirtyProblems EncounteredAWSCame into this class with 0 cloud experience and knowledge

Solution?Spent more time outside of class learning and reading about the cloud and running an application on AWS than inside of class.Problems EncounteredProject itselfMultiple aspects to this projectAnalyzer (coding)Running it on AWS (design, choices, and 0 exp)Visualizing the data (coding and design)

On the surface:Not an impressive use of AWSGood lesson on Scalability aspect of Cloud ServicesGreat lesson on Trust/Security of dataImplementation

ImplementationMATLAB on EC2

Implementation

ImplementationDevelopment and Testing:My home desktop and laptopGIT makes this easy

After Tobys presentation on AWS early in the semester I chose to isolate my development and testing environment (from the cloud).

PerformanceScalability:Time and effort saving

Fundamentally, space saving>6gb (6144 MB) of .txt files compressed/converted to 229MB (wow)

Visually pleasing?Lessons LearnedAWSApplication of some of the topics we discussedFirst hand account of security issues and reluctance to use the Cloud

Dont over commit

I was already addicted to Caffeine.

DEMO?Demo Fail???

Demo Fail???

Demo Fail???

Demo Fail???