Embed Size (px)
Citation preview
GamePostInsight Data Engineering FellowshipNew York, NY
My motivation● Compare popularity of game titles in real time using social media
as the data
Use cases
● Track popularity/awareness of a title, has marketing uses● Edge on competitors with this knowledge
○ Ex. release dates close together● Program can be generalized to phrases/events in general
Data Pipeline
Amazon KinesisAWS Lambda
Serverless Architecture
Very little maintenance - less time and headache for engineers
Focus on the actual functions, less on servers
Core functionality implemented, can switch tools anytime
Inexpensive!
Ex. AWS Lambda - 20 cents/million function calls
Algorithm Optimizations
● Efficiency - Memory for speedup
List: { God of War, Halo, Grand Theft Auto: San Andreas }
Naive search (two ways):
Search for game title in string (if string contains title)
Search if subset of string is game title (if subset of string = title)
Search for game title in string:
If string.contains(“God of War)
If string.contains(“Halo”)
If string.contains(“Grand Theft Auto: San Andreas) and so on..
Traverses over whole string m times, where m = # of game titles in the set
Complexity: O(n*m) per string n = length of string, m = # of game titles in list
Search for game title in string:
If string.contains(“God of War)
If string.contains(“Halo”)
If string.contains(“Grand Theft Auto: San Andreas) and so on..
Traverses over whole string m times, where m = # of game titles in the set
Complexity: O(n*m) per string n = length of string, m = # of game titles in list
Search if substring is game title
If “This tweet will” == “God of War”
If “tweet will not” == “God of War”
If “will not have” == “God of War”
If “not have any” == “God of War”
If “have any matches” == “God of War”
and so on ...
● Essentially a heuristic of method #1○ Word chunking
Same time complexity
Search if substring is game title
If “This tweet will” == “God of War”
If “tweet will not” == “God of War”
If “will not have” == “God of War”
If “not have any” == “God of War”
If “have any matches” == “God of War”
and so on ...
● Essentially a heuristic of method #1○ Word chunking
Same time complexity
Trie search tree
Traverse the string once
Building - O(m*l), where m = # of titles, l = avg. length
Searching - O(n) complexity, where n = length of string
About me
● Enjoy the outdoors● Wide range of hobbies from fishing to billiards● Graduated with New York University’s class of 2016
○ BA in Computer Science
