Microfinance : Data Tactics and USMA West Point Summer Camp 2013

Data Tactics & US Military Academy: A Prime in Microfinance using KIVA

WELCOME!

Thursday, August 8, 13


The Team:

Colin Fisk (Cadet 2014), Evan Hansen (Cadet 2015), Danny Marx (ODU 2015), Isaac Faber (MAJ USMA Faculty), Andrew Huang (sorta) Richard Heimann, Ted Procita, Nathan Danneman, Lee Shabe.

Rstudio: analytics.data-tactics-corp.com; PostgreSQL: analytics.data-tactics-corp.com Port: 5432

The GOAL:

Sustain a DT special project for nearly a year.

Operationalize ideas external to Data Tactics and find value.

Unique engagement with Military Academy faculty and students (share resources and experience).


http://analytics.data-tactics-corp.com/






ISW

USMA








Week 1 Literature Review (49 Papers, 3 books)

Week 2 Learning R & SQL || Data Analysis

Week 3 Data Analysis & Presentation








Understanding the complex nature of microfinance more completely: The US military is directly involved in microfinance (Iraq & Afghanistan), working primarily through Provincial Reconstruction Teams (PRTs). Funded by the DoD and DoS; the operational requirements of these agencies create a need to demonstrate quick impact on economic recovery and therefore the goal is to report high numbers of loans.

Technical complexities separate this data from other datasets: Heterogeneous forms: structured/unstructured/nominal,ordinal, quantitative/temporal/geographic/multi-lingual/multiple relationships(lenders to recipients) - multiple sectors/missing data. Data cleansing is hard!Big Data(ish): $420M (USD), 1.1 million lenders, 580,000 loans, 250 partners, 4.1M transactions, 3 WHOLE GBs. (https://vimeo.com/28413747)

Broad appeal:...government to defense to finance to banking to non-profit organizations to THE POOR.






https://vimeo.com/28413747

https://vimeo.com/28413747



75 views (8 days)#3 in last month








• 2,500 files with a total size of 3.6GB uncompressed• 3 Folders extracted from the Kiva JSON Snapshot

◦ lenders■ 1000 user accounts per file x 1,100 files (367 MB)■ user id, name, country, join date, loan count, occupation, user entered “whereabouts” (city, state), personal URL, and invitee

information■ All files were loaded directly into a database table

◦ loans■ 500 loans per file x 1,100 files (3 GB)■ Each loan object outlines:

■ loan ID, status■ activity and sector, example: butcher shop, food■ borrowers names and genders (one to many relationship)■ date of and amount of loan■ description of the planned use, sometimes in multiple languages (one to many relationship)■ currency exchange information■ lng/lat location and the accuracy level (country, city, town, etc.)■ payment information: date, amount (local and USD), and borrower comment (one to many relationship)■ terms: loan amount and payment history in native currency (one to many relationship)

■ Files loaded into 9 tables to accommodate the one to many relationships■ The main loans table links to all the other tables using the loan ID■ Location information separated and added to a PostGIS table to enable shapefile exports or imports into GIS software

◦ loan_lenders■ 1000 entries per file x 120 files (92 MB)■ Lists each loan ID with the user IDs that have contributed■ All files loaded into a single table that links users to loans and borrowers

• Lines of Python code: 380• Lines of SQL code: 186













PostgreSQL: analytics.data-tactics-corp.com Port: 5432


Shiny Sever: http://analytics.data-tactics-corp:3838/MY-APP/

R Studio Sever: analytics.data-tactics-corp.com


Mendeley: http://www.mendeley.com/groups/3523171/_/




http://analytics

http://analytics







http://www.mendeley.com/groups/3523171/_/

http://www.mendeley.com/groups/3523171/_/

On the horizon...

The CRADA begins this fall semester (2013) and continues through the spring semester (2014) materializing in several ways:

A series of blogs hosted on the DT website tracking the progress of the program and subsequent analyzes,

Maturing analytics infrastructure and sophistication of microfinance data (including but not limited to Kiva),

A journal quality paper with USMA and DT authorship,

A conference proceeding - Spring/Summer 2014 (forum yet to be determined),










Don’t forget...

Data Tactics Analytics Brown Bag:

When: Thursday, August 22, 2013 11:30 AM-1:30 PM

...food will be served.

Where: DT Conference Rooms D & T.






Data & Analytics

Microfinance : Data Tactics and USMA West Point Summer Camp 2013