Neural Networks

Automatic Model Building (Machine Learning)

Artificial Intelligence

McGraw-Hill/Irwin ©2007 The McGraw-Hill Companies, Inc. All rights reserved

7-2

High-Growth Product• Used for classifying data

– target customers– bank loan approval– hiring– stock purchase– trading electricity– DATA MINING

• Used for prediction

McGraw-Hill/Irwin ©2007 The McGraw-Hill Companies, Inc. All rights reserved

7-3

Description• Use network of connected nodes (in

layers)

• Network connects input, output (categorical)– inputs like independent variable values in

regression– outputs: {buy, don’t} {paid, didn’t}

{red, green, blue, purple}

{character recognition - alphabetic characters}

McGraw-Hill/Irwin ©2007 The McGraw-Hill Companies, Inc. All rights reserved

7-4

Network

Input Hidden Output

Layer Layers Layer

Good

Bad

McGraw-Hill/Irwin ©2007 The McGraw-Hill Companies, Inc. All rights reserved

7-5

Operation• Randomly generate weights on model

– based on brain neurons• input electrical charge transformed by neuron• passed on to another neuron

– weight input values, pass on to next layer– predict which of the categorical output is true

• Measure fit– fine tune around best fit

McGraw-Hill/Irwin ©2007 The McGraw-Hill Companies, Inc. All rights reserved

7-6

Operation• Useful for PATTERN RECOGNITION• Can sometimes substitute for

REGRESSION– works better than regression if relationships

nonlinear– MAJOR RELATIVE ADVANTAGE OF NEURAL

NETWORKS:YOU DON’T HAVE TO UNDERSTAND THE MODEL

McGraw-Hill/Irwin ©2007 The McGraw-Hill Companies, Inc. All rights reserved

7-7

Neural Network Testing• Usually train on part of available data

– package tries weights until it successfully categorizes a selected proportion of the training data

• When trained, test model on part of data– if given proportion successfully categorized, quits– if not, works some more to get better fit

• The “model” is internal to the package

• Model can be applied to new data

McGraw-Hill/Irwin ©2007 The McGraw-Hill Companies, Inc. All rights reserved

7-8

Business Application• Best in classifying data

mortgage underwriting asset allocation

bond rating fraud prevention

commodity trading

• Predicting interest rate, inventoryfirm failure bank failure

takeover vulnerability stock price

corporate merger profitability

McGraw-Hill/Irwin ©2007 The McGraw-Hill Companies, Inc. All rights reserved

7-9

Neural Network Process1. Collect data

2. Separate into training, test sets

3. Transform data to appropriate units• Categorical works better, but not necessary

4. Select, train, & test the network• Can set number of hidden layers• Can set number of nodes per layer• A number of algorithmic options

5. Apply (need to use system on which built)

McGraw-Hill/Irwin ©2007 The McGraw-Hill Companies, Inc. All rights reserved

7-10

Marketing Applications• Direct marketing

– database of prospective customers• age, sex, income, occupation, education, location• predict positive response to mail solicitations

• THIS IS HOW DATA MINING CAN BE USED IN MICROMARKETING

McGraw-Hill/Irwin ©2007 The McGraw-Hill Companies, Inc. All rights reserved

7-11

Neural Nets to Predict Bankruptcy

Wilson & Sharda (1994)

Monitor firm financial performanceUseful to identify internal problems, investment evaluation, auditingPredict bankruptcy - multivariate discriminant analysis of financial ratios

(develop formula of weights over independent variables)Neural network - inputs were 5 financial ratios - data from Moody’s

Industrial Manuals (129 firms, 1975-1982; 65 went bankrupt)Tested against discriminant analysisNeural network significantly better

McGraw-Hill/Irwin ©2007 The McGraw-Hill Companies, Inc. All rights reserved

7-12

Ranking Neural NetworkWilson (1994)

Decision problem - ranking

candidates for position, computer systems, etc.

INPUT - manager’s ranking of alternatives

Real decision - hire 2 sales people from 15 applicants

Each applicant scored by manager

Neural network took scores, rank ordered

best fit to manager of alternatives compared (AHP)

McGraw-Hill/Irwin ©2007 The McGraw-Hill Companies, Inc. All rights reserved

7-13

CASE: Support CRMDrew et al. (2001), Journal of Service Research

• Identify customers to target

• Customer hazard function:– Likelihood of leaving to a competitor (CHURN)

• Gain in Lifetime Value (GLTV)– NPV: weight EV by prob{staying}– GLTV: quantified potential financial effects of

company actions to retain customers

McGraw-Hill/Irwin ©2007 The McGraw-Hill Companies, Inc. All rights reserved

7-14

Models:

• Proportional Hazards Regression

• Neural Networks– Estimate hazard functions

• Baseline Regression Models– Models for longitudinal analysis

McGraw-Hill/Irwin ©2007 The McGraw-Hill Companies, Inc. All rights reserved

7-15

Data: Data Warehouse of Cellular Telephone Division

• Billing– Previous balance, access charges, minutes used, toll charges,

roaming charges, optional features

• Usage– Number of calls, minutes by local, toll, peak, off-peak

• Subscription– Months in service, rate plan, contract type, date, duration

• Churn– Binary flag

• Demographics– Age, profitability to firm (current & future)

McGraw-Hill/Irwin ©2007 The McGraw-Hill Companies, Inc. All rights reserved

7-16

Model Use

• Sample of 21,500 subscribers, April 1998

• Modeled tenure for 1 to 36 months

• Trained on 15,000 of these samples– Remainder used for testing

• Neural network models worked better than traditional statistics

McGraw-Hill/Irwin ©2007 The McGraw-Hill Companies, Inc. All rights reserved

7-17

SystemsA great many products

• general NN products$59 to $2,000 @Brain BrainMaker Discover-It

• componentsDATA MINING along with megadatabases other products

• librarycallable

• specialty productsconstruction bidding, stock trading, electricity trading

McGraw-Hill/Irwin ©2007 The McGraw-Hill Companies, Inc. All rights reserved

7-18

Potential Value• THEY BUILD THEMSELVES

– humans pick the data, variables, set test limits

• CAN DEAL WITH FAST-MOVING SITUATIONS– stock market

• CAN DEAL WITH MASSIVE DATA– data mining

• Problem - speed unpredictable