Deep LearningZachary C. Lipton

zacklipton.com

Critical Review of RNNs: http://arxiv.org/abs/1506.00019

Conditional Generative RNNS:http://arxiv.org/abs/1511.03683

Computation Revolves Around Functions

Computation Revolves Around Functions

“Time underlies many interesting human behaviors”

{0,1} {A,B,C…} captions …

Sometimes We Need to Learn These Functions From Data

Feedforward Neural Network

Recurrent Neural Networks

RNN Generative Model

Shakespeare GenerationPANDARUS:Alas, I think he shall be come approached and the day When little srain would be attain'd into being never fed, And who is but a chain and subjects of his death, I should not sleep.

Second Senator:They are away this miseries, produced upon my soul, Breaking and strongly should be buried, when I perish The earth and thoughts of many states.

DUKE VINCENTIO:Well, your wit is in the care of side and that.

Second Lord: They would be ruled after this chamber, and my fair nues begun out of the fact, to be conveyed, Whose noble souls I'll have the heart of the wars.

Clown:Come, sir, I will make did behold your worship.

VIOLA:I'll drink it.

Training Data: All of Shakespeare

Size: 4.4MB

Text generated one character at a time!

Past Supervised Approaches relied upon Encoder-Decoder Model

Bridging Long Time Intervals with Concatenated Inputs

Example

A.5 FRUIT/VEGETABLE BEER <STR>On tap at the brewpub. A nice dark red color with a nice head that left a lot of lace on the glass. Aroma is of raspberries and chocolate. Not much depth to speak of despite consisting of raspberries. The bourbon is pretty subtle as well. I really don’t know that I find a flavor this beer tastes like. I would prefer a little more carbonization to come through. It’s pretty drinkable, but I wouldn’t mind if this beer was available. <EOS>

Character-based Classification

“Love the Strong Hoppy Flavor”