View
0
Download
0
Category
Preview:
Citation preview
Roles of Data Mining and Machine Learning
in Computational Creativity PROSECCO Tutorial about Computational Creativity
ICCC 2016, Paris
Hannu Toivonen, University of Helsinki, Finland hannu.toivonen@cs.helsinki.fi
Based on joint work with Oskar Gross 27/06/16 1
www.helsinki.fi/yliopisto
– A purely preprogrammed generative system – only does what it was told to do – has little if any creativity
– Adaptivity or self-determinism – is necessary to attribute any originality, responsibility or
creative autonomy to a creative system – Transformative or meta-level creativity (cf. Boden,
Wiggins) can be attributed with higher creativity – …but how to build a system to deal with unanticipated
cases? → Opportunities for ML and DM
27/06/16 2
The opportunity for Data Mining (DM) and Machine Learning (ML)
www.helsinki.fi/yliopisto
– To highlight some possible ways of using DM and ML in computational creativity – (This is not a tutorial on DM and ML methods)
– A limited number of examples will be given – More information, examples and references can be
found in our review paper: Hannu Toivonen and Oskar Gross: Data Mining and Machine Learning in Computational Creativity. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery 5 (6): 265-275. 2015. (link)
27/06/16 3
Goals of this talk
www.helsinki.fi/yliopisto
– Covered already by Wiggins in the previous parts of the tutorial – What is creativity, computational creativity – Creativity as search, transformational creativity
– As a recap, consider the definition of creative autonomy by Jennings (2010)
27/06/16 4
Background on CC
www.helsinki.fi/yliopisto
1. Autonomous Evaluation The system can evaluate its liking of a creation without seeking opinions from an outside source. - Any opinion is formed by the system itself - However, it may consult others at other times - Examples: preprogrammed evaluation, evaluation
function learned from the user
Criteria for Creative Autonomy (1/3), Jennings (2010)
www.helsinki.fi/yliopisto
2. Autonomous Change The system initiates and guides changes to its standards without being explicitly directed when and how to do so. - External events and evaluations may prompt and guide
changes - The system decides when and how to change them - The system decides if new standards are acceptable - Fixed or learned evaluation functions can be used to
bootstrap the process
Criteria for Creative Autonomy (2/3)
www.helsinki.fi/yliopisto
3. Non-Randomness The system’s evaluations and standard changes are not purely random. - The two first criteria could be easily met by random
decisions - Not all randomness is excluded, however
Criteria for Creative Autonomy (3/3)
www.helsinki.fi/yliopisto
– Let’s use a simple generate-and-test model to illustrate uses of machine learning and data mining in CC (computational creativity)
– Assessing one’s own output is an elementary creative responsibility (and Jennings’ 1st criterion)
27/06/16 8
Illustrative setup: The generate-and-test model
gen() eval(a) Artefact a Ok?
no
yes
Learning to evaluate
27/06/16 9
gen() eval(a) Artefact a Ok?
no
yes
www.helsinki.fi/yliopisto
– Use ML to learn an evaluation function eval(a) from training examples – Supervised learning (from given examples) – E.g. a classifier that tells if the result is good
– Assuming a generator gen() exists, its outputs can now be filtered by the trained classifier without explicit directions by the programmer – More responsibility over the evaluation step – Fulfills Jenning’s first criteria of creative autonomy
27/06/16 10
Learning to evaluate
www.helsinki.fi/yliopisto
An example system: DARCI (Ventura et al) – Creates images that express an emotion – Emotion detection is based on artificial neural
networks trained by users of the system – A genetic algorithm is used as generator gen()
– Adapts to the evaluation/fitness function eval()
– http://darci.cs.byu.edu/ – ”DARCI, draw me a happy picture!”
27/06/16 11
Learning to evaluate
www.helsinki.fi/yliopisto A happy image by DARCI, http://darci.cs.byu.edu/ 27/06/16 12
www.helsinki.fi/yliopisto
Bottlenecks in learning the eval() function – Learning an evaluation (or fitness) function eval(a)
can be very difficult – E.g., how does one evaluate the quality of a poem?
– Generating complex artefacts, i.e., producing the function gen(), can be very hard – In practice, the generation step must be adaptive in
order to be effective
– Can lead to pastiche generation, i.e., mere imitation of training examples rather than creativity
27/06/16 13
Learning to evaluate
Learning to generate
27/06/16 14
gen() test(a) Artefact a Ok?
no
yes
www.helsinki.fi/yliopisto
Roughly speaking, two kinds of models can be obtained using machine learning methods: A. A predictive (or discriminative) model can label
given instances – E.g. a neural network or a support vector machine can
categorize a given example to one of the trained categories
B. A generative model can produce new instances – E.g. a Bayesian model for the joint distribution of all
values can be used to generate new instances from the joint distribution
27/06/16 15
Learning to generate
www.helsinki.fi/yliopisto
1. Completion of partial artefacts
– Given some part of the artefact, predict the values of the remaining parts
– Can be based on training on complete artefacts Example: harmonization of music: – Given a melody (possibly created by the system itself),
choose suitable chords to accompany the melody
27/06/16 16
A. Learning to generate using predictive models
www.helsinki.fi/yliopisto
2. Reduce the task of generating complex structures to selection.
Example (revisited): Generation of accompaniment for a melody by running a classifier to pick a suitable chord as well as a suitable pattern
27/06/16 17
A. Learning to generate using predictive models
www.helsinki.fi/yliopisto
3. Generate complex structures using instance-based techniques
– E.g. k-nearest neighbours and case-based reasoning – avoids using models, decision structures, or patterns
‒ models could be difficult to specify or learn
‒ models could be restrictive
27/06/16 18
A. Learning to generate using predictive models
www.helsinki.fi/yliopisto
Example: Corpus-based poetry by Toivanen et al. – Background: typical inputs to a poetry writing system
a. linguistic knowledge, e.g. a grammar or templates b. world knowledge, e.g. a knowledge base
– Poetry generation by Toivanen et al a. instance-based generation (copying) of grammar
using a large corpus of poetry b. obtains statistical world knowledge automatically by
mining patterns from (another) corpus
27/06/16 19
A. Learning to generate using predictive models
www.helsinki.fi/yliopisto
– Example of operation of the method (translation from Finnish)
– An original piece of text is selected (randomly): – how she once played in a big, green park […]
– Topic for new poem: (children’s) play –
27/06/16 20
A. Learning to generate using predictive models
she then whispering played daring in a daring, how under the pale trees. She had heard for fun how her whispering drifted as jingle to the wind.
www.helsinki.fi/yliopisto
Music swells, accent practises, theatre hears! Her delighted epiphanies bent in her universe: – And then, singing directly a universe she disappears! An anthem in the judgements after verse!
27/06/16 21
www.helsinki.fi/yliopisto
Generative models can be used more directly to generate artefacts (or to complete partial ones) – E.g. Markov models for sequencies such as text and
music – (Bayesian) models of joint probability distributions – Some artificial neural networks
27/06/16 22
B. Learning to generate using generative models
www.helsinki.fi/yliopisto
An example: Deep Dream by Google – A neural model trained with existing images – Learns their features and can generate (or
complement or re-render) images – http://deepdreamgenerator.com
27/06/16 23
B. Learning to generate using generative models
www.helsinki.fi/yliopisto 27/06/16 24
www.helsinki.fi/yliopisto 27/06/16 25
Mining patterns for creative tasks
27/06/16 26
www.helsinki.fi/yliopisto
Basic idea: 1. Use data mining to discover patterns in, say, text 2. Utilize these patterns in a generation function gen() Examples: – Corpus-based poetry of Toivanen et al
– Words used as substitutes come from text mining
– Metaphor generation (Veale et al)
27/06/16 27
Mining patterns for creative tasks
www.helsinki.fi/yliopisto
Example: metaphor generation (Veale et al) – Extraction step: extract similes from a corpus
– Look for patterns of the form “as P as a/an V”, e.g. “as strong as an ox”
– “strong” is a stereotypical property of “ox” if the pattern “as strong as an ox” occurs often
– Metaphor construction step: use stereotypical information – To express that ”John is stupid” in a metaphorical way, pick
a noun V for which ”stupid” is a stereotypical property – Donkey is found as stereotypically stupid – Compare John to donkey to express that he is stupid
http://ngrams.ucd.ie/metaphor-eye/
27/06/16 28
Mining patterns for creative tasks
www.helsinki.fi/yliopisto
Metaphor-Eye Why are scientists like artists? – Scientists
– …develop ideas like artist – …explore ideas like artist – …acquire skills like artist – …spread ideas like artist – …nurture ideas like artist – …develop techniques like artist
27/06/16 29
Learning in transformational creativity
27/06/16 30
www.helsinki.fi/yliopisto
– U: universe of all possible (partial) artifacts – R: rules that define the search space – E: rules that evaluate an artifact – TR,E: traversal rules to search for artifacts that satisfy
R and E
– Transformational creativity takes place when R, E or T is modified by the system itself
– Next: a quick review of issues and opportunities for ML, proposed by Wiggins
27/06/16 31
Recall Wiggins’ creative search
www.helsinki.fi/yliopisto
Wiggins suggests uses of ML/DM: – Automatic adaptation of R or T
– To remedy aberration: use aberrant concepts as positive or negative examples, depending on their value
– To remedy generative uninspiration: use positive (and negative) examples received from outside
– Automatic adaptation of E – Use feedback and evaluations received from outside
(not covered by Wiggins)
Transformational Creativity Using Data Mining and Machine Learning
www.helsinki.fi/yliopisto
– “Generative uninspiration”: TR,E does not reach anything valued by E
– A milder form: a lot of (highly) valued concepts are unreachable by TR,E
– Transformation of TR,E is required – Help from outside is needed, e.g., valued (and
unvalued) concepts not reachable to TR,E – use them as positive (and negative) training examples
to modify TR,E
Generative uninspiration
www.helsinki.fi/yliopisto
– “Aberration”: TR,E reaches concepts outside R – “Productive aberration”: TR,E reaches some valued
concepts outside R – “Pointless aberration”: the concepts outside R are
not valued by E either
– Need to transform T to avoid useless search – Possibly transform R to include the valued concepts – Learning: use aberrant concepts as positive or
negative examples, depending on their value
Aberration
Conclusion
27/06/16 35
www.helsinki.fi/yliopisto
– Learning to evaluate artifacts – Learning to generate artifacts
– possibly by reducing generation to selection, or to a sequence of predictive steps
– Mining patterns to be used in generation – Doing these adaptively also during the runtime – Also learning/adapting search space R (if possible) – Learning to frame (cf. FACE by Colton et al) – Higher meta-levels: Learning to produce new
generation, evaluation and search functions
27/06/16 36
Examples of creative responsibilities assumed by DM/ML
www.helsinki.fi/yliopisto
For more information and citations see the following two survey papers: – Hannu Toivonen and Oskar Gross: Data Mining and
Machine Learning in Computational Creativity. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery 5 (6): 265-275. 2015.
– Ping Xiao, Hannu Toivonen, et al: Review of Conceptual Representations for Concept Creation. Manuscript.
27/06/16 37
Literature
www.helsinki.fi/yliopisto
– Margaret A. Boden: Creative Mind: Myths and Mechanisms (2nd ed.). Routledge, Oxon, UK, 2004 (1st edition:1992)
– Simon Colton and Geraint A. Wiggins: Computational Creativity: The Final Frontier? ECAI 2012 - 20th European Conference on Artificial Intelligence, 21-26, Montpellier, France, August 2012.
– Wiggins, Geraint A : A preliminary framework for description, analysis and comparison of creative systems. Knowledge-Based Systems, 19 (7), 449—458, 2006.
– K.E. Jennings: Developing creativity: Artificial barriers in artificial intelligence. Minds and Machines 20(4): 489-501, 2011.
Some key references
Recommended