Fuzzy Logic & Composition - Dartmouth College

Peter Elsea 2/11/14 1

Some Applications of Fuzzy Logic in the Composition of Music

What can Fuzzy Logic do for composers? Fuzzy Logic is an attractive way to build expert musical systems because fuzzy algorithms can closely parallel the methods a composer follows in intuitive composition. Any process a composer can define as a set of rules, however vague, can be modeled with a fuzzy logic routine. The following points are also in Fuzzy Logic's favor:

• Although the origin and continued development of Fuzzy Logic is based in advanced mathematics, the practical application of fuzzy methods requires only basic math operations.

• Fuzzy routines are easy to modify and expand, so varied treatments of a given set of inputs can be quickly produced.

• Fuzzy routines are easy to debug. The simple expedient of turning rules off and on will quickly isolate problem areas.

• Fuzzy routines are lightweight and execute quickly in most languages, making the fuzzy approach ideal for real time applications.

• Fuzzy routines can easily be mixed with stochastic and probabilistic methods to create surprising results.

Representation of music with fuzzy sets

Pitch Pitch can be represented in two ways, depending on the situation. Inputs and outputs to fuzzy routines will usually be the familiar pitch class or list of pitch classes, which I will call a pitch list, or often just a list. For final performance or transcription, pitch lists will usually be expanded to MIDI pitches. I'll write such lists within brackets, such as [0 4 7]. Within a fuzzy routine, pitches and other structures will be represented as sets in the domain of pitch class, as shown in figure 1. There are twelve1 elements, and each element represents the membership of the associated pitch class in the set. Figure 1 shows how a single pitch "G" would be represented.

Figure 1.

1 In some circumstances I use extended sets which cover two or three octaves. I don't plan on discussing those in this paper, however.


To save space (not to mention the tedium of creating graphics for things that can be typed), the sets will be shown as a list of memberships within braces as: G ==> {0 0 0 0 0 0 0 1 0 0 0 0} The pitches represented by an element will always be the index of the element, starting with 0. With only one non zero value, sets that represent only one pitch are singletons. However, a set can represent chords of any complexity. Here are a few common chords for practice: Cmaj ==> {1 0 0 0 1 0 0 1 0 0 0 0} Dmin ==> {0 0 1 0 0 1 0 0 0 1 0 0} Bdim ==> {0 0 1 0 0 1 0 0 0 0 0 1} Bmaj ==> {0 0 1 0 0 0 1 0 0 0 0 1}

You can see how the patterns in the sets echo the position of the pitches on a keyboard.

Defining Scales When we work within a defined tonality, we will often derive structures from the scale. A complete scale2 is represented thus: C major scale ==> {1 0 1 0 1 1 0 1 0 1 0 1} D major scale ==> {0 1 1 0 1 0 1 1 0 1 0 1} C harmonic minor scale ==> {1 0 1 1 0 1 0 1 1 0 0 1}

Fuzzy time Fuzzy logic can work with time as well as pitches. In the system used here3, time is defined as ticks, which have a default duration of 1 ms. A quarter note has a duration of 1000 ticks, which implies a default tempo of 60. Changes in tempo are accomplished by changing the duration of the tick. Beats and other subdivisions will be handled with fuzzy numbers, and concepts such as "early in the piece" will be defined by fuzzy membership functions. Musical notes are coded as "events" a five part data structure consisting of:

• On-‐time in ticks • MIDI pitch value • Duration in ticks • MIDI channel number • MIDI velocity value

2 The examples are in common tonality, but fuzzy logic will work as well in any tonality. 3 It's actually David Cope's system


Options for coding Even though there are fuzzy logic packages for every known language, I encourage you to write your own version. Musical fuzzy does not fit well with standards like FCL, so you would have to code several missing routines anyway, and there are dozens of routines in the standard packages you will never need. Fuzzy sets are easily coded as arrays in languages like java and C or as lists in Max and Lisp. The operations needed should be clear from the following discussion.

Basic operations A fuzzy logic toolbox for music will require routines for the basic fuzzy operations as well as some that are unique to musical situations. For the purposes of this paper, I will refer to these routines by name4 instead of conventional math symbols. Some of the names are idiosyncratic, (ingrained after 20 years of use) but most should be clear. Here are the standard operations I use:

• Intersect( setA setB) returns the fuzzy intersection of sets A and B, i.e. the minimum value found for each element in both.

• Union( setA setB) returns the fuzzy union of sets A and B, the maximum of each element.

• Complement( setA) returns the fuzzy compliment of setA, each element subtracted from 1.

• Normalize( setA) returns a set with all members proportionately adjusted so the peak value is 1.

• Clip( setA valueN) returns a set with all members of setA that exceed valueN set to valueN.

• Membership( setA index) returns the value found at index in setA. If index is fractional, a value interpolated from previous and following members is returned.

• Find( setA value) returns the index of the first occurrence of value in setA. If memberships are found that bracket value, an interpolated index is returned.

• Concentrate( setA) returns a set with all members set to the square of the associated member of setA.

• Dialate( setA) returns a set with all members set to the square root of the associated member of setA.

• Centroid( setA) returns the index of the "center of gravity" of setA. The following are functions unique to my musical applications.

• RoR(n setA ) returns a copy of setA rotated by n steps to the right. In a right rotation the rightmost n elements are moved to the left side of the result, viz. RoR(2 {A B C D E F} ) ==> {E F A B C D}. If n is negative, rotation is to the left.

4 I have libraries of these functions in LISP and Max externals, with names that are similar but follow the naming conventions of those languages.


• Top(n setA ) returns a list of the indices of the highest n values in setA. If there are multiple values at the maximum (a plateau), the first n indices are returned.

• SetToList( setA) returns a list of all non-‐zero members of setA. • MakeSet (n,m.....) returns a 12 element set with members at index n, m and so

forth set to 1. • Dither( l, n) returns a set of length l with a randomly chosen element set to n. • Add lists returns a set which contains the sums of the elements of two lists;

i.e. addlists({1 2 3},{4 5 6}) returns {5 7 9}. • Add-‐to-‐n(setA n m) returns a set with the membership at index n increased

by m, but restricted to be between 0 and 1. • Sumup returns the sum of all members of a list.

Other functions will be described as needed.

Transpositions The first musical operation I have addressed in fuzzy operations is transposition. This is actually fairly difficult in most approaches because of the modulo 12 nature of pitch classes. However, any structure represented as a fuzzy set can be transposed by the rotation operation. Thus if we define a pitch C as a singleton on index 0, rotation by the desired interval produces any desired pitch. Ror 7 {1 0 0 0 0 0 0 0 0 0 0 0} = {0 0 0 0 0 0 1 0 0 0 0 0} Transposing any pitch works as well: Ror 7 {0 0 0 0 0 0 0 0 0 1 0 0} = {0 0 0 0 1 0 0 0 0 0 0 )} As do transpositions down: Ror -1 {1 0 0 0 0 0 0 0 0 0 0 0} = {0 0 0 0 0 0 0 0 0 0 0 1} This works with chords: Ror 7 {1 0 0 0 1 0 0 1 0 0 0 0} = {0 0 1 0 0 0 0 1 0 0 0 1} Even scales: Ror 7 {1 0 1 0 1 1 0 1 0 1 0 1} = {1 0 1 0 1 0 1 1 0 1 0 1 } The rotate operation is used heavily through my system.


Building intervals The operations discussed have been easily possible with crisp logic-‐-‐ in fact with the overhead involved in producing sets, rotation seems more cumbersome than simple math. However, music theory becomes fuzzy when we get to tonal intervals. The expression "a third above C" can produce two different results depending on context. In the simple case, the context is provided by the active scale (key). The full statement should be something like "a third above C in D major." To handle tonal intervals, I have defined a fuzzy set to represent the concept of "a third above" Fz3 := {0 0 0 1 0.9 0 0 0 0 0 0 0} This will generate the proper third when intersected with the current scale. Somewhere in my code is a variable (usually named key) that holds a set representing the current scale. To find a third above C in C major: Cmaj {1 0 1 0 1 1 0 1 0 1 0 1} Fz3 {0 0 0 1 0.9 0 0 0 0 0 0 0} Intersect {0 0 0 0 0.9 0 0 0 0 0 0 0} Top 4

Here the intersection has produced a set with element 4 set to 0.9. The top operation will identify that as an E. In different keys, we need different results. To find a third above C in Eb major: Ebmaj {1 0 1 1 0 1 0 1 1 0 1 0} Fz3 {0 0 0 1 0.9 0 0 0 0 0 0 0} Intersect {0 0 0 1 0 0 0 0 0 0 0 0} Top 3 Note that the Eb scale is a rotation of the C scale. Rotation is also used to find intervals on other roots. Here is a third above D in Cmajor Cmaj {1 0 1 0 1 1 0 1 0 1 0 1} Fz3 ror 2 {0 0 0 0 0 1 0.9 0 0 0 0 0} Intersect {0 0 0 0 0 1 0 0 0 0 0 0} Top 5 The set for "a third above D" is a rotation of "a third above C" by 2. Note that the memberships in FZ3 are not equal-‐ this favors the minor third when both are available in the scale. That does not come up in major keys, but is possible in some minor scales: Charm { 1 0 1 1 0 1 0 1 1 0 0 1} Fz3 ror 8 {0.9 0 0 0 0 0 0 0 0 0 0 1} Intersect { 0 0 0 0 0.9 0 0 0 0 0 0 0}


Top 11 (In point of fact, the harmonic rules of minor modes are so fluid that factors other than the base scale must be considered. Thus I define an alternate fuzzy third that favors the major version.) Using these principles, I have developed fuzzy sets for all intervals up and down. I usually define these as constants. intervals up Fz2 {0 0.9 1 0 0 0 0 0 0 0 0 0} Fz3 {0 0 0 1 0.9 0 0 0 0 0 0 0} Fz3a {0 0 0 0.9 1 0 0 0 0 0 0 0} Fz4 {0 0 0 0 0 1 0.5 0 0 0 0 0} Fz5 {0 0 0 0 0 0 0.9 1 0.5 0 0 0} Fz6 {0 0 0 0 0 0 0 0 1 0.9 0 0} Fz7 {0 0 0 0 0 0 0 0 0 0 0.9 1} intervals below Fz2b {0 0 0 0 0 0 0 0 0 0 1 0.9} Fz3b {0 0 0 0 0 0 0 0 0.9 1 0 0 } Fz4b {0 0 0 0 0 0 0.5 1 0 0 0 0 } Fz5b {0 0 0 0 0.5 1 0.9 0 0 0 0 0 } Fz6b {0 0 0.9 1 0 0 0 0 0 0 0 0 } Fz7b {0 0.9 1 0 0 0 0 0 0 0 0 0 }

Building chords If you look at the fuzzy 5ths in the above tables, you will note that there are three possibilities: diminished, perfect and augmented. These are used in building chords. In the base case, a chord is the union of the root, the third above the root and the fifth above the root. First state the root as a singleton: Root C {1 0 0 0 0 0 0 0 0 0 0 0} Then find the 3rd Cmaj {1 0 1 0 1 1 0 1 0 1 0 1} Fz3 {0 0 0 1 0.9 0 0 0 0 0 0 0} 3rd above C {0 0 0 0 0.9 0 0 0 0 0 0 0} And the fifth Cmaj {1 0 1 0 1 1 0 1 0 1 0 1} Fz5 {0 0 0 0 0 0 0.9 1 0.5 0 0 0} 5th above C {0 0 0 0 0 0 0 1 0 0 0 0}


Union {1 0 0 0 0.9 0 0 1 0 0 0 0} Top 3 [0 4 7] To find chords on any pitch, rotate the root and both interval sets by the pitch class desired. This will produce the proper triad in any key. Finding the chord on the 7th degree is usually the most complex operation in other languages. With fuzzy, everything falls into place. Here's how: Root ror 11 {0 0 0 0 0 0 0 0 0 0 0 1} Then find the 3rd Cmaj {1 0 1 0 1 1 0 1 0 1 0 1} Fz3 ror 11 {0 0 1 0.9 0 0 0 0 0 0 0 0} 3rd above B {0 0 1 0 0 0 0 0 0 0 0 0} And the fifth Cmaj {1 0 1 0 1 1 0 1 0 1 0 1} Fz5 ror 11 {0 0 0 0 0 0.9 1 0.5 0 0 0 0} 5th above B {0 0 0 0 0 0.9 0 0.5 0 0 0 0} Union {0 0 1 0 0 0.9 0 0.5 0 0 0 1} Top 3 [2 5 11] Note that the top 3 operation extracts a triad from the union of the intervals. It is also possible to apply the top operation to the interval result sets individually and list them with the root. In fact I usually do this, adding functions for each interval to my library. In following examples I will refer to these functions rather than spell out all of the sets. They take the form thirdAbove(r) where r is the root, and return a pitch class. Thus the general form for a triad built on root r is List(r,thirdAbove(r), fifthAbove(r)) One advantage of this code is the pitches are returned in order of root, third, fifth. More complex chords can be derived by including additional intervals. Chords of the seventh which are natural to the scale can be derived by adding an interval of a fifth above the third of a triad constructed using either of the above algorithms. List(r, thirdAbove(r), fifthAbove(r), fifthAbove(thirdAbove(r))) When non-‐scale seventh chords are desired, they can be created directly by rotation of a template set: Dom7 = {1 0 0 0 1 0 0 1 0 0 1 0}


Chords from root, third, fifth The above chords were built with a given pitch as the root. As we look into harmonization, we also need to be able to build chords where the given pitch is the third or fifth. To do this, we use the fuzzy sets for interval below. To review that, here's how the sets work to generate a third below G in C major: Cmaj {1 0 1 0 1 1 0 1 0 1 0 1} Fz3b ror 7 {0 0 0 0.9 1 0 0 0 0 0 0 0} 3rd below G {0 0 0 0 1 0 0 0 0 0 0 0} Top 4 Note that even though we are looking for pitch below the given pitch the "interval below" set is rotated to the right. Now to generate a triad with a pitch p as the third: List(thirdBelow(p), p, fifthAbove(thirdBelow(p)) Note that the fifth is generated from the root once we know what the root is. Likewise, to produce a chord with the given pitch as the fifth: List(fifthBelow(p), thirdAbove(fifthBelow(p), p) Of course functions that produce these would store values like thirdBelow(p) in a local variable for efficiency. Don't be fooled by the fact that these examples are basic tonal harmonies. You can create exotic yet consistent harmonies by using different interval combinations or by defining your own scales.

Simple harmonizer With the fundamental chord constructors available, we can turn our attention to deciding what chords to ask for. As an example, here is a simple harmonizer that does little more than select appropriate chords to harmonize individual notes in a melody. (I call this "left hand harmony" after an elementary school teacher who would conduct a singing class with one hand while hitting chords on a piano with the other.)

Common tone rules The underlying principle of this harmonizer is to choose chords that maintain common tones. This is achieved with these rules:

• If a chord with the melody note as root has common tones with the previous chord, use the chord with the melody note as root.

• If a chord with the melody note as third has common tones with the previous chord, use the chord with the melody note as third.

• If a chord with the melody note as fifth has common tones with the previous chord, use the chord with the melody note as fifth.


Last chord rules But we also want to mix these up-‐-‐ otherwise we'll always get repeated chords when the melody has repeated notes or motion of third or fifth.

• If the last chord was built on the root, use a chord built from a third or fifth. • If the last chord was built on the third, use a chord built from a root or fifth. • If the last chord was built on the fifth, use a chord built from a root or third.

The fuzzy mechanism for these rules creates a solution set with three members. You can think of the members as { desirability of using the note as root, desirability of using the note as third, desirability of using the note as fifth} or {as-‐root, as-‐third, as-‐fifth}. The operation of the common tone rules is simple. The number of pitches shared by the possible chord and the previous chord are counted and divided by three. Then a solution set is generated with the membership of the possible chord set to that value. So if the as-‐root chord has two pitches in common with the previous chord the set { 0.6 0 0} is returned. The number of common tones is found by intersecting the previous chord set with the candidate chord set and counting the 1s. Say the pitch is G and the previous chord was C major: {1 0 0 0 1 0 0 1 0 0 0 0} previous {0 0 1 0 0 0 0 1 0 0 0 1} G as-root 1 Common =>{0.3 0 0} {0 0 0 0 1 0 0 1 0 0 0 1} G as-third 2 Common =>{0 0.6 0} {1 0 0 0 1 0 0 1 0 0 0 0} G as-fifth 3 Common =>{0 0 1} Union and normalize => {0.3 0.6 1} If the new pitch were A: {1 0 0 0 1 0 0 1 0 0 0 0} previous {1 0 0 0 1 0 0 0 0 1 0 0} A as-root 2 Common =>{0.6 0 0} {1 0 0 0 0 1 0 0 0 1 0 1} A as-third 2 Common =>{0 0.6 0} {1 0 1 0 0 1 0 0 0 1 0 0} A as-fifth 0 Common =>{0 0 0} Union and normalize => {0.6 0.6 0} The last chord rules are controlled by the solution set from the last time around. The code remembers what happened and generates one of these: As-root => { 0 1 0.5} As-third => { 1 0 0.5} As-fifth => { 1 1 0 } Note that I'm giving less weight to the possibility of deriving the chord with the pitch as the fifth. This is a blatant example of the composer's thumb on the scales.


The common tone rules and last chord rules each produce a solution, and the two are added together to get the result5. You can think of it as a voting process. So a Cmaj chord on a C followed by a G in the melody would work out like this: {0.3 0.6 1 } {0 1 0.5} {0 1 1.5} The top function will pick the highest member of the solution set. Here the chord would be constructed with G as the fifth, so we'd get a continuation of the Cmaj chord. If the new melody pitch were A, we'd have: {0.6 0.6 0 } {0 1 0.5} {0.6 1.6 0.5 } This would generate an Fmaj chord. Figure 2 is an example of these rules on a simple scale.

Figure 2. The top line in figure 2 shows the given pitches and the bottom line shows the generated chords. The ascending scale forced an alteration between as-‐root and as-‐fifth because moving up a step leaves no common tones between as-‐root and as-‐third. The melody of figure 3 jumps a round a bit more and includes a note foreign to the scale.

Figure 3. 5 This will result in a set with memberships greater than one, but since we are just going with the highest member in the final result, normalizing is a waste of code.


In figure 3 we see all three possibilities in the first four chords. We also see that a foreign pitch does not break the system, simply finding intervals from the scale. Admittedly, a tritone step is not the optimum resolution here. That results from the fact that the rules so far have some memory but no notion of what is coming up. We can add a rule that encourages harmonic motion of a fifth that is similar to the keep common tones rule:

• If the as-‐root chord has common tones with the dominant of the following pitch, use as-‐root

• If the as-‐third chord has common tones with the dominant of the following pitch, use as-‐third

• If the as-‐fifth chord has common tones with the dominant of the following pitch, use as-‐fifth

This set is normalized as a group so it has equal weight as the common tone and last chord rules. Adding this rule produces figure 4. Not only has the F# been incorporated in a Dmaj, many of the other chords have been changed to be the dominant of the following melody pitch. You can easily add as many extra rules as you like to this system, just adding their result into the solution set. In fact, There is one more unmentioned rule in play here:

• if pitch is tonic, favor tonic chord. That is responsible for the final Cmaj.

Figure 4. These examples show that appropriate chords are chosen. The next task is to break up all of the parallel fifths with some inverted chords.

Voice leading Once the chords are chosen, voice leading can be improved by mixing inversions6. Inversions are easily produced by rotation of a normal ordered pitch list. Note that rotating a chord one step produces the second inversion: {0 4 7} >> 1 => {7 0 4} {0 4 7} >> 2 => {4 7 0}

6 Do not mix up the notion of generating a chord with the given pitch as the third with the inversion of that chord with the third on the bottom.


Here are some rules:

• sustain common tones • if too many root, then 6 or 6-‐4 • if too many 6 then root or 6-‐4 • do not follow 6-‐4 with 6-‐4 • if motion is fifth, favor root • If pitch is tonic, favor root a bit

The solution set for the three possible inversions is similar to the one for the chord options: {root, 6-‐4, 6}. The sustain common tones code is practically identical The repetition rules require fuzzy definitions of each case of "too many" TOO-MANY-ROOT= {0 0.2 0.6 1 1 1 1 1) TOO-MANY-6 = {0 0.2 0.6 1 1 1 1 1 1) TOO-MANY-6-4 = {0 1 1 1 1 1 1 1 1 1) Note that I have little tolerance for the 6-‐4 chord in this kind of harmony. The code counts occurrences of each inversion in a row and pulls out the membership in the appropriate set. This is used to weight the consequent set. For instance the consequent set of "too-‐many-‐root" is {0 0.8 1} if the membership in {too-‐many-‐root} is 0.7, the consequent becomes {0 0.64 0.8}. Note that this rule never encourages a root position chord to follow a root position chord. Root to Root movement will only happen as the result of other rules.

Figure 5. Figure 6 puts these rules into action with the scale passage of figure 2.

Examples

Figure 6. Figure 6 has a fairly satisfactory harmony, with nothing that stands out as glaringly wrong. In figure 7 we see the melody of figure 3:


Figure 7. Note the sustained chord on the first measure where the melody moves down a third. This is a result of the keep common tones rule.

Other fuzzy rules There's a bit more happening here than I have discussed so far. These are cosmetic issues designed to make the chords look good in this particular scoring program (MuseScore). One issue is simply choosing the octave for the chord. If we just keep the bottom note on a chosen octave, a progression like Croot -‐G6 will look odd because the G would be a seventh above the C. To solve this problem, I've added code to choose octaves. There are three options for placement: low mid high

• If mid is close to previous choose mid • If high is close to previous choose high • If low is close to previous choose low • If previous is near bottom of clef choose mid or high • If previous is near top of clef choose mid or low

We also have to consider the problem of initial conditions. Since several of the rules are based on the previous chord or inversion, the code must have some provision to set the appropriate variables before generating any harmony. These settings can have surprising effects on the resulting harmony.

Examples: Here is how the system harmonizes some simple tunes.

London Bridge


Lullaby

Amazing Grace This paper is not intended to be a prescription for generating harmony, but rather to demonstrate the application of fuzzy logic to musical processes. Since composition is often concerned with issues like "if too many" and context based rules, fuzzy tools are a powerful to any algorithmic toolbox.

Documents

Fuzzy Logic & Composition - Dartmouth College