STRUCTURE AND INTERPRETATION OF MUSIC CONCEPTS ______________________________________________ ============================================== CLASS 6: Modeling Music-Structure Rules and Constraints _______________________________________________________ ======================================================= The music concepts that we formulated include different kinds of constraints and rules. * The MUSIC-PIITCH-SEQUENCE concept, that serves as the basis for the MELODY-SEGMENT and the MELODY concepts. Narmour rules -- are sort of advice/characterization for melodic sequences. * The RHYTHM rules: hierarchy constructing rules + constraints, for those rhythms that are acceptable in tonal/atonal music. * The MELODY concept -- hierarchy constructing rules. Constraints on the hierarchy. The constraints can be applied only for fully instantiated hierarchy (with MELODY-SEGMENT leaves). * The Schoenberg serial music rules -- besides the serial operations, there are rules/advice for the music creation. Our goal is to observe various formal/quasi-formal models for capturing these characterizations. We first consider LANGUAGE specification models. Our goal: OBSERVE SIMILARITIES AND CHARACTERIZE DIFFICULTIES IN USING THESE MODELS FOR MODELING TH MUSIC CONCEPTS. LANGUAGE CONCEPTS: ================== ** A LANGUAGE is a set of sentences. In the area of Formal Language studies, a language is defined as a set of words.. ** A GRAMMAR is a formal model for characterizing the sentences/words of a language. * The grammar defines the language. * The grammar provides further information about the sentences/words. - structure. - semantics. - inter-relationships among parts. - ... ** A SENTENCE in a natural language is a sequence of WORDS. In formal language, the equivalent of a sentence is the WORD: a sequence of SYMBOLS. The words/symbols define the VOCABULARY of the language. ** A grammar defines the language using DERIVATIONS. ** FORMAL LANGUAGE GRAMMARS: A (context-free) grammar is a quadruple: (N, T, S, P) where N is the set of non-terminal symbols (categories). T is the set of symbols. S is the start symbol. P is a set of derivation (rewrite) rules. The vocabulary of symbols of a grammar is the union of the sets N and T. A derivation rule has the form: A -> w where A is a non-terminal symbol, and w is a word over N union with T. The LANGUAGE OF A GRAMMAR (denoted L(G)) is the set of all terminal words that can be derived (using the transitive closure of the derivation relation) from the start symbol S. Each derivation is associated with a single DERIVATION-TREE (also right-most/left-most derivations). The derivation tree assigns STRUCTURE to the derived sentence/word. Relevance of the structure: The structure implies the MEANING (intended semantics) of the derived sentence. USAGE of grammars: Generation of sentences. Analysis-parsing of sentences. NATURAL LANGUAGE PROCESSING -- NLP ================================== In natural languages, context-free grammars are considered insufficient, since: 1. They cannot express inter-component dependencies. 2. Multiple parameters/dimension modeling. NLP grammars are more complex. They include FEATURE STRUCTURES that enable the INEDEPENDENT AND INTERLEAVED SPECIFICATIONS of different dimentions. ************************************************** * Grammars are models for SYMBOLIC MANAGEMENT OF * * LANGUAGE STRUCTURES. * ************************************************** ************************************************************************* Here is an example of using context-free grammar for generation in NLP (taken from P. Norvig: AI Programming -- Case Studies in Common LISP. 1992. morgan-Kaufmann) NORVIG: Chapter 2 -- File simple.lsp =================================== ;;;; -*- Mode: Lisp; Syntax: Common-Lisp -*- ;;; Code from Paradigms of Artificial Intelligence Programming ;;; Copyright (c) 1991 Peter Norvig TASK: Generate random English sentences, that are described by a simple grammar. Given the following TINY grammar: Sentence ==> Noun-phrase + verb-phrase Noun-phrase ==> Article + Noun Verb-phrase ==> Verb + noun-phrase Article ==> the, a, ... Noun ==> man, ball, woman, table, ... Verb ==> hit, took, saw, liked, ... I. == A straightforward solution: Represent each grammar rule by a separate LISP function: (defun sentence () (append (noun-phrase) (verb-phrase))) (defun noun-phrase () (append (Article) (Noun))) (defun verb-phrase () (append (Verb) (noun-phrase))) (defun Article () (one-of '(the a))) (defun Noun () (one-of '(man ball woman table))) (defun Verb () (one-of '(hit took saw liked))) These are no-argument functions. But they can return different values since they use the random functions (these are not functions in the mathematical sense). ;;; ============================== (defun one-of (set) "Pick one element of set, and make a list of it." (list (random-elt set))) ; All functions in the grammar return lists. (defun random-elt (choices) "Choose an element from a list at random." (elt choices (random (length choices)))) => (length '(hit took saw liked)) 4 => (random 4) 3 => (random 4) 1 The function 'random' returns an integer between 0 to n-1, when applied to n. => (elt '(hit took saw liked) 0) hit => (elt '(hit took saw liked) 1) took => #'elt # => #'random # => #'length # => (sentence) (a man hit a man) => (sentence) (the ball hit the man) => (sentence) (the ball saw a man) => (verb-phrase) (liked the ball) => (trace sentence noun-phrase verb-phrase article noun verb) (sentence noun-phrase verb-phrase article noun verb) => (sentence) sentence [call 1 depth 1] with no args noun-phrase [call 1 depth 1] with no args article [call 1 depth 1] with no args article [call 1 depth 1] returns value: (the) noun [call 1 depth 1] with no args noun [call 1 depth 1] returns value: (table) noun-phrase [call 1 depth 1] returns value: (the table) verb-phrase [call 1 depth 1] with no args verb [call 1 depth 1] with no args verb [call 1 depth 1] returns value: (took) noun-phrase [call 2 depth 1] with no args article [call 2 depth 1] with no args article [call 2 depth 1] returns value: (a) noun [call 2 depth 1] with no args noun [call 2 depth 1] returns value: (ball) noun-phrase [call 2 depth 1] returns value: (a ball) verb-phrase [call 1 depth 1] returns value: (took a ball) sentence [call 1 depth 1] returns value: (the table took a ball) (the table took a ball) => (untrace) (verb noun article verb-phrase noun-phrase sentence) ;;; ============================== 2. Further complicate the grammar: Noun-phrase ==> Article + Adj* + Noun + PP* Adj* ==> 0, Adj + Adj* ;; Adj* denotes 0 or more adjectives. PP* ==> 0, PP + PP* ;; PP* denotes 0 or more prepositional phrases. PP ==> Prep + Noun-phrase Adj ==> big, littel, red, blue, ... Prep ==> to, in, by, with, ... The rules for Adj* and PP* contain choices: (defun Adj* () (if (= (random 2) 0) nil (append (Adj) (Adj*)))) (defun PP* () ; A different implementation (if (random-elt '(t nil)) (append (PP) (PP*)) nil)) (defun noun-phrase () (append (Article) (Adj*) (Noun) (PP*))) (defun PP () (append (Prep) (noun-phrase))) (defun Adj () (one-of '(big little blue green adiabatic))) (defun Prep () (one-of '(to in by with on))) => (adj*) nil => (adj*) nil => (adj*) (blue) => (adj*) (little little adiabatic blue) => (random-elt '(a b c d)) a => (random-elt '(a b c d)) c => (PP*) (on a blue little big woman in the man) => (PP) (on the adiabatic ball) => (sentence) (a big man hit the ball in the woman) Bad versions for Adj* and PP*: ;; Simply incorrect: (defun Adj* () (one-of '(nil (append (Adj) (Adj*))) )) adj* => (adj*) (nil) => (adj*) (nil) => (adj*) ((append (adj) (adj*))) ;; Infinite recursion: (defun Adj* () (one-of (list nil (append (Adj) (Adj*))) )) PROBLEMS: Writing new grammar rules becomes complicated. ;;; ============================== ;;; ============================== II. == A rule-based solution: Easier to write grammar rules. Worry later about how to use the rules: The grammar: Sentence ==> Noun-phrase + verb-phrase Noun-phrase ==> Article + Noun Verb-phrase ==> Verb + noun-phrase Article ==> the, a, ... Noun ==> man, ball, woman, table, ... Verb ==> hit, took, saw, liked, ... Each rule is captured as a list. A complex right-hand-side is captured as a list. The grammar is captured as a list of lists. (defparameter *simple-grammar* '((sentence -> (noun-phrase verb-phrase)) (noun-phrase -> (Article Noun)) (verb-phrase -> (Verb noun-phrase)) (Article -> the a) (Noun -> man ball woman table) (Verb -> hit took saw liked)) "A grammar for a trivial subset of English.") (defvar *grammar* *simple-grammar* "The grammar used by generate. Initially, this is *simple-grammar*, but we can switch to other grammers.") ;;; ============================== The sentences are generated by "parsing" the grammar, and applying its categories. We need functions to parse rules: (defun rule-lhs (rule) "The left hand side of a rule." (first rule)) (defun rule-rhs (rule) "The right hand side of a rule." (rest (rest rule))) (defun rewrites (category) "Return a list of the possible rewrites for this category." (rule-rhs (assoc category *grammar*))) 'assoc' is a builty-in function. It takes a "key" and a list of lists. Returns the first list that starts with the key: => (assoc 'noun *grammar*) (noun -> man ball woman table) => (assoc 'Noun-phrase *grammar*) (noun-phrase -> (article noun)) => (rewrites 'noun-phrase) ((article noun)) => (rewrites 'noun) (man ball woman table) => (rewrites 'sentence) ((noun-phrase verb-phrase)) ;;; ============================== ;;; The generate function: 3 cases: ;;; 1. The argument is a list of categories, as in (noun-phrase verb-phrase). ;;; All categories must be generated. ;;; 2. The argument is a category that has several possible rewrites. ;;; One category is chosen (in random). ;;; 3. The argument has no possible rewrites -- it is a word. Its list is ;;; returned. (defun generate (phrase) "Generate a random sentence or phrase" (cond ((listp phrase) (mappend #'generate phrase)) ((rewrites phrase) (generate (random-elt (rewrites phrase)))) (t (list phrase)))) => (generate 'sentence) (the woman saw a man) => (generate (random-elt (rewrites 'sentence))) (the table hit the ball) => (generate '(article noun)) (a man) => (generate 'noun-phrase) (the man) => (generate 'verb-phrase) (took a table) A different version of 'generate', using if: (defun generate (phrase) "Generate a random sentence or phrase" (if (listp phrase) (mappend #'generate phrase) (let (( choices (rewrites phrase))) (if (null choices) (list phrase) (generate (random-elt choices))) ))) => (generate 'verb-phrase) (hit a man) The 'let' construct: introduce variables that are not parameters of functions. Never use a variable that is not introduced first!! ;;; ============================== ;;; ============================== III. ==== Changing the grammar without changing the program: (defparameter *bigger-grammar* '((sentence -> (noun-phrase verb-phrase)) (noun-phrase -> (Article Adj* Noun PP*) (Name) (Pronoun)) (verb-phrase -> (Verb noun-phrase PP*)) (PP* -> () (PP PP*)) (Adj* -> () (Adj Adj*)) (PP -> (Prep noun-phrase)) (Prep -> to in by with on) (Adj -> big little blue green adiabatic) (Article -> the a) (Name -> Pat Kim Lee Terry Robin) (Noun -> man ball woman table) (Verb -> hit took saw liked) (Pronoun -> he she it these those that))) (setf *grammar* *bigger-grammar*) => (generate 'sentence) (he hit terry) => (generate 'sentence) (these hit she) => (generate 'sentence) (the ball by the table in these saw the little table by a ball by a man by terry ) => (generate 'sentence) (the woman on the big green woman saw a woman in a blue table) EVALUATION OF THE TWO SOLUTIONS: 1. Straightforward SOLUTION for the exact task -- DIRECT SOLUTION. 2. Straightforward NOTAION (DESCRIPTION) of the problem. The solution comes as an EXTRA step -- DIRECT REPRESENTATION. Approach (2) is more flexible: Easier to MODIFY and to EXPAND. Typical to most AI problems. ;;; ============================== ;;; ============================== IV. ==== SAME DATA, DIFFERENT PROBLEM: TASK: Generate the SYNTAX TREE for a sentence: Instead of: (a woman took a ball) Generate: (sentence (noun-phrase (article A) (noun WOMAN)) (verb-phrase (verb TOOK) (noun-phrase (article A) (noun BALL)))) A solution with approach (1) is hard -- whole rewriting. A solution with approach (2): a. cons the category in front of each rewrite. b. List the results -- mapcar, instead of mappend. (defun generate-tree (phrase) "Generate a random sentence or phrase, with a complete parse tree." (cond ((listp phrase) (mapcar #'generate-tree phrase)) ; here is the mapcar. ((rewrites phrase) (cons phrase ; here is the extra cons. (generate-tree (random-elt (rewrites phrase))))) (t (list phrase)))) => (generate-tree 'Sentence) (sentence (noun-phrase (article a) (adj*) (noun ball) (pp*)) (verb-phrase (verb took) (noun-phrase (name kim)) (pp* (pp (prep in) (noun-phrase (name lee))) (pp* (pp (prep with) (noun-phrase (name robin))) (pp*))))) => (generate-tree 'Sentence) (sentence (noun-phrase (article a) (adj*) (noun table) (pp*)) (verb-phrase (verb saw) (noun-phrase (name pat)) (pp* (pp (prep with) (noun-phrase (name terry))) (pp* (pp (prep with) (noun-phrase (article the) (adj*) (noun woman) (pp* (pp (prep on) (noun-phrase (article the) (adj*) (noun ball) (pp* (pp (prep in) (noun-phrase (pronoun those))) (pp*)))) (pp* (pp (prep to) (noun-phrase (pronoun it))) (pp*))))) (pp*))))) ;;; ============================== TASK: Generate ALL possible rewrites of a phrase. (defun generate-all (phrase) "Generate a list of all possible expansions of this phrase." (cond ((null phrase) (list nil)) ((listp phrase) (combine-all (generate-all (first phrase)) (generate-all (rest phrase)))) ((rewrites phrase) (mappend #'generate-all (rewrites phrase))) (t (list (list phrase))))) (defun combine-all (xlist ylist) "Return a list of lists formed by appending a y to an x. E.g., (combine-all '((a) (b)) '((1) (2))) -> ((A 1) (B 1) (A 2) (B 2))." (mappend #'(lambda (y) (mapcar #'(lambda (x) (append x y)) xlist)) ylist)) Work with *simple-grammar*: => (generate-all 'Article) ((the) (a)) => (generate-all 'Noun) ((man) (ball) (woman) (table)) => (generate-all 'noun-phrase) ((the man) (a man) (the ball) (a ball) (the woman) (a woman) (the table) (a table)) => (length (generate-all 'sentence)) 256 ************************************************************************* BACK TO MUSIC: RHYTHM IN TONAL MUSIC: ~~~~~~~~~~~~~~~~~~~~~~ Language "words": Mdur = {1/8, 1/4, 3/4, ...} Let TM-dur(M) denote the language of legal durations in rhythms with meter M = . We write a GRAMMER G(M) that defines TM-dur(M). G(M) includes: a VOCABULARY, a START CATEGORY, and a set of derivation rules. * The vocabulary is the set of music durations Mdur. * The start category is (Measure . (n . m)). G(M) = (Mdur, n/m, P) where P is the set of derivation rules: 1. --> k1/n * n/m k2/n * n/m ... kl/n * n/m = = k1/m k2/m ... kl/m where k1+k2+...+kl=n 2. For a music duration p= s/t> and a natural number q, p --> k1/q * p k2/q * p ... kl/q * p where k1+k2+...+kl=q The language of G(M) is the set of all music duration combinations that can be derived from n/m. rule: lhs -> (implicit . (( create the rhs ))) How to think about these rules? Of course -- we can implement a straightforward constructor... FIRST SUGGESTION -- implicit right hand side: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1. (Measure . (n . m)) -> (implicit . ((lambda(x y)((split "x = k1+...kl") (list k1/y k2/y ... kl/y))) n m)) 2. (Measure . (s . t)) -> (implicit . ((lambda(y)((split "x = k1+...kl") (list k1/x * y k2/x * y ... ))) s/t)) PROBLEM: It is not clear how and at what direction we wish to use these rules. Still another approach SECOND APPROACH -- implement the constrained rules as CONSTRAINT NETWORKS: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Advantages: No direction for the implementation -- can be used top down, or bottom up, or both. For example, the rule For a music duration p= s/t> and a natural number q, p --> k1/q * p k2/q * p ... kl/q * p where k1+k2+...+kl=q can be implemented as the network: -*------- -/--------- | m1-|-------|-res d1-|----q p1----pr | | | | m2-|---p | d2-|-k1----| --------- ----------- | | | -*------- -/--------- | | m1-|-------|-res d1-|----q | p2----pr | | | | -+n----- | m2-|---p | d2-|-k2----| | | --------- ----------- |--list-- s-|--q | | | . | -------- . | . | | -*------- -/--------- | | m1-|-------|-res d1-|----q | pl----pr | | | | | m2-|---p | d2-|-k2----| --------- ----------- | For the rule: --> k1/n * n/m k2/n * n/m ... kl/n * n/m = = k1/m k2/m ... kl/m where k1+k2+...+kl=n the network is the same, with: q = n p = n/m: -/------- | m1-|---n p----pr | | m2-|---m --------- If we abstract the above network as --------- | -|---higher level category p p1---- | | -|---split q p2---- | . | | . | | pl---- | |-------- then every rule is abstracted as: (rewrite Category) --> (network category) --> a list of output categories. FORMULATING THE MELODY CONCEPT ============================== A similar network approach can do. ... to be continued.