TotoPoetry versus ChatGPT: An Introduction

TotoGEO is the umbrella name for an AI Lab at INSEAD; toto in Latin signifies everything, universal or total, and GEO is an acronym for Global Education and Outreach. TotoPoetry is a collection of algorithms specific to poetry created by the TotoGEO lab. The following two videos explain the breadth of the lab’s activities:

Purpose - TotoPoetry

While sometimes entertaining, poems generated by TotoPoetry algorithm(s) are for educational or didactic purposes. The graph-generated poems created by this methodology are designed to mostly benefit non-English speakers (by far the largest users of this dictionary) about various English-language poetic forms (especially the forms used to teach English to non-English speakers). In many “English for non-English speakers” courses, poetry exercises are used to allow learners to explore the complexities of English, but also to help students retain vocabulary (some persons benefit reading didactic poetry as a form of learning). Poetic forms taught in such classes, or those used to teach poetry to English speakers, are highlighted (e.g., acrostics, cinquain, diamante or diamond, haiku, sonnets, etc., with more being added over time).

Another purpose is to have the poems, themselves, give insight into the dictionary headword or expression, with an express goal to highlight ambiguity for as many words or expressions as possible in all of the world’s documented languages (e.g., millions of poems in English alone). The graph-generated poems posted take on a lexicographic purpose, therefore, which may lead to further exploration and/or learning of the language (e.g., a poem about “pope” may refer to “ruffe” which is a type of fish, called a “pope”). As more experience in gained in generative techniques, TotoPoetry will expand to all the other languages covered in the dictionary (about 100 languages at first, and then the languages with smaller populations, except those where ownership rights are claimed by academic field linguists or local community leaders).

This project introduces the potential usefulness of graph-generated prose, crossword puzzles, language learning tools, and in this case poetry. Wikipedia.org provides a useful definition of graph theory:

In mathematics and computer science, graph theory is the study of graphs: mathematical structures used to model pairwise relations between objects from a certain collection. A "graph" in this context refers to a collection of vertices or 'nodes' and a collection of edges that connect pairs of vertices. A graph may be undirected, meaning that there is no distinction between the two vertices associated with each edge, or its edges may be directed from one vertex to another; see graph (mathematics) for more detailed definitions and for other variations in the types of graphs that are commonly considered. The graphs studied in graph theory should not be confused with "graphs of functions" and other kinds of graphs.

TotoPoetry was created with an enthusiasm for dictionaries, words and non-random generative methodologies. The following is the summary of first-generation poems posted from those created via TotoPoetry algorithms:

Genre	Posted (version 1.0)
Acrostic	608,309
Butterfly Cinquain	97,272
Cinquain	353,084
Diamante	183,644
Ekphrastic	204,001
Fibonacci	223,772
Gnomic	179,977
Haiku	561,526
Kural	261,783
Limerick	344,491
Mirror Cinquain	176,993
Nonet	156,540
Octosyllable	180,090
Pi	194,492
Quinzaine (nn2)	104,092
Rondelet	162,119
Sonnet	18,407
Tanaka	141,470
Unitoum	163,748
Verse	58,816
Waka	138,031
Xenia Epigram	363,583
Yoda	195,566
Zedd	195,162
Total Posted	5,266,968
Total Created	7,699,957

More recently, TotoPoetry was adapted to handle millions of additional phrasal inputs, expanding the dictionary size by orders of magnitude.

TotoPoetry versus ChatGPT

Interest in comparisons across generative techniques have dramatically increased since the popular use of ChatGPT (in its latest versions). Some methodologies leverage deep learning algorithms, others focus on rule-based or cognitive methodologies, and still others are hybrids of mixed methodologies. Fundamentally, TotoPoetry is rule-based but leverages a trained linguistic graph that is derived, in part, from trained neural networks on massive quantities of input text; TotoPoetry is, therefore, a hybrid methodology.

For this exercise, since Totopoetry does not use a question decomposition model on input promts, it uses a single seed word for comparison to a question posed to ChatGPT focusing on the two criteria: (1) didactic style, and (2) form or genre relevance. We begin with limericks.

Task #1: Write a didactic limerick about love

A limerick is a form of verse, usually humorous and frequently rude or vulgar (referring to a local person, especially female, who is discussed in an Irish pub setting, without using her name), in five-line, predominantly anapestic trimeter with a strict rhyme scheme of AABBA, in which the first, second and fifth line rhyme, while the third and fourth lines are shorter and share a different rhyme.

Task #2: Write a Pi poem about love

A "Pi Poem" is composed of words where the length of each word is equal to each digit of Pi in order. So, for example, the beginning of the poem would be made of a 3-letter word, then a 1-letter word, followed by a 4-letter word, 1-letter word, and 5-letter word, thus following the sequence of Pi which starts as 3.1415926 … etc.

Task #3: Write a short poem that talks like Yoda about anger

This genre is specific to TotoPoetry and would not be available for a deep learning language model to train on.

Task #4: Write a Sonnet about truth

English sonnets generally have 14 lines, written in iambic pentameter (10 syllables each with the pattern no stress/stress/no stress/stress, etc.), with a turn or surprise around line 9. They often have a rhyming pattern ABAB CDCD EFEF GG or similar. A large number of traditional and experimental variations exist for this form that is heavily used by William Shakespeare. Virtually all Anglo-Saxon students are exposed to this form of English poetry. The title or key subject may be suggested in the first lines. This last comparison is noteworthy. The “sonnets” in TotoPoetry follow the rules established by only one sonnet: Shakespeare’s Sonnet 76. This sonnet has been referred to as “the writers sonnet” where Shakespeare is meta-discursive and self-reflexive, writing about writing using a particular style. Such metapoetry often deals with the topic of poetry being a formulaic endeavor.

William Shakespeare (Sonnet 76):

Why is my verse so barren of new pride,
So far from variation or quick change?
Why with the time do I not glance aside
To new-found methods, and to compounds strange?

Why write I still all one, ever the same,
And keep invention in a noted weed,
That every word doth almost tell my name,
Showing their birth, and where they did proceed?

O know, sweet love, I always write of you,
And you and love are still my argument,
So all my best is dressing old words new,
Spending again what is already spent:

For as the sun is daily new and old,
So is my love still telling what is told.

While the poems generated by TotoPoetry are in no way “better” than ChatGPT, they follow pre-assigned rules with high similarities across poems within each genre, but show wide differences across genres. Having each further focus on “defining” a topic for didactic purposes, TotoPoetry’s output narrows the variances across and within poems, for all potential seeds. Translation: one gets bored reading too many generated poems because they all start sounding the same because poetry is rather formulaic and computer algorithms mimic the rules ad nauseum.

TotoPoetry Methodology

TotoPoetry poems are created based on programmed heuristics (following rules from accepted poetic genres) relying on edge values from a type of large linguistic graph. TotoPoetry uses these values to mimic the human brain when it is asked to write a poem on a particular topic using a particular poetic form (as assigned to grade school or college students), for a specific purpose (didactic). Simplifying the process, most of what happens can be boiled down to four basic activities (they are more involved, and programmed for, of course).

First, when prompted to write a poem that describes “love,” the human brain quickly searches over a semantic web that has been learned; it is a type of experiential linguistic graph. This graph then intersects with a second set of constraints imposed by a given poetic form, or set of elaborate rules, to create poetry within the genre specified (rules can be vague, and not hard, which allow for half-rhymes, metaphors, puns, poetic violations, etc.). These first two steps alone can create a virtually infinite number of poems (millions of poems for a given seed word). Third, the algorithm must therefore also choose a good poem to publish, among many, that makes the reader “happy” or satisfied with the outcome. The third activity, therefore, is a mathematical problem of constrained optimization, where an economic utility function is being maximized. Finally, there is a portfolio problem given that there are exogenous social constraints, especially the perceived preferences of the reader, especially for variety. For example, the person reading the poem who is not the author might want to recognize it as being within the genre, evaluate it, enjoy it, understand its meaning or find some other value in it (e.g. certain “rules” beyond poetry may need to be adhered to – perhaps using less obscure words in a poem making it more accessible to non-English speakers, whereas other readers may prefer less common words, poems having unusual grammar, revolutionary ideas, vulgar themes, etc.). In economics, this is similar to a matching function between “buyer” and “seller” where there needs to be an equilibrium of sorts between consumers and producers (e.g., creating a poem that one person enjoys may not be a poem that “sells” best in the marketplace). Humans, solving a portfolio problem, implicitly try to select the right poem to get good grades from teachers who may give better marks for “cleverness” or something they prefer. Solving this problem becomes important when there are many poems that each maximize the utility function, or when social constraints are more “important” than the traditional utility maximization problem.

So, for a given page with multiple genres on the same page, efforts are made to avoid redundant use of the same vocabulary across poems. For example, the following three didactic acrostics do not share the same vocabulary, but are all generated from the same seed word:

Acrostic Prefix

	Loving
	Old
	Valuable
	Expensive

Acrostic Diagonal

	Liking
F	ondness
De	votion
Aff	ection

Acrostic Suffix

Deal
Gusto
Luv
Affectionate

Low redundancy is thus achieved within a poem and across poem genres for the same seed word or prompt.

The TotoPoetry Model(s)

TotoPoetry is the generator of graph-based poems using the four concepts described above. It has been “trained” using large quantities of contextual and linguistic data (e.g. observing how many times millions of words or concepts have been described, perceived or experienced by hundreds of independent sources), and been instructed on the rules of grammar and various poetic conventions (e.g. concepts like meter - iambic, trochaic, pyrrhic, etc. or feet – dimeter, hexameter, etc., line counts, stanza counts, rhyming rules, stress sequences accepted, violations accepted, frequency of violations, etc., are coded constructs). TotoPoetry then maximizes an abstracted objective function (e.g., a hybrid multi-attribute utility model), so as to attain a highest level of “satisfaction” – constrained to exogenous factors. The basic functional form, for a given writing exercise, is:

U(X, Y) = (aX₁ + bX₂ + …) Y₁^alphaY₂^beta …

where Y₁, Y₂ etc. are threshold variables (i.e. “must haves”), and X₁ and X₂ are compensatory variables (e.g. one variable value can compensate for the other), and U(X, Y) is the utility derived from the exercise (a, b, alpha, and beta are assigned coefficients). TotoPoetry is then “motivated” to get what it thinks is the best result in terms of what real people (who share a similar linguistic graph and knowledge of poetry in their minds) would expect to see. In essence, TotoPoetry is forecasting a human’s preference. Any number of functional forms can yield similar results. Basically, TotoPoetry is not inspired to write a poem, it is solving a problem (poems are not randomly generated). In the field of artificial intelligence, an unsupervised graph is used in a model that predicts likely success in satisfying a reader, then machine learning control (MLC) is the final stage to select the portfolio to publish.

These are called “graph-generated” poems so as not to confuse them with traditional poetry (the point is not to deceive the reader), but also to acknowledge that the primary factors driving the quality and choice of the poems to publish are the edge values in a large linguistic graph, not the utility function itself. Not all edges are created equally. Each edge in the graph is associated with a large array containing word pairs but also various quantitative measures including “intensity” weights and other data (e.g. phonetic information, suffix/prefix information, positions of speech, frequency of use by context, age of the word in English and other statistics). Here is an example of such weights for the word “love”, for the first 15 closely related terms which care denoted as synonyms (the lower the intensity, the lower one might find them related to the word “love”.

Love: single word to single word intensities

Rank	Intensity	Word	Synonyms	Synonyms of synonym
1	414.0091	love	affection	love, fondness, attachment, liking, tenderness
2	287.1197	love	like	as, similar, love, alike, liking
3	235.0498	love	loves	affections, likes, love, lovings, wants
4	213.0396	love	loving	liking, love, wanting, affectionate, fancying
5	210.0095	love	want	need, wish, desire, lack, require
6	164.0093	love	fondness	affection, love, liking, attachment, tenderness
7	158.0498	love	loved	liked, wanted, love, fancied, deared
8	154.0093	love	darling	dear, beloved, sweetheart, love, pet
9	152.0194	love	liking	loving, like, wanting, equaling, wishing
10	143.009	love	attachment	affection, love, connection, devotion, fondness
11	141.0094	love	fancy	imagine, fantasy, think, imagination, conceive
12	134.0095	love	dear	beloved, darling, expensive, precious, costly
13	134.009	love	sweetheart	darling, beloved, sweetie, lover, dear
14	125.0096	love	beloved	dear, darling, lovely, sweetheart, sweet
15	117.0092	love	cherish	love, foster, fondle, nurture, nourish

The relationships continue across all known words, with the lowest values of word pairs equaling 0 for the vast majority of word pairs across the graph (e.g., love and glioblastoma).

Similarly, one can map a single word to expressions, with related intensities.

Love: single word to expression intensities

Rank	Intensity	Word	Synonyms	Synonyms of synonym
1	55.009	love	be fond of	love, like, appreciate
2	49.4096	love	to love	love, to like, like
3	39.4094	love	make love	have sexual intercourse, have sex, love
4	36.1088	love	to appreciate	appreciate, like, enjoy
5	34.4096	love	in love	bosom friend, amorous, love
6	34.1094	love	to like	like, love, to be fond of
7	28.0088	love	enjoy oneself	enjoy, have a good time, have fun
8	27.0092	love	want to	want, like, love
9	24.4091	love	fall in love	love, enamor, enamour
10	23.1494	love	loved one	dear, sweetheart, beloved
11	23.1093	love	to adore	adore, worship, love
12	21.0092	love	care for	look after, treat, take care of
13	17.0492	love	love affair	affair, romance, amour
14	17.0093	love	I love you	love, love you, like
15	14.4094	love	true love	sweetheart, love, lover

Finally, one can map expressions to expressions, and calculate intensities.

Love: expression to expression intensities

Rank	Intensity	Expression	Synonyms	Synonyms of synonym
1	68.6094	love affair	affair	matter, business
2	56.009	love affair	romance	fable, love affair
3	49.4096	to love	love	affection, like
4	42.109	love affair	amour	love, love affair
5	42.0089	in love	bosom friend	in love, mating
6	40.1082	make love	have sexual intercourse	have sex, screw around
7	40.0093	make love	have sex	have sexual intercourse, screw around
8	39.4094	make love	love	affection, like
9	39.019	love affair	liaison	connection, link
10	36.0093	in love	amorous	love, amatory
11	35.9096	to make love	make love	have sexual intercourse, have sex
12	35.9096	make love	to make love	make love, have sexual intercourse
13	34.4096	in love	love	affection, like
14	33.0093	in love	The Lovers	in love, bosom friend
15	31.0088	I Love Her	Blur (it’s a band)	I Love Her, blur

Note that proper nouns can enter the graph (like song titles or band names, like Blur). Such distances can then be ranked in order. From an artificial intelligence point of view, the graph can be characterized as an unsupervised machine learning exercise (e.g., via multidimensional scaling, or MDS, but with potential asymmetries). A rank from one word to the next may be low in one direction, but high in the opposite direction. Bear to carry, versus carry to bear, for example, have differing rank order distances to/from each other.

Thus, when word vertices are connected by an edge, their level of “adjacency” can vary. In addition to simple edit distance metrics (e.g., Levenshtein or others), various intensity metrics (similar to degrees of membership in fuzzy sets) and adjacencies are captured by “overlapping cliques.” Overlapping cliques, as defined in TotoPoetry, exist when two words are frequently found in pairs (or triads, etc.) to describe a particular word or expression node or each other. For example, consider a first set for words that can describe “love,” like affection, adoration, and zero.

“Love” can signify “zero” in tennis, so both are in clique 1 with the node z₁. Similarly, many words can describe infatuation = love, affection and adoration. Note that “zero” is not connected directly to z₂:

The “strength” between w₃ and z₂, therefore, is rather weak (a poem about infatuation using the word zero may look odd to a reader having the above cliques in their mind). The relationship between z₁ and w₃ is “stronger.” The strength between z₁ and w₂ is probably greater still than between z₁ and w₃ (e.g., people are not likely to think of “zero” when they think of love but they are likely to think of affection). The strength between w₁ and w₂ is also rather strong given the co-occurrences (they are in 2 separate cliques that overlap in terms of these being members). So, the frequency with which a word describes another can be important, but also the extent to which two words seem to be used across cliques can be telling. If the above cliques are reconstructed hundreds of thousands of times across hundreds of languages, then direct and indirect intensity measures become apparent to and from all words in English, but also across all words within and across all languages (e.g. from French to Norwegian). One of the benefits of using edge values is that they allow asymmetry across poems authored in this manner. For example, a poem about “mavericks” may include references to beatniks, but a poem titled beatnik may not include a reference to mavericks. Similarly, edge values allow the poem to drift into metaphors, half rhymes, small but noticeable violations of hard rules (poetic license), puns or other surprising devices used in traditional poetry. The line in the TotoPoetry sonnet which defines the word “truth” ends in this couplet:

Okay, the truth is harsh, I horse you not,
I render now the fact, I’m just a bot!

The rhyme at the end is predetermined given the genre. The surprise poetic line is the first one. Clearly, the phrase “the truth is harsh” might have been plagiarized from existing text found on the Internet. The part “I horse you not” is however, more interesting. It is clearly meaningful and sensical. A query on Google for this exact phrase is found on blogs which have cited this poem. Within the graph, it equate “truth” with “not lying”. Lying is proximate to kidding, joking, fooling you, and horse play. Poets have a soft rule when faced with two words that might not follow iambic rules “horse play” = “stress stress”. One can drop a word, and the human reader’s mind can fill in the blanks. Which word to drop? The phrase “I play you not” would have worked. TotoPoetry chose “I horse you know” which seems to aliterate well with “harsh” – another rule within the genre.

Given a large graph, with edges having differing “intensities” and other indicators, poetic constraints (e.g. rules of structure, grammar, or verse) are then used to narrow the writing style to a feasible set. Coupled with usage frequency information, grammar rules/frequencies, parsed fragments “learned” and other “poetic” inputs, the program then optimizes (maximizes an objective function) using aggregate intensities (e.g. averages across words, lines, etc.) or similar “quality” measures to select which, of the virtually infinite number of poems possible, to “publish”. Since TotoPoetry is allowed to include fragments of existing text to be valued within the graph, it can thus mimic the practice of some traditional poets to “find” poetry in non-poetic extant literature. This is done to increase didactic impact for some forms or sub-genres.

TotoPoetry also generates author pseudonyms in a similar manner. For example, a poem about “heel” is authored by “James Wilson”. James is “proximate” to heel in a graph (i.e., James signifies "the heel holder"). Similarly for Wilson. In diamante or diamond poems, the family name is often based on the antonym of the dictionary headword or seed. The proper noun graph is rather weak for many words, given the limited meanings of names. TotoPoetry poems should be seen as the experimental fruit of an intersection between human linguistic history, economic optimization, and information technology, and not of an individual author or artificial brain. In fact, given the sheer volume of poems, no-one has read even one-tenth of one percent of those generated. The dictionary user typing in a word, therefore, is likely the first reader of the poems presented.

TotoPoetry and Turing Tests

The benefits of TotoPoetry is the ability for poems to be generated on topics or subjects that traditional poets might ignore, for which an audience might exist. Likewise, using edge values, the process avoids the possibility of a poem reading as if a group of monkeys sat a keyboard … or avoids us having to wait for a group of monkeys to come up with something reasonable. It also helps avoid having the poems read like a Mad Libs exercise. This begs the question. Are graph-generated poems indistinguishable from traditional poems? In one sense they always will be different. Traditional poems often spring from human emotions or specific events known to the author. Graph-generated poems cannot have such inspiration, but can only reflect these, if at all, in the aggregate and to the extent that language reflects our shared emotions or experiences.

That being said, one might ask if graph-originated poems are “better” than traditional poems. For obvious reasons, this question can never be answered. From limited information, speed advantages aside, it seems that graph-generated poems are better than ones that the average person comes up with, if given the same task – TotoPoetry programs simply knows much more than the typical human and better follows the rules (and can find cool violations) than average authors ever could – basically, this may mean that the average person is not a good poet, or that traditional poetry is hard to write for non-poetic people (try writing a simple acrostic poem for the word “book” – it is not obvious).

However, from a Turing test point of view, preliminary informal work (read “not peer reviewed”) suggests that graph-generated poems are basically indistinguishable from traditional poems in blind reviews, and in some cases judged by many to be of better “quality” than traditional poems (e.g. where reviewers go through a revealed preference exercise, or rate poems using Likert scales). The results vary based on the age of the reviewers or authors, and any “priming” or sample matching designs that might take place prior to the assessment. For example, if the reviewer is asked to compare poems written by first graders (6-year old authors) versus TotoPoetry poems, then graph-generated poems are preferred, and the person does not suspect the generated poems are written by a computer program (comparing one diamante poem to another). As the age of the real human authors increases, so too do their writing skills, and they converge to the graph generated poem in terms of “quality”. The above assumes that the poems are written on the same topic (e.g., “hate”) using the same poetic form (e.g. “diamante”) and the same didactic purpose. Once a poem is revealed to be computer written, the reviewer tends to changes the ratings in various magnitudes and directions, depending on the person (i.e., the attribute “generated” has intrinsic qualities).

In some cases, reviewers show a strong preference for TotoPoetry poems compared to Shakespeare’s poems, before and after the revelation of authorship. One can attribute this to the fact that most people generally do not like Elizabethan sonnets in a general sense (not that the graph-generated poems are better – i.e. people may dislike both); likewise, someone who prefers Shakespeare over all other authors will never prefer TotoPoetry poems, whether they believe them to be written by computer or not. Persons who are indifferent will generally not suspect that a computer has written a poem, as the concept may not seem feasible (though this has changed recently due to ChatGPT and other generative approaches). If told that a computer can, in fact, write sonnets, their chance of picking one written by a computer increases (but not necessarily their preference for traditional poems over generated poems). The above relates to your “average” person on the street as an author or reviewer (not necessarily a poet laureate or English major), and the “graph poet” described above. In general, graph-generated poems, matching genre and topic, are no better than any others, but they are really not much worse either, unless you are a fan of a certain poet who uses a particular form or style you recognize – graph-generated poems cannot compete in this case. In a similar vein, some will stick to preferring graph-generated poems over traditional poems because they are authored using computer algorithms (e.g., they would see it as “cool”), not because the poems are any better in any other way. The real frontier test will be for Epic poems, not yet generated by TotoPoetry.

Summarizing, TotoPoetry poems are seen as indistinguishable from others in a blind review when reviewers are given the task of writing or evaluating poetry on the same topic, using the same poetic form designed for the same purpose (educational). Readers are thus not aware or can not independently detect that a poem might be authored by a computer program, if (1) they are not first told to look for a computer-authored poem, or (2) are told that a computer might be used, but can not detect which poem amongst many others was created in this way (beating random odds). In other words, if one asks someone to evaluate a graph-generated poem in terms of “what do you think?” – few if any will say that it is obviously written by a computer unless they are told of its origins in advance. When told that it has been generated by a computer, after the fact, then they tend to see how this might be the case (though this is mostly derived from the fact that most poetic forms looks formulaic). A similar reaction occurs when one is told a traditional poem is generated by computer, when it has not. Limited experience have been ad hoc; please feel free to use the poems on this site to conduct your own tests.

Future

As time goes on, this website will be augmented for educational poetic forms that are fitting with the mission of the TotoPoetry dictionary. Unfortunately, it is faster to write graph-generated poems than to program them and load them on the server, so it may take time posting them all. TotoPoetry is being improved and tweaked as time goes on, so newer, perhaps more interesting or complex poems will appear on the site in the coming years. Programs are currently being investigated for TotoPoetry to endogenously invent its own poetic forms which might prove interesting as well.

The above raises an interesting question. TotoPoetry has focused on didactic poems simply due to the mission of this dictionary project. A recent Wikipedia entry defines didacticism as following:

Didacticism is an artistic philosophy that emphasizes instructional and informative qualities in literature and other types of art. Didactic art intends not primarily to "entertain" or to pursue subjective goals. The opposite of "didactic" is "non-didactic." If the artist is more concerned with artistic qualities and techniques than with conveying a message, then the work is considered to be non-didactic, even if it serves instructive or educational purposes. … The term "didactic" also refers to media that are "burdened" with instructive, factual, or otherwise "educational" information, sometimes to the detriment of a reader's (or viewer's) enjoyment.

The article goes on to state the following (on the version available August 2, 2009):

Some have suggested that nearly all of the best poetry is didactic.

This view does beg the question. Will computer-authored poems, be one day considered the “best poetry” given that algorithms can cover vastly more subjects and languages than a single or even a large group of authors? Time will tell.