Thursday, September 18, 2014

Stylometrics and Edward II by Peter Farey

One piece of stylometric evidence which seems at first sight to throw quite a large spanner into the Marlovian works appears in Shakespeare, Computers, and the Mystery of Authorship, edited by Hugh Craig and Arthur F. Kinney.1 The book itself is not concerned with the Shakespeare authorship question as such, but with whether certain parts of Shakespeare's works or apocrypha are either by him or by a collaborator. For example, Craig argues quite convincingly for Marlowe having made significant contributions to parts one and two of Shakespeare's Henry VI trilogy. 

It is in a chapter called 'The authorship of The Raigne of Edward the Third' (by Timothy Irish West), however, that the item having most significance for the Marlovian theory appears. It is a chart (Fig.6.8, p.130) in which he is simply testing the validity of an approach being used to see whether Shakespeare wrote either the 'Countess' scenes (I.ii–II.ii) or the 'French campaign' scenes (III.i–IV.iii) in Edward III.

In this graph there are 90 shaded circles representing segments of 6000 words each from 27 plays which are taken to be solely by Shakespeare. There are also 236 diamond shapes representing 6000-word segments from 85 other single-author plays written between 1580 and 1619.

The horizontal (X) axis represents the extent to which each segment includes lexical words2 which have been identified as more characteristic of Shakespeare's works than the others'. The vertical (Y) axis shows the extent to which it uses words more typical of the others' works than Shakespeare's.3 Not surprisingly, they divide into two fairly distinct clusters – the circles in one and the diamonds in another.

In this particular chart, West has removed the three 6000-word segments of Marlowe's Edward II from the data, and treated them as if it were all of unknown authorship. Shown as black triangles on the chart, all three fall quite clearly within the 'other authors' cluster, and not within the 'Shakespeare' one. Here is the result. (The caption should of course read 'segments of Edward II', not 'segments of Edward III'.)

Against this, he shows (pp.127–8) segments from Shakespeare's King John, Henry IV (part 1) and Henry V, all of which fall in the 'Shakespeare' cluster, albeit at the edge nearest to the other one.

There is no doubt that this is a fairly strong piece of evidence that the author of Edward II (i.e. Marlowe) was not also the author of the Shakespeare canon. A few points need to be borne in mind, however, before we all admit defeat.

1. Although the data for Edward II are ignored in arriving at the characteristic words for the 'other authors', all of the rest of Marlowe's plays (except Dido, because of the possible input by Thomas Nashe) are included, whereas they would all need to be ignored too if the intention was to assess Marlowe as a Shakespeare authorship candidate – which it wasn't.

2. The three Henry VI plays, Titus Andronicus and The Taming of the Shrew – those which because of time proximity are most likely to have similarities to Marlowe – play no part in this calculation.

3. This also means that there is no play in the 'Shakespeare' set which is known to have been written less than five years or so after Edward II.

4. Furthermore, the Shakespeare set includes plays as late as The Tempest, written some twenty years later.

This question of date is of crucial importance in any stylometric attempt to assign authorship. Let me give an example which is in essence a very much simplified version of the method employed by Craig and Kinney (and West).

Suppose that there are two bodies of work, one which we will ascribe to playwright A and the other playwright B.

We work out that the frequency with which they each use the words 'most' and 'then' differs greatly. In fact, if we add up the total for both words in a play by either of them and find what percentage of them are 'most' we can be fairly sure that:

* if it's less than 40%, it's by playwright A

* if it's more than 40%, it's by playwright B

(In fact this works for all of A's 21 plays bar one, and all of B's 16 except two. You would need to get a bit more complicated to get 100% in each case!)

Now let’s imagine that we have a play where we suspect collaboration between the two playwrights. We find that Acts 1 & 2 are well below 40% (so probably playwright A) and Acts 3 & 5 well above (playwright B). Act 4 is more doubtful at 43%.

So does this tell us anything at all about whether the two playwrights are different people? No. In fact playwright A is Shakespeare before 1600, and playwright B is Shakespeare after 1600. And Twelfth Night (1601?) was the play in question, if you were wondering.

What we can see, therefore, is that to claim that this tells us they were different people is circular reasoning. If you start with an assumption that they are two different people, and take no account of time, then it’s hardly surprising that this is just what the figures will seem to show.

Don't get me wrong, though. This is a powerful piece of evidence against the Marlovian theory, and it would be wrong to think otherwise.

© Peter Farey, September 2014

1Craig, Hugh; Kinney, Arthur F. (2009). Shakespeare, Computers, and the Mystery of Authorship. Cambridge University Press.
2According to Craig & Kinney (p.224), "Words can be classified into functional words and lexical words (with just a few doubtful cases). Function words have a grammatical function; examples are the, and, she, before, and of. [...] Lexical words [are] nouns, verbs, adjectives, and adverbs which can be substituted for each other in a given sentence." They give king and mother as examples.
3The most characteristic words for Shakespeare (the horizontal axis) are found by calculating the proportion of 'Shakespeare' segments within which a given word appears and adding this to the proportion of 'others' segments within which it does not appear (giving a theoretical maximum of +2). The 500 words with the highest scores are the ones used. For the vertical axis, the same procedure is followed, but finding those words with the highest combination of proportions 'within the others' and 'not within Shakespeare'.


Dan Sayers said...

Thanks Peter, interesting stuff.

Not surprisingly, they divide into two fairly distinct clusters – the circles in one and the diamonds in another.

Actually this does seem fairly surprising to me. As you know, stylometric tests rarely achieve such clear results - especially when one of the categories is as broad as "other authors". I'd be interested to know how the "words Shakespeare uses more than others" and "words others use more than Shakespeare" were arrived at.

Interesting that they discounted Dido - it seems to me that apart from the title page attribution, there's more reason to suspect multiple authors in Faustus, or The Jew of Malta.

daver852 said...

There are many problems with this type of stylometric analysis. First, the sample size of Marlowe's works is small, and at least one of the plays (The Massacre At Paris) exists in such a poor quarto it is meaningless to include it. Secondly, there is the temporal gap between the time the plays attributed to Marlowe were written, and most of the works in the Shakespearean canon. Thirdly, a great many of "Shakespeare's" works are comedies, which of necessity employ a different vocabulary than plays written in blank verse. I wonder what the results would be if the exact same technique were applied to works that were written fairly closely together in time, and for which we have very reliable transcripts, namely "Hero and Leander," "Venus and Adonis," and "The Rape of Lucrece."

Anonymous said...

Interesting as always, Peter!

A couple of notes: Craig and Kinney don't just ignore time as a factor, they actively dismiss it: "...In addition, style changes over time (as Jonson had observed): it is well known, for instance, that early Henry James and late Henry James sound quite different. But what computational stylistics has shown is that, even so, tests of common words, rare words, and word pairings, especially when used in conjunction, can detect the similarities that continue to ride as a foundation underneath such easily perceived changes. While even the most perceptive, informed, and experienced readers may be challenged to find the basic (and telling) consistencies, the benefit of computational stylistics is that it can, in fact, do so." In their heavily sourced book, this is stated without so much as a footnote, as if it were common knowledge.

Second, the book contains a second chart, Fig. 4.4, which describes a function words test for segments of Arden of Faversham in comparison to accepted segments by Marlowe and Shakespeare. Kinney states that the "cluster" of Marlowe segments overlaps "a good deal" with the Shakespeare cluster. It would be interesting to see this chart with the segments identified by play and date, because that could reveal that these two overlapping clusters work just as well if viewed as a single cluster that reflects word use changing over time. It would, in fact, be very similar to the many charts that you have done yourself, Peter.


Peter Farey said...

Dan, I said that it was not surprising that the segments divide into two fairly distinct clusters because this is what I think the method forces. In particular, whilst 500 may seem to be a pretty large number to have as a cut-off point, the number of words from which they are selected is huge. For example, Hamlet alone apparently has well over four and a half thousand different lexical words overall.

As it happens, the cut off point in each case was +1.03, but had they used all words with a positive score I suspect that there would have been far more overlap between the two clusters.

Here is how the calculation was done in this case.

* Take the text of all 27 plays believed to have been solely by Shakespeare and divide each play into segments of 6000 words (any parts left over being included in the final segment each time).

* Identify all of the different words he uses, and remove all function words.

* For each word in turn, count how many segments there are which have at least one example of that word, and divide that sum by the total number of segments overall (i.e. 90). We'll call the result A.

* Take the text of of all 85 plays by other authors and divide each of them into 6000-word segments.

* For each of the 'Shakespeare' words in turn, count how many of the 'other author' segments do not have that word, and divide that sum by the total number of 'other author' segments overall (i.e. 236). We'll call that result B.

* Add each word's A and B figures together to get a measure of how 'Shakespearian' each word is.

* Arrange all of the words in numerical order, and take the 500 with the highest scores. We'll call these 'Shakespeare markers'.

* Applying the same method to all of the words in the other authors' works, this time finding words which appear more in these plays and less in Shakespeare's, find the 500 marker words for the 85 plays by other authors. We'll call these 'non-Shakespeare markers'.

* Take each segment in turn and count how many different Shakespeare marker words it contains. Divide this by how many different words that segment contains in all. The result is the value to be plotted on the horizontal (X) axis.

* Take each segment in turn and count how many different non-Shakespeare marker words it contains. Divide this by how many different words that segment contains in all. The result is the value to be plotted on the vertical (Y) axis.

Dave and Rory, thanks for your comments.

I think the main point to make is that the technique is appropriate only for the specific question it is designed to answer.

It is also worth noting that precisely the same method could be used, for example, on Shakespeare's works pre- and post-1600. Two equally distinct clusters would result, and (just like Edward II's did) the segments of any single play would fall within its 'correct' cluster.


Dan Sayers said...

Thanks Peter.

There is certainly a concern that such a test will lead to "overfitting" of the learning data (to use machine learning terminology).

The standard way to demonstrate that this isn't occurring would be to set aside portions of the Shakespeare data, as well as portions of the non-Shakespeare data, and then show that the same tests, when later applied to these portions, correctly categorise them into the right zones - despite them not being included in the original "learning" sets of data. Can you tell if such a check was carried out, as part of this study?

Peter Farey said...

Dan. They chose six different plays to test the validity of the method they planned to use for examining Edward III: Shakespeare's King John, 1 Henry IV and Henry V together with Greene's James IV, Peele's Edward I and Marlowe's Edward II.

In each case they started again from the beginning, recalculating how 'Shakespearian' each word was from a database which excluded the play in question. Although the effects were relatively minor, one can see how the precise positioning of the circles and diamonds changed slightly with each new graph.

Is this what you had in mind? Because if not I'm afraid I have no idea!


Tim Nash said...

If the Marlovian theory is correct, most of the Shakespearean canon would have been written in exile.. From the references in the plays, Marlowe would have been living in Italy where he would have spoken and read mostly Latin, French and Italian, depending on where he was at the time. Language is a living construct and it would be surprising if his usage of written English in the plays didn’t progressively drift away from that of the contemporary playwrights in London. From this the expectation would be that the differences become greater over time.

Dan Sayers said...

Peter - yes that sounds like the right methodology more or less, I think. It would be more usual to separate the testing data entirely, rather than redoing the comparison for each removed segment. I suppose they did it this way because of the limited sample size. I can't think offhand of any major issues with doing things that way.

It does feel like there may be some issues here with sample size, generally (i.e., when dealing with 6000 word segments of text, the number of available segments is very small), as well as, as you say, the potential for isolating an author by date as well as style.

Another question is, are all the Shakespeare texts used taken from the First Folio? Because then there's the factor of any editing work that might have been done to be considered, which would tend to unify those texts against others, and also add to the chronology factor since the early plays may well have been updated considerably at a much later date. I don't think it's too heretical these days to suggest that there may have been editing work done prior to publication of the First Folio.

Also relevant to a study of this kind is whether they used modern English version of the texts. If so, that would need to be done consistently across the whole corpus. If not, well that again adds to the chance of a false positive in identifying the Shakespearean texts.

Peter Farey said...

Tim, that's an interesting thought, and it wouldn't have taken all that long for such an effect to manifest itself. He wouldn't have needed more than a year at the most in any country before being fluent in the language. I'm not all that keen on the idea of his remaining overseas for too long, however, as I find it hard to imagine him not having had opportunities to see his plays performed, and using that knowledge in developing his technique.

Dan, in answer to your questions, I don't know why West decided to use 6000-word segments for his project. Craig and Kinney themselves use segments of 2000 words elsewhere.

No, the Shakespeare texts weren't all taken from the First Folio – only 13 of the 27 were – but I'm not sure what criteria were used in selecting which of the earlier versions were chosen.

They used the original spellings of all texts, but then collected them into 'families' representing the same word and used a single head-word for that family in counting. Different forms of the same word (e.g. singular or plural nouns, and past or present tense verbs) were treated as different words.

Hope that helps.


Anthony Kellett said...

I must start by restating my natural aversion to these kinds of studies. Whilst in an ideal world, they may have some validity, the dubious provenance of many texts – notwithstanding the assumption that no writer could ever imitate another – always makes me skittish about using them to colour my opinions, to any great degree.

Having said that, I am also loathe to discount them completely, and Peter’s claim, that it is evidence against the Marlovian case, is accurate; though I would beg to differ with his use of the phrase “powerful piece of evidence”; unless you meant ‘powerful’ in the sense that it could be abused quite successfully, Peter? Notwithstanding anything else, this fact alone makes it imperative we are made aware of such studies ad give them due consideration.

In view of all the above, I am not moved to examine more closely my following observations, to see if they are valid, but several issues strike me as problematic. If Marlowe were Shakespeare, then one would expect to see the dots for Edward II flow effortlessly to merge with the Shakespeare grouping to the right. Not only does the omission of the ‘transition plays’ (if I may call them that) negate the possibility of that process being observed, the very fact that Marlowe’s canon was separated at outset, from that of Shakespeare, would skew the right-side group even further from Edward II, it seems to me. For clarity, assuming I am visualising this correctly, if the ‘words most used by Shakespeare’ were determined using Marlowe’s canon combined with Shakespeare’s, it seems inevitable to me that any Marlowe play would move closer to the right-hand group, one way or another. Is this correct, Peter?

Whilst I would fall short of criticising the authors, since they were not trying to prove (or cast doubt) on wider authorship matters, assuming authorship in order to calibrate a model that will be used to determine authorship, is fraught with danger. Also, as Peter illustrates perfectly (and thereby knocks a huge hole in this ever being classed as ‘strong evidence’ by me), studies like this, which exclude chronological data, combined with the creation of a ‘standard candle’ from the very data one is testing, seems flawed from outset, to me.

It is well worth taking all these things on board, studying them and bearing them in mind. However, I’m not sure it has any effect on the logic that deduces Marlowe more likely survived Deptford than not. It does not explain how Shakespeare came to know things without any tangible means of doing so. It does not explain why those close to the Stratford man seemed oblivious to his achievements, either. Therefore, any study that reduces Marlowe’s claim to authorship by mathematics, which assumes Marlowe did not write the works of Shakespeare, as its fundamental constant, is not going to sway my opinion on these other issues. I should also add that the reverse is equally true, insofar as any mathematical study that proved Marlowe wrote Shakespeare, by assuming he did from outset, would be equally invalid, in my opinion.

Tim Nash said...

Peter, until you proved Marlowe wasn’t Monsieur Le Doux, I wouldn’t have put this forward as an idea. The longer the time spent in London to see his and other plays, the more the risk to Marlowe, so I believe visits are likely to have been short and he would have spent little time talking to people who weren’t in the know. So, unless there is good evidence for an extended stay, there would have been little to correct this effect.

Anthony Kellett said...

Moreover, Tim, we might expect to see him invent words derived from an amalgamation of these languages, which might support your idea.

However, as Peter says, I cannot see that Marlowe was so isolated from London for an extended period. There is also some reason to believe that whoever wrote the plays had some knowledge of current affairs in that circle, for which we must account.

Notwithstanding this, in order to make this claim you would have to show that the differences were actually down to the specific language divergence. You cannot simply observe that differences exist and claim it is due to him being in Italy (or some other place) without demonstrating that influence and its effect. Without this, surely we could just as easily claim the difference was because Shakespeare spent most of his time in Stratford, rather than London, could we not. In this respect, the Stratford man might also be expected to drift away from the crowd, in terms of his language usage.

In other words, can any differences noted be accounted for by a prolonged exposure to Latin, Italian or some other language; or merely by mixing with yokels who speak local English dialects with a piece of straw hanging from their mouths? Without this, any discussion about the reasons for this divergence could be a complete red herring. I would be more inclined to look favourably on your hypothesis if specific differences could be attributed to the more exotic languages. Even then, I would have to be further convinced that English scholars were far less exposed to Latin, specifically, than Italians. I’m not quite sure that would be the case, in any significant way.

As an aside, though Marlowe may have been behind a 'Chinese wall' to some extent (whether here or abroad), one can hardly deny the work itself was accessible and influential; unless, one subscribes to a view that Shakespeare’s writing had no influence on any of his contemporaries’ styles. So whilst Marlowe may have evolved away from ‘the crowd’, I’m not sure this excludes the crowd closing that gap by drifting towards him. Yet again, this makes chronology very important. What does this distribution look like for plays written at a similar date? The differences may be even more pronounced, if one does not allow time for styles to coalesce.

Ron Maimon said...

In cases like this, it is extremely important for Marlovians to reproduce the study for themselves, so as to make sure that the claimed stylometry does separate Shakespeare from non-Shakespeare. It is never ok to rely on the figures of others without replication, because there is a large amount of scientific dishonesty here, and Peter Farey tends to be more charitable towards others than he should, considering that he himself has shown the major failures of previous statistical separators. A plotting of the favorite 500 words over time, excluding Marlowe, and including Marlowe, would give a clue about whether the separator is valid. All previous separators have been invalid, and it is a mistake to presume this one is going to be any different.

These types of tests can't give two answers. If they are the same person (as I firmly believe), then all the tests must converge on this answer, and if they are different people, ALL the tests must converge on this answer. It is simply impossible for their to be inconclusive evidence in this area, as both Marlowe at ~100,000 words and Shakespeare at ~900,000 words each represent an effectively infinite corpus for the purpose of stylometry.

Rado Klose said...

I am not at all sure why this this exercise in stylometry should be taken as in any way definitive.It seems to me that a writers word choices are under conscious control and therefore subject to creative change. Surely patterns, such as the distribution of word lengths, which occur without conscious control are more significant.
Anyway. Some few years ago a schoolboy won a science prize for a bit of software he had developed, specifically a text compression algorithm. The idea was, as i remember it , to combine two texts under investigation and to squeeze them down. The more they compressed the greater the commonality between the two and the more likely it was that they were by the same author. Basically doing pretty much the same thing as this study. His results were strongly supportive of Marlowe's authorship. Does anyone else remember this?

Tim Nash said...

“Moreover, Tim, we might expect to see him invent words derived from an amalgamation of these languages, which might support your idea.”
Anthony, there is little point in inventing a word unless at least part of the audience appreciates it and it doesn’t interfere with the others’ enjoyment. This would naturally have pushed a talented playwright, even in exile, to inventing english sounding words. If he wanted only the well educated to appreciate it, then Latin, Greek and French too were possibilities.

“However, as Peter says, I cannot see that Marlowe was so isolated from London for an extended period. There is also some reason to believe that whoever wrote the plays had some knowledge of current affairs in that circle, for which we must account. “
Unless you believe it wasn’t made clear to him when his death was faked that if he reappeared in London he was still at risk of torture and death, visits to London would have been brief and he would have only met with those in the know. Over a meal or two the latest happenings would have been discussed.

“Notwithstanding this, in order to make this claim you would have to show that the differences were actually down to the specific language divergence. You cannot simply observe that differences exist and claim it is due to him being in Italy (or some other place) without demonstrating that influence and its effect. “
Stylometric studies like this are based on differences in the use of language. I have suggested a way this could have occurred for the Shakespearean canon.

“In other words, can any differences noted be accounted for by a prolonged exposure to Latin, Italian or some other language; “
As I wrote above, language is a living construct, which is a reason why dictionaries are updated. In exile, he would have had limited contact with his circle of friends and acquaintances and culture in London and unless he was attached to an Embassy, may not have spoken or read english evey day, apart from his own writings. Immersion in another culture and speaking and reading another language(s) is the other major factor which would have lead to a change in his use of english, compared to that of his contemporaries.

“or merely by mixing with yokels who speak local English dialects with a piece of straw hanging from their mouths? “
It’s amusing to think of Shakespeare, as a budding playwright, spending time like this but Anthony, as you discuss elsewhere, he was a successful businessman with interests in London as well as Stratford. Can you see him cutting himself off like this while he had a successful business in London?

As an aside, though Marlowe may have been behind a 'Chinese wall' to some extent (whether here or abroad), one can hardly deny the work itself was accessible and influential; unless, one subscribes to a view that Shakespeare’s writing had no influence on any of his contemporaries’ styles. So whilst Marlowe may have evolved away from ‘the crowd’, I’m not sure this excludes the crowd closing that gap by drifting towards him. Yet again, this makes chronology very important. What does this distribution look like for plays written at a similar date? The differences may be even more pronounced, if one does not allow time for styles to coalesce.
Being a playwright at this time was not a way to make a successful living.Most of them needed a second job; Marlowe as a spy; Shakespeare, as a businessman; Ben Jonson, as an actor and writer of court masques; Alleyn, as an actor and businessman. While they would have watched plays to study structure and plot and see what worked, unless they were writing together, when they likely would have wanted the play to flow, there was little incentive to study anothers use of language. Only with widespread further education could many make a living and reputation from studying Shakespeare’s every word.

Donna Murphy said...

Another article in the book Craig and Kinney edited demonstrates the difficulties with stylometric studies. Stylometrists Thomas Merriam and Robert Matthews had earlier trained a multi-layer perceptron neural network to discriminate between works by Marlowe and Shakespeare, and found that Marlowe was more like the author of the first quarto versions of “II Henry VI” and “III Henry VI” and the folio version of “III Henry VI” than Shakespeare, while Shakespeare was more likely the author of the folio verison of “II Henry VI” than Marlowe. In “The three parts of Henry VI,” Hugh Craig ran a series of tests based on lexical and function words for various playwrights on 2000-word segments of the folio versions of “II Henry VI” and “III Henry VI” and concluded that there were insufficient grounds to doubt that Shakespeare wrote “III Henry VI,” but that Marlowe likely wrote Jack Cade scenes in “II Henry VI.” Craig’s findings were the opposite of the Merriam and Matthews results.

Moreover, J. D. Wilson had previously argued that Thomas Nashe wrote the Jack Cade scenes in “II Henry VI” on the basis of numerous parallels and a similar thought process. One of my favorite examples: Wilson found that the author of these scenes confused the 1450 Jack Cade rebellion with the 1580 Jack Straw uprising, as had Nashe in his letter to William Cotton. Wilson showed how Nashe’s confusion could not have come from viewing “II Henry VI,” but rather from fusing two chronicle accounts he had read into one. I don’t trust stylometric analysis.

Laraine said...

@Rado Klose: Ian Cummings from Clearfield, Utah, at 15 was a finalist for the Discovery Channel Young Scientist Challenge, 2003. I saved a short article about it a few years ago, which I can't easily find online anymore:

"Ian's father shared an article with him that discussed how the language of a computer file could be determined without opening it. The article suggested applying a compression algorithm to the file. A compression algorithm shrinks a file by either reducing or eliminating all repetitive data. In a comparison, a small text file of unknown origin is attached to the original, known file and the resulting file is recompressed. In a comparison, the more alike the two original files are, the smaller the resulting compressed file is.

The article noted that this technique could also be used to determine authorship of a file. In this case, the best compression ratio would be obtained in a comparison of two works by the same author.....

Ian collected texts representing many authors from the Internet and ran them through a compression algorithm to see if he could identify the author without looking at the file. This worked for all the examples he tried.

Ian's second test was to see if he could answer the question regarding Shakespeare's work. His research indicated that there's doubt regarding the authorship of Shakespeare's work and that Christopher Marlowe, a contemporary, had the strongest claim. Ian analyzed samples of writings by Shakespeare, Marlowe, and other authors....."

Don't know if Mr. Cummings has done any more work on it, or has gotten a paper published and peer-reviewed, but googling "text compression authorship" gives some more recent results from others concerning general computational forensics. Can't read PDF at the moment, but will look at it when I'm on another computer.

We'd like to know, of course, what exactly is being compressed, is it words, phrases, characters, etc., or something else, and how. And who was the author of the original article/idea...

Sounds too simple to be correct, but who knows, maybe there's something to it.


Anthony Kellett said...


I believe this method would be as badly biased in favour of Marlowe as many others are against him.

What you describe would conclude that ‘of all the contemporary writers, Marlowe was most like Shakespeare’. I do not think even the most ardent Stratfordian would deny that assertion. It seems we are then still left with the question of whether they are one person or two.

Otherwise, one would have thought that Shakespeare would be the most closely matched to the writings of Shakespeare, and would have yielded a compression of virtually nil; or was that possibility not allowed for, in the process?

Of course, I might be missing something.

Laraine said...

Anthony Kellett,

I agree, we need some comparisons of quantitative measurements here. In other words, just how close was Marlowe to Shakespeare in the study, and what does that really mean in terms of this method...

Whether Cummings tested Shakespeare against Shakespeare or Marlowe against Marlowe, etc., I can't judge because we don't have his report from the science contest, or any further detail. He might have done all that work, however. Just to assume the traditional canon, however, would still be an OK first step, I think.

The idea of a compression, IIUC, is to eliminate "redundant"
material in order to get at the core of the author's style, i.e., to obtain a sort of "fingerprint."

But say the police have two (real) fingerprints, how do they know if they match... How closely identical do they have to be... Is there really an "absolute" way to enumerate small enough differences that would indicate identicalness? (And I've heard of one false conviction on a partial match anyway.)

And are measures of authorship as easy to visualize and use as real fingerprint measurements? It doesn't seem like it.


Rado Klose said...

Laraine thank you for adding some more details of the study. A couple of points . I notice the passive voice ( never a good thing) as in " which have been identified as more .." . Identified by who ? I think we should be told. In comparison to what other texts? It can't surely be that a person or persons unknown have taken a restricted and incomplete number of plays and made a selection based on what? personal feelings? Would you not have to compare to other texts to discover which words were more peculiarly Shakespeare's. If that was how it was done then isn't the whole thing skewed from the start. I must be missing something. Presumably you could group Marlowe and early Shakespeare count them as one writer and do similarly neat separation from other writers of the period. Is it not now time to add stylometrics to lies damned lies and statistics?

Rado Klose said...

A further thought on the algorithm study. I would have thought that a point in its favour was that it eliminates human choices ( as does Mildenhall's method) since these are bound to reflect an priori position.
Not knowing what in any detail what the results were the question one would want answering is this. Did Marlowe compress into Early Shakespeare/ late Shakespeare in a similar way to Shakespeare into himself. If the experience of running the program was that Marlowe's result was by orders of magnitude different to all the other writers of the period one can see why it's creator suggested that it strongly supported a common authorship.

Anthony Kellett said...

I’m not entirely sure you’re grasping what I’m saying, Laraine, but I think we are sort of in agreement, to some extent; trying to be as non-committal as possible.

To use your fingerprint analogy, I think this is a little more problematic than deciding if two fingerprints are the same. The main problem is that the first, ‘definitive’ fingerprint is being constructed by disparate sections of a fingerprint, which are assembled to create what the ‘detectives’ believe to be a single print of one finger. Only then do they use this to match against other prints. Whether there is a match is one point of debate, certainly, but the other debate is whether the original fingerprint is a single, definitive sample at all. Moreover, this does not seem to account for the fact that these particular ‘fingerprints’ change over time; so when was the original print taken, relative to the prints being compared to it? This is as much a test of deductive logic as it is a maths problem.

As I’ve said, it seems to me that any study, which starts with the assumption that the 36 plays of the first folio are one body of work, and tries to assess that for comparison against other individual plays, is riddled with logical fallacies, potentially. The likelihood is that this is not just an informal fallacy (such as circular reasoning) but equally likely to be a formal fallacy. Perhaps it would be described as a ‘base rate fallacy’, in formal logic – a form of ‘non sequitur’. Peter has pointed this out, in numerous ways over the years, when discussing stylometric studies (although he usually does it more succinctly than me!).

In short, since Marlowe and ‘Shakespeare’ are not (on the whole) writing simultaneously, then, at the very least, we should take the 36 FF plays, add to that six of Marlowe’s, and divide these 42 to create three ‘fingerprints’ of 14 plays. Unfortunately, even in that scenario, I would have to question the validity of including Marlowe, in a base sample designed to prove Marlowe was Shakespeare.

At this stage, the only study that I have found persuasive (or that gave food for thought) is Peter’s argument against Oxford’s authorship, which also appears to show a sort of transition and evolution in Marlowe’s writing, which could be seen as the formative years of the man who would become Shakespeare. It is not definitive, but it does give the kind of result one would expect, if Marlowe wrote much of the Shakespeare canon. However, and most importantly, it does not make any claim about who wrote what. It is simply an analysis of individual plays, irrespective of who wrote them, and, in this way, it avoids the trap of the potential fallacies I mentioned, above.

Laraine said...

There are a few things to be aware of, IMO. (Sorry if I don't address every question-- I am basically in agreement with much of what is being said.)

1) Shakespeare Changes Over Time

Peter F has given a good example in his item 4) above to illustrate that a particular Shakespeare idiosyncrasy can indeed change over time, and that it can even vary quite a lot within one play.

I'll look up PF's Oxford study.

2) Textual Issues

As others have mentioned, one needs to think about issues such as --Should the A or B text of Faustus be used when running such tests...

I still have a lot to learn about Shakespeare textual issues. And other Renaissance writers...

And is there a repository of texts for Shakespeare stylometric studies? One would hope that everyone would be looking at the same data.

3) Be Careful about Assumptions

I don't believe (unless I have missed some important studies) that we can assume that early Marlowe matches extremely closely to early Shakespeare.


Laraine said...

In 3) of my previous post, I meant to say "Marlowe and early Shakespeare" , not early Marlowe..."

So, because of the extra complication of authorial changes over time, it does make sense to me, as has been suggested, to start with the simpler case of a comparison of Marlowe to early Shakespeare rather than to all of Shakespeare's works.

And that suggestion should hold for other early writers such as Kyd, Peele, etc. Try first comparing Peele to early Shakespeare.


daver852 said...

I still feel that an analysis of the poems would be a more telling comparison for several reasons. First, we can have a high degree of confidence that the texts are not corrupted. Secondly, there is little chance of collaboration. Third, the genre is the same. Finally, the works were written within a few years of each other. I have searched to see if anyone has ever done a stylometric analysis of "Hero and Leander" versus "Venus and Adonis" and "The Rape of Lucrece," but haven't found anything.

The Other Great One said...

Very interesting read, as always!

Jim Fredericks said...

One thing that really hasn't been taken into account is the dramatic change in Marlowe's circumstances. While geography might play a role in a writer's development, imagine the stress and impact of exile and the loss of friends family, and friendly environs. I don't think one can chart out a development curve for a writer that could take such impactful change into account. It would certainly upset the "standard" development curve!

Ron Maimon said...

Come on Peter, as you yourself demonstrate right here, this is fraudulent statistics, like all the others which discount Marlowe's authorship. When careful vocabulary studies are done which include Henry VI, Titus, Richard III, it is obvious that Marlowe and Shakespeare overlap in vocabulary to at least associate his authorship to Henry VI, Richard III, etc. There is a reference to Hero and Leander in Two Gents, Marlowe is all over the comedies too. You can't say "This bogus statistical study constitutes evidence against Marlowe", because actively using Marlowe to devise the separator and only using late Shakespeare is stacking the deck so far that it constitutes willful fraud. You yourself have made a better stylometric separator using function words that drift with time, you could have published it as a "stylometric separator that conclusively proves that Marlowe and Shakespeare are different authors", but you chose to be honest instead. They didn't. This is crap evidence, your homemade stylometric graphs by themselves have more weight.

Alfa said...

Hi Peter,

Just popped in as I felt I had to comment on Ron's rather defamatory comment on Craig & Kinney. I wrote one of the first SAQ reviews on Craig & Kinney's work as I was interested in what was new in their work.

One of the most important differences, as you have also pointed out, is that their analysis uses big computers and big data to eliminate subjective 'test banks'. It uses all of the available data in every available test. All the test discriminants are derived algorithmically. This is the exact opposite of what Ron is claiming for their method. Far from being bogus, it is the only truly clean stylometry on the books and all stylometry in future will resemble their approach, even if it does not follow their method.

Their 'stylistic separators' are what is new. These are not new methods of cooking the books, the way Oxfordians love to cook the books. They are very extensive tests, using a single method, which can be focussed like camera lenses on what factors make the best visible discriminants.

With almost everything available online, thanks to Chadwick Healey and EEBO, their dataset always aims to be the largest possible relevant set. So if they are testing Webster against the whole of English theatre (to see if an anonymous work, or a work treated as anonymous, can be attributed to him, they work with everything written during Webster's career. They produced a most-like-Webster profile and a least-like-Webster profile using PCA to assign a value to every word in each dataset.

This is where the business of definition comes in. To be able to see the results most clearly they build their charts using most-like-Webster and most-like-everyone-else AND least-like-Webster and least-like-everyone-else. Extensive testing of passages whose attribution is not in dispute proves the method. I exchanged a number of emails with Prof. Craig before writing this article which he was kind enough to review, before it was published, for any misunderstandings.

So the idea that their work is "bogus" or "crap" or "fraudulent" is not something that serious students can countenance. And I know you don't.

This new stylometric work HAS to be incorporated in theory as the technique will improve in both resolution and accuracy. Rubbishing it is the only tactic for the likes of Oxfordians since it has brought Hand D into the fold and pushed their Earl so far from Shakespeare that no current instruments can find him again. He's gone.

And you wouldn't want to be seen in their tent, would you?

Peter Farey said...

Thanks for the comments, Alfie. In fact you reminded me that I had myself intended to post a very similar response to Ron's remarks, but never got around to doing so. As you are only too painfully aware, of course, not all Marlovians see things in quite the same way!

The important point is that Craig and Kinney weren't trying to show that Marlowe and Shakespeare were different writers, and the fact that it might be tempting to interpret their findings as doing so is hardly their fault!

For such a conclusion to be justified at least two elements of their methodology would need to be changed. The first is one which you pointed out in your article when you said "More disappointingly, the method will not allow the drawing of conclusions from alterations in stylistic development over a period". One would have to eliminate the possibility of any difference in style being due to changes over time. Second, one cannot conclude that two sets of work are by different people if that has been assumption upon which the database itself has been sorted. Such circularity would also need to be eliminated.

The problem is that the nearer one can get to approaching this ideal, the more complicated are the maths which are probably needed to achieve it. And the harder it will therefore be to use it to convince those who, like most of us, have only a very rudimentary (if any!) grasp of such matters.