The Rhetoric Of Text Analysis

There's a Toy in my Essay! Problems with the Rhetoric of Text Analysis

This is work in progress. It is neither complete nor stable.

Words are meant to measured not counted. (Fortune Cookie)

Stéfan Sinclair and I have been struggling with how to make text analysis accessible. (See Experiments.) One of the issues is how to report it. As Willard McCarty puts it in a HUMANIST post:

This is a problem all of us who try to teach computer-assisted text analysis face - there are very few examples of concrete humanities research essays that expose computer assisted techniques. There are papers that respond in traditional ways to original literary, historical or philosophical problems, but they do not describe how computing methods modulated the response. There are papers about tools and methods, but they don't give us concrete examples of techniques in question. No, I am tempted to conclude that computer-assisted tools are like Wittgenstein's ladder - they are discarded once one climbs up them to purchase a new view. I am tempted to go further and speculate that method is invisible to the humanities - that there is something fundamental to the rhetoric of the humanities that is antithetical to method, though not to reflection about method. You can have theory or method, but not both.

This is a problem for computer assisted text analysis as it means that we not only have to develop methods and apply them to problems in the humanities, but we also have to figure out how to talk about the insights that emerge without either boring our audience with technical details or hiding the methods. It is possible that we many never grasp the holy grail of writing works that engage our colleagues while documenting new methods rigorously. My intuition is that we have to develop a new genre of electronic writing where text analysis results are presented in a hybrid form that is not an academic essay nor just a tool. We may have to abandon the essay and monograph for an interactive hybrid that can sustain two threads, showing both conclusions and the processes used to reach them.

Examples of Tool Rhetoric

But first lets survey some of the types of writing that report on technique.

• Interactive Workbooks like the TACTweb Workbook provide narrative with embedded interactive panels. We can imagine version of these workbooks where the narrative is not pedagogical, but a research narrative with panels where readers can recapitulate the analysis.

• Research Maps like those discussed in Eye-ConTact which was an experiment in visual programming for text-analysis. Eye-ConTact adapted a programming paradigm used in scientific visualization tools. D2K and T2K provides a similar visualization of what they call "itineraries". Such visual maps can represent an overview of a research process and allow one to manipulate the process.

• Analytical Objects are, for lack of a better word, toys that are meant for the study of a particular text. An example would be the New York Times Visualization of the Republican Debates. The difference between a tool and an analytical object is that that object combines a particular text with particular functions in a seamless whole. The object invites exploration of the text by means other than reading pages.
• Recipes are tutorials aimed at particular humanities tasks. A recipe starts with a research task and discusses what "ingredients you need, what tools you can use, and how you can use the tools to complete the task. TAPoR has been experimenting with TAPoR Recipes to help people learn to use the portal and related tools. Here you can see a recipe that has links to tools.

None of these really reflect on the phenonmenon analyzed, with the possible exception of Analytical Objects.

The Problem with Tools and Rhetoric

The problem with text analysis tools is that of wearing glasses. If glasses are the tool you shouldn't notice the tool when looking at something else. Either you take the glasses off and look at them or you look through them, but it is hard to see both the glasses and that seen through at the same time. For this reason we also don't read about libraries in scholarship even if we suspect they were used by the author. Archives are mentioned, they are unique, but rarely mentioned are those common tools like a library, word processor, Google, or the pen and paper used.

There are, however, some problems specific to prose and technology that affect the weaving text analysis results in.

1. Results like long lists of words often take up a lot of space.
2. The results have to be interpreted. A list of words by itself doesn't mean much.
3. Most text tools are being used for interpretative research which is confirmed through rereading and expressed in conventional ways. The tools show anomalies that help one rethink a text in ways better explained with a quote.
4. Focus on the methods can be distracting from the argument. In fact, among humanists a computer based method might prejudice readers or turn them off.
5. In general we don't write about methods in the humanities.

Some Examples

It is useful to look at some examples of books that are grounded in computer assisted analysis.

The Computation of Style by Anthony Kenny is an important early discussion of stylometrics by an important philosopher. He applied his techniques in, <i>A Stylometric Study of the New Testament</i> (Oxford: Oxford University Press, ISBN 978-0198261780) and in his book on <i>Aristotelian Ethics</i>. I believe he used OCP and consulted with Susan Hockey.

The Englishing of Juvenal is an essay by John Burrows who is a pioneer of text analysis and a Busa award winner.

Gerald Ledger's book, Recounting Plato (Oxford 1989) is an important example of a sustained work that tries to ask a question important to Plato studies by using stylometrics. It uses stylometrics and various multi-variant analysis techniques to try to answer the question about the order and authenticity of the Platonic dialogues. The work gets criticized by others on methodological grounds, but none the less seems to be treated as important work on the issue. To me it is interesting how many people love to poke holes in this and then return to traditional, and unexamined chronologies. See a review http://serials.infomotions.com/bmcr/bmcr-v2n07-keyser-recounting.txt. See also <i>Plato: Critical Assessments</i> - especially the section on "Plato and Computer Dating" by Charles Young which reviews Ledger and others.

Bordieu's book Homo Academicus is very different example of the incorporation of evidence of computer analysis into an important academic work. Homo Academicu has 2D charts in the text that must have been produced by some form of MVA on a dataset. Not really text analysis, but an important book with the results of computing methods woven in. Similarly Paul McLean's book The Art of the Network has what look like MVA charts. Bordieu doesn't make much of the methods as if his audience would be comfortable with statistical methods and resulting graphs. I suspect that forms of text and data analysis are far more common in the French academic community where the line between the social sciences and humanities is much softer.

Finally we can look at Franco Moretti's Graphs, Maps, Trees, but that could be argued to be a book about computing rather than about a humanities subject.

Interactive Prose

One obvious solution, and the one we are experimenting with in experiments like Now, Analyze That is to take advantage of the interactive opportunities of online prose. An essay online, like this one, or the TACTweb Workbooks, can have panels or links woven into the text. The TACTweb Workbook took the panel approach, partly for pedagogical purposes. The Try It panels started with verbose instructions and slowly evolved to the panels you get in TACTweb.

In "Now, Analyze That" we have taken a threefold approach.

• We include certain truncated results as they would appear if you ran the tool. We do this were we think there is something interesting in the raw results that the reader might want to think about without our interpretation. The essay ends with a visual collocation of the words "black" and "white" with a web of words that collocate with each. It is meant to provoke further thought and we don't comment on it. We get around the problem of long lists by truncating the list to just the top.

We also chose to not edit the list - you can take word lists and put them into columns to fit the essay.

• The second approach we took was to have a Resources panel at the beginning with annotated links to the resources we used including all the texts. The link for the texts takes you to myLinks page generated by TAPoR where we did the experiment. For each text the user can open an Analyze panel where they see the text and can try their own queries.

• The most interesting rhetorical experiment is that we wove interactions inline in the text. This is an approach like Edward Tufte's sparklines, though he would probably complain about how awkward the button is.

Here you can see how the field (with the word searched for in it) and the submit button have been integrated into a sentence.

We had originally moved the interactive panels and buttons down to footnotes, but instead tried to integrate them right into the text. Everywhere we make a claim based on text analysis we give the reader the chance to check it themselves.

Claims in Conversation

This essay has so far been about how one can exhibit text analysis in interpretation, but now we have to ask whether one wants to exhibit methodology at all and why. Here are some of the reasons for showing text analysis.

Recapitulation - With analytical handles woven into an interpretative essay the reader can recapitulate the analysis.

Web of Method - Interpretations are not alone. They connect with each other into webs of interpretations linked by references. These webs are the slow conversations of the academy where essays respond to interpretations. These conversations hide as much as they show. They come to us fully formed and adult and therefore hide their tentative and experimental births through practice. Essays with text tools woven in can recover some of the tentative by letting others try.

Ludic Interpretation - When the reader can experiment they can play with the text and through your text and tools. This changes the reading - the reader is more of a player interpreter than just a reader. I'm not saying they are not a reader, but the handles to tools turn the text into an interactive panel and the reading into fiddling. The knobs and dials let the reader play through the text, a form of "dia - logos" ("dia" = through, "logos" = argument). This is not to say that playing through text is new, but that it is something we rediscover when there are toys in our essays. Just as we discovered that we have had hypertext all along only when we had automated hypertext, we can rediscover the playful in interpretation now that texts are embedded with interactivity and texts are turned into games. And this is something only possible now that computer games have brought attention to the ludic and the importance of games to youth imaginary.

Engagement - The ludic potential of the interactive online essay is important now. I am tempted to try the pedagogical argument, that we can engage more youth in the traditions of the humanities if we spark up our essays, but that misses the immediacy of the ludic. It is important now because it is now that the game is reinventing the novel and the simulation is reinventing the essay. Once we can put the essay online and we can see interactive objects online in or out of essays, it changes our perception of the prose essay forever. It is by definition something with the potential for engaging other texts and media, even when it doesn't. It is by definition engeared in a new configuration of technology, even without buttons. The essay is now pledged to play, even when serious.

Now, analyze that!

References

http://ccat.sas.upenn.edu/bmcr/1992/03.01.12.html is a fragment of BMCR 03.01.12 which has a long review of stylometrics in Plato with a brief history of stylometrics by Paul Keyser. This is worth following as it traces the history of counting words.