Experimental Interfaces 2014

These are my conference notes on the INKE conference Experimental Interfaces for Reading 2.0.

Note: these are being written live and will have all the usual problems. I tend to leave out everything I have to think about and furiously write down what I think I'm thinking about.

Thursday, September 18th, 2014

We gathered for an Expert Panel on Reading Interfaces. I was one of the "experts". We talked about conversation interfaces.

Friday, September 19th

Jennifer Roberts-Smith: Theatrical Interfaces

Roberts-Smith began by talking about some of the projects she has worked on. SET (Simulated Environment for Theatre) is a 3D theatrical modeling environment where you can move avatars around a stage and link to various materials.

Staging Shakespeare is a prototype mobile app that is a mini version of SET. The idea was that players can costume avatars, assign them voices, block a scene, and post the scene to a server where critics can comment on them and assign points. A player could then work at staging scenes until they become a master director. Alas the game was never implemented and the reasons are probably interesting.

They are now collaborating on The Stratford Festival Online where they are creating parallel texts. They are asking why the Stratford Festival would want to be online at all? One answer is that we have an obligation to meet our students and audience where they are. We also have an obligation to ask about the nature of the space where we meeting. In the case of social media we need to learn about that space.

We often locate agency in interactivity. We tend to think that giving something to the user to do is good enough.

She packaged interior and exterior sensory experience as the "real". Then she set this against the "virtual". Then she moved to the "imaginary" which is not real, but is generated in response to the virtual. Finally, she used this to define the theatrical which is functional (not fixed or inscribed) and instrumental in that uses space and sets and so on as instruments.

She stood before us with her notes on her head as a hat. Watching theatre we don't willingly suspend disbelief, we willingly accept a paradox where we have realities and theatricalities. Perhaps this is the difference between theatre and interface. Interfaces are meant to be transparent (usually) while theatre is meant to be noticed.

Then she talked about the play text as a "book." Reading of plays doesn't lead to a complete imaginary the way reading a novel does. The reading of a play is insufficient. What we need is an embodied functional virtuality - namely a performance of the play. Then we have a complete imaginary.

What makes an imaginary complete or not?

She talked about how we bring text (play books and programmes) to the theatre. There are thus multiple virtualities and these are not a modern phenomenon. (Tiffany Stern "Watching as Reading"; W.B. Worthen's "tedchnology" Drama: Between Poetry and Performance (2010).)

She then talked about humanities visualization and argued that visualizations are inscriptions as are instances. As you interact with a closed system on its terms you are creating new inscriptions.

Social media are performative, but not theatrical. You can post a selfie, but others don't have access to the embodied self. We operate in mostly closed systems where the system prescribes and constrains interactions. Are there constraints to theatre?

Traditional interfaces are governed by procedural rhetoric (code.) She asked "What arguments do traditional interfaces makes about human experience? Procedural rhetoric uses process to make meaning. When we design interfaces we are curating the processes that others can access to make meaning.

What then is a theatrical interface? To know, to make, and to be all get collapsed temporarily in theatre. It should be a medium for multimodal, emergent virtualities. The sites of transformation are the participants rather than objects. SET is not a theatrical interface in her terms.

She talked about The Last of Us and Brothers (2 games) as examples. These games dramatically change the interface at certain points.

She ended by asking how we might imagine interfaces. She talked about a staging of Richard the 3rd that was multi-modal.

Elika Ortega: The Remediation of Reading

Ortega looked at what readers did when they read or what they dealt with when reading. She talked about how books can have annoatations and marks by multiple readers. (Heather Jackson, Marginalia: Readers Writing in Books (2001)). She mentioned some neat projects like:

• The Art of Google Books
• Book Traces

She is developing a methodology to extract annotated pages out of digitized collections and then study them for recurring practices.

Dana Milstein: Manga de dokuha as Visual Novel: Ren’Py and Reading Marx through textual gameplay

Milstein started by talking about Manga de dokuha which are manga that translate important works (like Das Kapital.) She gave her presentation with a handout, rather than PowerPoint. She worked with her students to read these manga. Now they are looking at Ren'Py to create an interactive fiction of Das Kapital.

Cindy Malone: Noses in Books: Orientation, Immersion, Paratext

Malone started by talking about people worry that e-books and e-reading are a threat to the humanities. She thinks we haven't yet developed the appropriate interfaces. She took us back to think about how medieval scribes designed pages for different types of reading. Scholars and clergy where in the centre of scribe's minds when developing all the paratextual aides we now take for granted.

Then she talked about letterpress. A lot of things that are easy in calligraphy are difficult in letterpress. Digital books should let us break free from some of the limitations of print. What annoys us that e-books can help us with? One idea is orientation - the e-book should be able to do a better job at orienting us to what we are reading and who wrote it.

She then talked about "burying ones nose in a book." Nose/Book (Codex) > >. Maybe the relationship between the face and codex helps us orient ourselves. Perhaps the flat screen hinders things. She also believes that visibility of front matter and paratext would help.

Stéfan Sinclair: Looking at Open Access, 'Parcours numériques', and the Centre de Recherche sur les Humanités Numériques (CRIHN).

Sinclair started by talking about the case for open access. He used that to lead to introducing Parcours Numériques which publishes both in print and online. They have thought about how these two forms could complement each other. The print privileges the reading of a linear argument (thesis). The online encourages non-linear wandering in different directions. They are now experimenting with integration of Voyant into online books. Sinclair showed four different ways of integrating Voyant in:

• Standard launch of Voyant with text
• Embedded panels
• Special skins
• Tools that compare text to other collections

INKE panel on Reading at a Distance

This panel reported on work of the INKE ID group of which I am a part. Stan Ruecker introduced the panel talking about how he prefers Ramsay's idea of Algorithmic Criticism to Moretti's idea of Distant Reading.

John Montague talked about "Abstracting and Visualizing Big Data for Exploration and Discovery". He talked about what makes a good visualization. He quoted Bertin on the Semiology of Graphics. He showed some of our experimental interfaces like the Dendogram Tool at http://analytics.artsrn.ualberta.ca/viz/ . He then showed ideas for a Topic Modelling ZUI (Zoomable User Interface).

Tianyi Li talked about "Reading INKE’s Collaborative Publication Network." She showed a visualization of the publications of INKE and how the people are connected. She analyzed the network for modularity and looked at professors vs students. It is clear that Stan Ruecker and the INKE ID group were central to the network.

Luciano Frizzera talked about "Decoding Location-Based Information: Mobile Media Approach." He presented work that he did for his thesis at U of Alberta. He started by talking about how we have information all over physical spaces, but the information is usually governed. With hybrid (augmented) spaces that combine the physical and digital individuals can now augment public spaces. Maps have always been political. Mobile media opens up mapping to the rest of us. He talked about how we need to ask questions about the algorithms that mediate

I ended with a short paper on "Visions for a Distant Reading Machine" by Laurentia Romaniuk.

Paul Conway: Towards Enhanced Book Illustrations: World and Image in Photographically Illustrated Books

Conway works with large book collections. He is interested in "What do you do with 1 million illustrated books?" He is interested in the problem of book illustrations and their relationship to the surrounding text. One solution is the Flickr British Library approach to clip them and make a database with illustrations with no context. Likewise the Internet Archive snippet database pulls the illustrations out of the texts. He asks if we can't do better. We need prototypes that do things at scale.

Conway has been working on photographically illustrated books of the mid-19th century. There was a period when photographs and books fought each other. He showed one example, Jenson's Walking Tour In Brittany which was sold with a separate box set of stereographs. There are illustrations in the book and the stereo images in the separate box. The book has been digitized by the Internet Archive, but the stereographs have not. He has done a small test one Hathi Trust materials and things he can get much more meaningful texts to accompany the illustrations.

He is inspired by Ratto's ideas of critical making and Svensson's ideas of critical construction. In effect he is constructing/making new texts. He believes we are in a period of digital incunabula where we are still remediating the old.

Matthew Hiebert and William Bowen: Implementing a Social Knowledge Creation Environment: Iter Community

Ray Siemens read a paper for Hiebert and Bowen about the New Iter Community which brings together through social media (experiments) the community of medieval and Renaissance scholars. Iter itself started as a bibliographic database.

The original experiment in Drupal Commons failed in that no one used it. There is a lesson here about social media. They are now reimplementing and imagining something closer to a co-creator model.

Kevin Henry: The ‘intraface’ comes before the ‘interface’

Henry talked about how the light switch is an interface. We only think of the switch and the light, not all the infrastructure underneath. Switches then get put closer to the output (light) and eventually the switch is put right in interface (touch screen.) He is thinking about the inside of the device (the "intraface") and how it grows outwards. We once had discrete media - now they are all on one device. We don't look at information devices, but through them.

He asked about the grain and texture of this new mediaface. The media on a device is itself the interface. He calls for impoverished media. He quoted Scott McCloud about the gutter - the gap between panels which readers have to jump. The active reading of filling in the gutter engages readers in ways that may not be true of digital media. The Radiolab effect is when something appears dumbed down in order to create a gutter into which people can imagine. Henry mentioned RSA Animate as an example.

Teresa Dobson: The Interface Implications of Understanding Readers

She started by showing a video of a child trying a magazine and finding it doesn't work the way a tablet does. "A Magazine Is an iPad That Does Not Work" is an article that draws conclusions from the video on YouTube. The "digital natives" trope rules. The child is a native - we are immigrants. This language persists despite all sorts of problems. It isn't binary, it isn't supported and neurological evidence is mixed. We need to move beyond neuro-essentialism. Digital literacy doesn't stand in opposition to print but part of a continuum.

Dobson gave an overview of the reader usability probes they have run. She prefers to talk about the reader to user. Her methods are qualitative. She uses "sideshadowing" (talk-aloud). She works with small groups of participants. She seeks rich, generative, and proof-of-concept feedback. They look for feedback three levels (from Stan Ruecker):

• Functional: What are the bugs?
• Proof of concept: Is this instantiation of the concept reasonable?
• Generative: I s the concept worth pursuing?

They look at a lot of low fidelity prototypes and "End of Funding Cycle" testing.

She then showed and talked about three projects. The first was PlotVis which played with a different way of visualizing the plot that isn't linear. She showed all sorts of neat ideas that came from students as they asked them to draw ideas (before being exposed to an actual model.) Then she showed the Dynamic Table of Contexts. Finally she showed the Workflow tools Luciano developed.

Saturday, September 20th

Geoffrey Rockwell: Reading Tools from a Distance

William Wueppelmann: Metadata and Canadiana.org

Wueppelmann started by talking about Canadiana.org. The have done the easy stuff for which they had metadata. Now they are tackling the difficult stuff like the handwritten stuff including ledgers. There are some finding aides, but each one is different. They are now experimenting with a crowdsourcing transcription of a lot of materials. Users will be able to tag or transcribe a page. They will transcribe parts to seed the process. They also are looking for partners that are interested in sets of materials who would transcribe and tag whole collections if supported.

They are also looking at linked data as a way of storing metadata about the collections.

Alan Darnell: Metadata and the Scholar's Portal

Darnell talked about the Scholar's Porta. that makes all sorts of materials available to Ontario libraries and users. They do content aggregation and preservation at scale. They are now creating the Ontario Library Research Cloud. They are building a petabyte scale distributed storage network using cloud technology.

Darnell asked "How can libraries address new scholarly demands for reading at scale?" Ranganathan has two relevant rules "Every reader his (or her) book" and "Every book its reader." They are making content available fairly well, but not doing well at making the materials available for data mining. He then proposed a new rule:

With machine based forms of distant reading they have to rethink their services. He took us around the library and reader processes and the issues. One concern for them is the "boutique factor". Libraries support at scale which often means that they can't scale easily to boutique projects. How can they find the right balance? He asked for help.

Jon Saklofske: NewRadial: Challenging scales and standards of humanities scholarship through new knowledge environment prototypes

Jon talked about the NewRadial project. They are working with the idea of the book as a place like Bob Stein's SocialBook idea. The idea is that one can "occupy, search, sort, and annotate databases in a visual field." NewRadial is a bit like Collate in NINES, but it doesn't depend on a common metadata standard. It has its own shims for different databases so you can create your own collections and connect them as you wish.

It is an approach that tries to avoid ontology by committee or automation. I wonder what we lose when no one has to worry about the metadata of others.

Sayan Bhattacharyya: Reading via Fragments: Non-consumptive Reading as Practical Necessity and as Post-humanist Performativity

Bhattacharyya is at the HathiTrust Resaerch Center. He is mainly interested in situating their work in the history of reading. What can one do with a million books, especially if those books are under copyright?

One solution is fragments and fragments have a long history. Due to copyright issues a lot of content is locked up. A "non-consumptive" approach brings the algorithm to the data (rather than the other way around.) You can only run processes within a walled garden.

Another approach is to run the algorithm on features rather than the full text. This is a form of interface. Features are fragments for example per-page features packaged in file per volume with page section identification. They could have counts of part-of-speech word counts. You can't reconstitute the text from the features, but you can do things like topic modelling.

He then asked what this feature interface means? He thinks the breaking up of texts into fragments has a history and anxieties. We inherit a fragmentary past and try to read the gutters between them. We are anxious about giving up our human agency in these post-human times when machines do more and more. He concluded with two ideas:

• Interfaces can have agency
• Computationalism can be neo-humanist

We can imagine a way past the anxieties towards a neo-humanist peace with computing.