Main »

Advances In Visual Methods For Linguistics

These are my notes for the Advances in Visual Methods for Linguistics (AVML) conference in Tüebingen.

Note: these notes are being written live so they will have all sorts of problems. Do not trust me ... I am an unreliable conference reporter.

Thursday, September 25th, 2014

Geoffrey Rockwell: The Big Watch: Textual Visualization in the Humanities and its Publics

I gave a keynote surveying the history of text visualizations and showing Voyant.

Vit Baisa: Visualizing corpus data for lexicographers

Baisa talked about Sketch Engine a corpus query system. They have a number of general language corpora and they have a cool tool called WebBootCaT for building your own corpus.

It is called Sketch Engine as it can create word sketches that are profiles of words. It can then compare profiles. The profiles show the "word's grammatical and collocational behaviour." You can define what you are looking for in CQL (Collocate Query Language). It doesn't just fine salient collocates, but organizes them by the grammatical relation.

He showed an interesting visualization of the distribution of collocates between two keywords like Mother and Father. You see the words that are closer to Mother and those closer to Father. He showed a version of a word cloud that shows the clustering of collocates.

Maria Chinkina: Visual exploration of n-gram frequencies

Chinkina has developed an alternative to the Google NGram viewer. With her tool you can load your own corpus and explore different things. One can look for tokens - as many as you want. You can look for lemmas and POS if the corpus is properly tagged. She quoted Zchneiderman to the effect of "overview, zoom and details on demand."

Martin Hilpert: More flying bubbles

Hilbert talked about time series and how they might be used in linguistics. He began by showing Hans Rosling's famous animated time series. http://www.coolinfographics.com/blog/2011/3/15/hans-rosling-visualizing-mortality-history-video-infographic.html

He then walked us through various experiments and concluded with some ideas about how these work for inquiry.

The Rosling visualization has now been taken on by Google and can be used here http://www.google.com/publicdata/

Daniil Sorokin: Web-based Visualization of Toponyms and Their Derivational Patterns

Toponyms (names of places) are interesting in that even when language change and people move on the toponyms remain. At St. Petersburg State University they collected linguistic data on the Ingria area. They manually collected and analyzed the toponyms. Sorokin and colleagues created an application that would provide for geographical and group exploration.

They looked at GIS systems and then Google Maps. They asked questions like "how do you indicate toponyms?"

Their current version is at http://top-clusters-vis.appspot.com - It does an interesting job showing groups of toponyms that are similar in some way (having the same ending.) They work over Google Maps.

Johann-Mattis List: Visualizing Genetic Language Relations in Geographic Space

How to represent language history? August Schleicher is considered the scholar who invented the three diagram. He published in 1853 before Darwin. Historical linguists didn't like trees. Johannes Schmidt drew attention to the limitations of the tree and proposed the wave, which never really worked. Meillet (1908) try to represent a wave. Bloomfield tried again. None of these showed history. Hugo Schuchart imagined something that might be like movies.

What can visual analytics contribute? Interactivity and spacial representation. They have tried a sunburst visualization. They showed the WALS Sunburt Explorer http://th-mayer.de/wals/ . They also used Voronoi tesselation for area representation.

The whole tool is in Javascript using D3. They call it TREX - Tree Explorer. It combines panels for historical (tree) and and panels for geospatial data. https://github.com/tmayer/TREX/blob/master/index.html

Thomas Mayer: Interactively Exploring Patterns of Dialect Syntax

Mayer worked on the TREX environment above. He is exploring interesting ways of visualizing and providing interactivity. The Hessen Dialekterkenner is for testing user's input on dialects. He showed the tool and then talked visualization ideas like picking colour palettes. He showed some beautiful visualizations with dramatic colours.

He then talked about cartograms. Cartograms distort space to show data. You can distort countries to show their wealth or population.

Friday, September 26th

Sophie Kerttu Scott: Speech Perception and Speech Production - Some Views from Functional Imaging

Scott started by explaining functional imaging. Functional imaging is an indirect and slightly delayed measure of neural activity, but still very useful. When a part of the brain starts doing something there is a flow of oxygenated blood and that is what is imaged. When you measure blood activity you are measuring changes from baseline activity which means the baseline is important. For something like speech you have to ask what they are doing when you take the baseline, which is always something. When someone is talking there is also hearing and seeing someone and all sorts of stuff happening that isn't speech.

She talked about the neuroanatomy of the speech and what we know. Primate brains have a distinctly different way of processing spatio-temporal information. Primates seem to have rediscovered a way of seeing colour that other mammals lost. Other mammals focus on smell. Even though primates don't have human language it is likely that we build on primate ways of processing.

She showed us a hierarchy process where the active part of the brain shifts from auditory parts to linguistic areas. She asked if it is really a hierarchy. She thinks we are seeing multiple streams of processing that generate a higher form of speech. They are doing a lot of syntactic processing. She then showed a slide with a spot that seems to do processing on words independent of whether they are read or heard.

They are interested in the plasticity of the brain. Perception is active - we are always trying to hear things. Kids who get cochlear implants adapt to it in amazing ways.

She then showed how much of the brain kicks in once you have enough information to start understanding. And she talked about various ways of testing production of speech vs listening to speech. We have areas of the brain that respond to our own speaking. If we can't hear ourselves in time it becomes hard to speak. It seems that listen to ourselves differently.

She ended by showing how we can use functional imaging on mouth to see not just the brain.

Thomas Wielfaert: Parallel Coordinates as a complementary tool for exploring word similarity matrices

The group Wielfaert is part of studies lexical variation. Words don't map onto concepts cleanly. He studies lectal variation (sociolects, regiolects). Their goal is automatic modelling of lexical semantics.

He talked about the distributional hypothesis - "you shall know a word by the company it keeps" (Firth) A word's meaning can be induced from its collocates. Then he talked about semantic vector spaces in computational linguistics (Turney and Pantel 2012 for survey.) They create a multidimensional space of words against words with the cells being the wight of collocation. Then they can calculate a word similarity matrix. They use token-level semantic vector spaces. This gives them a second order co-occurrence matrix and then a similarity matrix of different tokens of the same word.

Why do they need visual analytics for their models? Callibration can benefit. They can identify misclassified. He showed some visualizations that he is trying. Scatterplot matrixes look promising. They are also working on statistical measures of cluster quality.

He had "Onomasiology" on some slides. The word in linguistics for the field concerned with naming - expressing through words. A new word for me.

Carsten Görg: Jigsaw: Integrating NLP with interactive visualizations for document exploration

Görg is a bioinformatics researcher and he talked about the Jigsaw project for visualizing large document collections. His goal is to help analysts with:

• Information foraging - looking for information for a specific task
• Sense-making - trying to bring the pieces of information

There are lots of metaphors for too much information. The needle in a haystack is a bad metaphor as it obvious how it can be found. He prefers solving 10 jigsaw puzzles - hence the name of the tool. He has two scenarios:

• Targeted analysis scenarios - for example in policing you have a particular situation and you are trying to predict what a criminal will do
• Open ended, strategic analysis - an analyst learns about a situation generally and is asked what will happen in a month

He is trying to bring together text mining and interactive visualization to provide visual text analytics. Automated approaches don't help when you don't know what you are looking for.

He then showed the different visualization windows. The tool has a number of different visual tools like one for looking at topics, one for seeing document similarity, and one for sentiment ordering.

He talked about an hourglass approach where people use automated methods like clustering to narrow the focus from too many documents. Then, when they find something interesting they start following connections to expand again.

He uses multiple screens (six!) to do analysis. He then demoed Jigsaw working on a large collection of articles from conferences. He had a nice list view where, if you clicked on an item in one list it would show the connections to items in the others. (Click on an author and see what keywords connect.)

He had a video of another use of Jigsaw.

Natalia Levshina: Semantic Maps: an Attempt of Catalogization and Comparison

Levshina started by talking about semantic maps. She showed us examples across languages. She showed a cool visualization of semantic connections in languages around the world. See http://clics.lingpy.org/

Sascha Fagel: Multimodal Spoken Language

Fagel works for Zoobe ("Say it with character") a company that makes avatars for speech generation. He works on getting the lip-synching working well for avatars. His background is in how we make speech. He talked about how audiovisual signal increases intelligibility of audio. Seeing lips moving increases hearing of speech. Young seem to spend less effort than older folk on listening. Adding tactile tasks adds effort. He showed interesting clips of faces that were visually one way (happy) and their voice another way (angry) - the McGurk effect.

They have different systems for tracking speech movements of face. They have sensors that go on the face, for example. Passive markers are big in the movies (think Avatar.) They also have 3D scanning solutions. They can scan for shape and texture and then do PCA on that to get simple measurements of face in motion. You can then synthesize a new face with the same parameters (change from male to female.) Then they can create synthetic lip sync. (Visemes are the equivalent to phonemes.)

His examples were astounding and some have passed the Turing test (ie. people can't tell the difference.) But natural lip sync isn't necessarily intelligible (to someone who lip reads.) Animation companies want natural not intelligible.

He showed some text to animated face systems along with cool tools like facerig where you talk into a camera and it shows your avatar talking (with your audio.)

Zoobe provides avatars to the healthcare environment.

Chris Montgomery: Visualization of listeners' real.time reactions to voice samples

Montgomery started by talking about what linguistic features are "salient". Researchers have looked at the social salience of things like "ing". The results can vary from place to place. Can one figure out what features communicate class or other social features?

He talked about an experiment they ran and showed a visualizer for looking at the results.

Yannick Versley, Alexandra Hagelstein: Mapping discourse connnectives across languages

Versley started by talking about what we map language data onto in visualizations. We can map onto real spaces (GIS) or generated spaces. Then he explained "connectives" that connect clausal units - words like "if", "and", and "because." He showed a nice visualization that mapped connectives in two languages.

Dominic Watt: Facial Composites

Watt talked about running experiments in which a vowel was changed and asking listeners where the boundary is from one vowel and another. Now he wants to add faces so he needs tools for generating variants of faces. He looked at the facial profiling systems used by the police. He showed a bunch of tools around today. It used to be that the tools focused on features. Now they are holistic. Watt wants to adapt these tools, but the parameters you can play with in the new tools aren't what they want for linguistic experiments. EvoFIT seems the best.

Then he talked about forensic speech science. Can they identify the phonetic cues that listeners will associate with threat? Can the face change association? He then talked about a study of sociophonetics his colleague is running. What variants of words do people associate with different classes or ages? Facial composite tools like EvoFit let them add a face to the pronunciations to see how that changes things.

They think there is clear potential for EvoFIT in research as it allows one to keep a tight rein on the faces you present.

And that was the end of the conference!