Watch video of the session here
A sequence of two demonstrations: The VAT: Video Analysis at Scale Virginia Kuhn
For the 2015 HASTAC conference I propose a thirty-minute Project Demo centered on the VAT (video analysis tableau), which makes large-scale video archives available to researchers. The VAT joins the emergent field of cultural analytics, which applies computational analysis to the study of art and culture. Filmic media—whether natively digital or recently digitized video—is particularly ripe for computational analysis, given its increasing ubiquity in contemporary culture.
Yet the challenges to making these archives useful are both technical and methodological in nature. Human tagging is too labor intensive, and computer vision is not yet sufficiently reliable: Neither approach is sufficient on its own.
As such, the VAT project uses a human-machine hybrid approach, deploying multiple image recognition algorithms and crowd-sourced tagging. The goal is to allow researchers to move with agility from textual description and collection management, to manual inspection, to automated analysis, to novel visualizations of discrete films.
Methodologically speaking, this is a new approach to doing research. Conventionally, cinema scholars conduct close readings of individual films based on knowledge of a larger corpus, which serves as context. This is a vital approach that will and should continue, and yet the corpus is rapidly expanding necessitating some form of automation of the processes of indexing and querying. Just as some digital humanists have used “distant reading” (cf: Moretti) to study literature, the VAT allows researchers to enhance close readings with distant readings of filmic archives.
With a user-friendly interface (Medici), and backend tools on a supercomputer, the VAT employs 17 descriptors for image search, along with searchable fields for comments and tags. Finally, the VAT team is experimenting with novel visualizations that employ scientific visualization processes to footage in ways that treat video as more than simply a stack of single frames. Cardamom of the Dead: virtual reality storytelling for Oculus Rift Caitlin Fisher
Written in Unity for use with Oculus RIFT glasses, Cardamom of the Dead is a literary VR environment - the user wanders through a virtual environment filled with a vast collection of things a narrator, heard in voice-over, has hoarded over years (decades? centuries?). The environment is filled with debris and stories and the piece is ultimately a meditation on collecting as madness, consoling practice and memory palace.
For HASTAC I would like to demo the work and talk about both Cardamom and the associated issues a piece like this raises for digital humanities practice. Built in the Unity game engine, the piece is the concretization of art+science collaboration and provides an opportunity to talk about working in game environments, translation between augmented reality and virtual reality environments (the first version of this piece was in AR), working in different scales and work supported by my STEAM-oriented research lab.