Data Beyond Vision

How do we represent tangible objects in a visual medium? We use words, pictures, and diagrams. We describe, share, show, and fail.

Humanists continue to expand the range of objects they study, but the range of scholarly outputs has not seen a similar expansion. While there are movements within Digital Humanities to consider nontraditional formats, the presentation and publishing of these experimental works (such as installations and project demos) are still secondary or sidelined, where they exist at all. What would it look like to consider non-textual research outputs as first-order scholarly work? The historian David Staley suggests the terms “interpretive objects” or “humanistic objects” for creative scholarly acts that are not limited to text;¹ Catherine D’Ignazio and Lauren Klein offer the broader term “rhetorical objects.”² This kind of innovative work is carefully researched and theorized. It deserves scholarly engagement and intellectual rigor even if it does not fit into established modes of scholarly communication.

Academic research has a long history of textual practice and citation that we haven’t yet figured out how to adapt to non-textual scholarship. Both the Journal of Open Humanities Data and the Journal of Open Source Software can be seen as steps in this direction: they provide venues for the review and publication of data and software, respectively, accompanied by brief “metapapers.” But even these journals rely on transforming the content they review—data and software—into text in order to function! What are the implications if we truly expand the range of accepted scholarly outputs to include such interpretive objects as data structures, databases, software, datasets, physical objects, and augmented reality experiences? Will scholars need to become experts in all these modes, or can we find a way to become conversant in multiple forms of argumentation, as with other important scholarly theories?

The pieces that follow describe four different data physicalizations, which we consider to be one class of interpretive object. This is an exploration of our work as we wrestle with how to present physical objects in a non-physical medium, objects meant to be held, touched, or viewed from different angles. Not quite metapaper, manual, or manifesto — these are guides toward reading and thinking in creative new scholarly modes.³

Data physicalization represents data in physical form.⁴ Like other approaches to understanding and representing data, it highlights particular senses to communicate information, specifically touch and sight.⁵ But data physicalization is distinct from other sensory approaches in that it bridges the gap between creative, physical, and conceptual exploration, a nexus often associated with critical making.⁶ Data physicalizations surface the amount of labor involved with data production and representation; they lend data different perspectives and dimensions.⁷ These physicalizations also create an opportunity for viewers to become active participants in the making of a piece using data.

Data physicalization attempts to defamiliarize us from the many two-dimensional data representations we have seen by literally placing data in the mise en scène of a conceptual exploration. There is something unique about turning data points into physical forms and placing them in space, something that triggers the mind to understand data in a distinctive way.

There is an ethics of drawing on other senses. Feminist philosopher Donna Haraway describes “visualizing technologies” as the “god trick of seeing everything from nowhere.”⁸ Klein and D’Ignazio expand on this, demonstrating that the assumed neutrality and objectivity of even the simplest visualizations always come from a particular perspective, usually a dominant cultural view that fundamentally excludes and marginalizes.⁹

This is especially the case when making data visualizations accessible to vision-impaired readers. The typical solution is to provide a table with the data underlying the chart or graph. This isn’t practical for large datasets, and it’s clearly not the same experience; otherwise, we would provide the tabular data to all users. Another approach is to provide an extended description sharing the insights gained from the chart.¹⁰ This is helpful, but pre-digesting the chart in this way doesn’t allow readers to perceive and interpret the patterns and draw their own conclusions. Tactile data physicalizations provide sensory forms that offer individuals the opportunity to explore and discover patterns in the data for themselves.

Approaching an object like a data physicalization on display encourages bodily engagement in physical space. It encourages the person encountering it to consider multiple angles and perspectives, and it should raise questions about how the objects are meant to be read. This touching requires proximity and a certain intimacy.¹¹ For Emmanuel Levinas, the “ocularcentrism” of western civilization has produced a false sense that vision is synonymous with objectivity; he proposes instead the metaphor of touch as the basis for ethical engagement with the Other. Ocularcentrism requires distance or separation and encourages objectification and mastery of that which is viewed.¹² French feminist philosopher Luce Irigaray extends this metaphor in her notion of the “caress,” which “weds without consum(mat)ing.”¹³ Because touch requires intimacy, boundaries, and consent, it offers connection without the taint of mastery.¹⁴

These are not artist statements because this is not art; these objects might look like Data Art, but the goals and methods are different. This is representation, correspondence, laborious translation. These are our attempts to communicate our goals, to help you to read and interpret these unfamiliar objects, and to be challenged by the potential of physicalization.

We invite you to participate in the embodiment and visible labor of data work. Download the following models and instructions, use your hands to recreate the data physicalizations we developed, or use them as inspiration to make your own interpretive objects. If you make any of these physicalizations, please share them on social media with the hashtag #DataBeyondVision.

Choose an object

Folding in the Lesser Known Members of the Shakespeare and Company Lending Library

Nick Budak, Xinyi Li

Goal

The Shakespeare and Company lending library is best known for its famous members — writers such as Gertrude Stein, James Joyce, Ernest Hemingway, Aimé Césaire, and Simone de Beauvoir. We wanted to highlight the activity of the relatively unknown members — many of them women — who in fact represent a much larger portion of the library’s day-to-day activity and thus arguably better represent it than do the prominent names. This piece makes use of unit origami to create a larger, cohesive form from small folded units, mirroring the relationship between the overall membership of the library and a single member.¹⁵

Description

The physicalization contrasts the activity of the better known members of the lending library — those linked by researchers to an entry in the Virtual International Authority File (VIAF)¹⁶ — with the activity of relatively unknown members with no known authority record.¹⁷ Activity is represented by the total number of borrowing events that would plausibly have brought members into the library, namely checking out and returning books. Names of the lesser known members are printed on the paper used to create the octahedron as a way of corporealizing and “re-humanizing” humanities data.¹⁸ By holding the physicalization in two different ways, the user can “grasp” two separate sets of data: the octahedron (non-famous members) and the cube (famous members). The ratio of the volumes of these two solids reflects the use of the library by these two different groups.¹⁹

Insights

A pie chart can be used to present the same ratio of data conveyed in the physicalization; this representation is useful when we want to illustrate a situation where we know the totality of the data. Pie charts also depict a world where data fit neatly into mutually exclusive categories. The act of grasping the two intersecting solids in our physicalization is a response to this approach: the membership data of the lending library is a work in progress, updated as researchers comb through archives that are fragmentary and incomplete. One cannot see all sides of a three-dimensional solid simultaneously. By representing our data as intersecting solids, we instead mirror the fuzzy distinction between “famous” and “non-famous” members, acknowledging the intersection of the varied identities of the library’s members.²⁰

Next Steps

Using cut, punched, or embossed paper would make the piece more tactile; instead of simply printing names, we could add unique patterns to represent membership and borrowing activities for individual members. In the future, we could use generative methods to create unique folding patterns for individual library member activity and make them available via print-on-demand. This would enable viewers to become participants and turn folding into an act of recovery of the stories of the lesser known library members.²¹

Choose a different object

Modeling Shakespeare and Company Library Membership

Rebecca Sutton Koeser

Goal

This data physicalization demonstrates the affordances of three dimensions for representing data: time series data are displayed with sequential months and years adjacent to each other, which makes it easier to discern seasonal and annual trends. I hope to inspire others to try experimental approaches to representing data; writing software to generate printable 3D models directly from the data makes the process reproducible, and may eventually enable others to create and print their own physicalizations. The tactile nature of the object suggests the possibilities of 3D printing to create more accessible representations of data.

Description

This is a two-variable, three-dimensional lollipop chart showing the membership of the Shakespeare and Company lending library and bookshop, by month and year, from November 1919 when Sylvia Beach opened her bookshop to its official closing in 1941.²² Membership data are drawn from two different sources, both of which are incomplete: broad membership information comes from logbooks (although logbooks for 1930, parts of 1931–32, and 1937 are missing); detailed borrowing histories come from lending library cards for a subset of members.²³ The white octahedrons represent the number of members with an active membership in each month; the green icospheres correspond to the number of members with borrowing activity in each month.²⁴ For any month where the value is zero, there is no lollipop. Representing the two different datasets as adjacent, half lollipops exposes the discrepancies between the stories these sources tell us about the membership of the library without privileging either of them.²⁵ Using two different shapes makes the two parts of the physicalization distinguishable to touch. The two lollipop charts are designed to be printed independently and then assembled, so that any 3D printer can be used. In this version, the two pieces slide together; this is both a simplification and an improvement over the previous version, where one piece was placed on top of the other.²⁶

Insights

The same membership data can be presented in a two-variable bar chart. Overall trends are easy to see, and both representations of the data make it possible to compare the two data series against each other. Seasonal trends are visible in the bar chart, but it’s difficult to identify distinct months; in contrast, changing perspective on the 3D lollipop chart allows us to focus on yearly or monthly trends. Missing data in one variable are visible in both, but seem more striking in the 3D version where the base of the piece is bare without any lollipop tops. At the current scale, touching the piece requires focusing attention on just a portion of the object but invites exploration, which can proceed in any direction.²⁷ While the bar chart demands sequential reading from left to right, the 3D printed object doesn’t provide or demand a particular starting point or sequence.

The bar chart conveys a sense of certitude and exactness that does not reflect the missing and partial data that underlie it; the 3D printed object, with its irregularities and fragility, is more representative of the contingent, historical data.²⁸

Next Steps

The current version uses different shapes for the two variables, but adding textures would make the model even more tactile. Simple 3D printed labels with text and braille have been added for display alongside the piece, but they could be incorporated directly on the model, and refined to provide a scale for the axes.²⁹ The 3D printed objects could also be augmented with other media: lights or sound could convey the intensity of borrowing activity, or threads connecting months could represent the number of subscription renewals and convey a sense of continuity. The Python code used to create these models could be generalized for reuse, and eventually made available as a Blender plugin.³⁰ New shapes or approaches could be used to leverage innovations in 3D modeling, such as generative design, to create objects that are more inviting to touch and even more distinct from 2D data visualizations. 3D models could be revised for fabrication with CNC machines to create objects out of wood instead of plastic, which could make them more inviting to touch.

Make one >

Choose a different object

Weaving Derrida’s References

… we all of us, grave or light, get our thoughts entangled in metaphors… George Eliot, Middlemarch

Rebecca Sutton Koeser, Gissoo Doroudian

Goal

With this piece, we aim to literalize the metaphor of weaving as writing, embedded in the very words “textile” and “text,” by representing Derrida’s intertextuality as a woven tapestry.³¹ The textures of the yarn and woven fabric invite touch, but by showing an in-progress weaving with the pattern and instructions provided, we move viewers beyond seeing and touching to enable them to become participants in reconstructing the data. Showing the weaving in progress also foregrounds the labor of data work, since curation, collection, and visualization all take an enormous amount of work and skill, often from a range of different individuals.

Description

This weaving represents the references in Chapter 1 of Jacques Derrida’s de la Grammatologie (1967). The references have been cataloged and categorized by the research team of Derrida’s Margins.³² Each type of reference (epigraph, citation, quotation, footnote) is represented by a distinct yarn and weaving pattern. Derrida’s highly intertextual writing suggested the idea of weaving.³³ Using yarn to symbolize the foundational work of deconstructionism, which operates by finding the place where a text unravels, gives additional depth to this physicalization.

Working with textiles is often stereotyped as female activity; therefore this piece also raises questions of gender and other false binaries such as art versus craft, high- versus low-tech. Based on anthropological research, women produced most of the textiles in the ancient world, but that work can be read as female authorship involved in the earliest textual practices.³⁴ The loom itself runs the gamut from high to low technology: a backstrap loom can be assembled at home from dowels, rods, and cords; and yet, Joseph-Marie Jacquard’s 1801 power loom, which used punch cards to automatically create elaborate woven patterns, was an important precursor to early computers.

Insights

The data encoded in the weaving could be represented as a stacked bar chart, a familiar and easily available choice for communicating types and quantities. However, the bar chart may be the least effective for communicating the depth and conceptual nuance of data on multiple levels.

The ability to feel the density and frequency of each type of reference, color coded above, creates a unique experience for each participant, specific to their own perspective. These organic experiences bring to light the depth and complexities of the original work as well as the labor involved with gathering this data. This woven piece, which represents the first half (thirteen pages) of the first chapter of De la grammatologie is thirty-seven inches long, a little more than three feet. The physical nature of this data representation required that materials and dimensions be carefully calculated and measured.³⁵ The process of creating this piece is embodied and experiential, which naturally leads to conversations that effortlessly surface the labor of data work and the depth of the original text. Unfortunately, this is less likely to happen naturally when creating data visualizations.

Next Steps

Adding conductive thread and sensors could turn the weaving into an interface, so that touching the fabric would bring up the relevant reference on an associated screen. Data weavings could also be augmented with other media, such as light and sound, to convey other aspects of the same or related data. Incorporating other work on automated weaving and knitting machines would add to the variety of options for data textiles.

Make one >

Choose a different object

Stacking New and Continuing Membership Activities of the Shakespeare and Company Lending Library

Xinyi Li

Goal

This piece aims to reveal the continuity and growth of Sylvia Beach’s lending library and bookshop by showing the extent of activity and recorded membership based on logbooks and lending library cards. Multiple variables are encoded in the dimensions of stacking boxes based on the technique of pop-up box folds. By exhibiting the evolution of the library over time while contrasting activities of new and old members, this piece enables multiple ways to compare and interpret the data. By transforming a flat surface to a three-dimensional form with play of light and shadows, this production technique serves as a metaphor for the larger purpose of the Shakespeare and Company Project — bringing archival data to life and facilitating rich interpretations.

Description

Shakespeare and Company Project lending library membership data from 1919 to 1941 are represented as a hybrid of time-series and stacked bar charts showing part-to-whole relationships made from paper and folding. Each unit, a cuboid in space and sometimes its stacking child, represents one year and displays nine variables for that year. The height corresponds to the number of active members recorded in the logbooks; the depth depicts the number of members with borrowing activity, according to each member’s lending library card; the length along the timeline is based on the total number of borrowing events.³⁶ Each of the variables is split into two parts: previous members who have renewed a membership contrasted with new members. The upper portion shows the growth and the activities of new readers. Viewers can see the rise and fall of members, inspect the difference between members with borrowing activity and the members as represented in the logbooks, compare the growth over time by viewing the stacking part from the front, and survey the involvement of continuing members versus new members, to name a few possibilities. In some cases, a small number of new members were very active readers based on their borrowing activity.

This physicalization made use of kirigami technique, which transforms single sheets of paper into three-dimensional forms. Data was mapped to the shapes with Data Illustrator, semi-manual calculation, and vector drawing. The materiality and the process have a poetic connection to the subject matter, metaphorically: stories and knowledge can be embedded on flat papers, but the act of reading unfolds the surface into interpretive space even beyond three dimensions.

Insights

With current off-the-shelf visualization tools like the ones that come with Google Sheets, these three data series can generate three separate stacked bar charts. Since the numbers have different ranges, the vertical axes are drawn in different scales, which makes comparison across series impossible. The aggregate version presented here required manual adjustment to combine the separated charts and to make the Y axes comparable. In this 2D version, various activities of the growing membership body are not linked, and it’s difficult to draw connections between active members and the intensity of their activities because spatially these bars are not adjacent to each other. In the conventional pie charts, although the part-whole relationship between new membership activity and all activity is apparent, comparing across the three types of activities is not possible.

Extending into physical space allows data to be encoded in three axes and provides multiple possible angles to view the piece, depending on the relationships one is interested in. Different angles can reveal new interactions between logbook members, members with lending library card activities, and borrowing events. The separation of renewing and new members makes it possible to juxtapose and compare activities between the two groups of members. Spatial factors communicate different facets of the data, and color coding is no longer required.

Throughout the process of developing this project as practice-based research, making and reflecting are in constant oscillation, and knowing happens in actions. As I worked through this piece and observed the artifact, new questions emerged: how do we measure the level of liveliness of the lending library? Is it by the number of members or by the number of borrowing events? Instead of drawing comparisons by the lengths along each axis, perhaps perhaps we might look at the vibrancy of the lending library in a different way: through the volumes and relative sizes of the cuboids and rectangles, which factor in both the number of members and the borrowing activity.

Next Steps

The pattern generation process is very programming-friendly and the materials required are also easily accessible. The data encoding process could be automated by custom code, which could then be made available as a tool for presenting part-to-whole relationships in other datasets. With the addition of dynamic media such as projection mapping, this piece could convey more context and narratives around the lending library.

Make one >

Choose a different object

Acknowledgments

Photos of the data physicalizations on display and Gissoo Doroudian weaving are by Shelley Szwast. The high resolution capture of the data weaving used for the deep zoom and margin image was created by the Digital Imaging Studio, Princeton University Library. The photo of the CDH main space with empty tables is by Mana Winters. The custom visuals for this essay were created by Doroudian; the icons for each section were created by Doroudian and revised by Doroudian and Xinyi Li. All other photos, charts, and models were created by the authors.

Thanks to our collaborators on the Shakespeare and Company Project team and Derrida’s Margins project team for their work to create the data we have experimented with and physicalized here; to the ACH2019 conference organizers for the generous CFP with the option of installations, which led in part to this work; and to the Center for Digital Humanities at Princeton and our colleagues there for the tremendous support and encouragement for this project. Special thanks to our colleagues at Princeton University Library, Meghan Testerman and Annette Jushchuk, for participating in a weaving exhibition test run and giving us feedback to improve the instructions.