How is humanities and social science knowledge impacted by the introduction of three-dimensional visualization technologies? While 3D visualization may seem far removed from the everyday work of scholars in the social sciences and humanities, it has great potential to change how we conduct and communicate our work.
Three-dimensional visualizations can be used for creating models, supplementing maps, developing games, printing objects, developing virtual environments, enhancing telecommunications, and housing simulations. They can be used to support retrospective and prospective analysis, exploration of counterfactuals, and representation of hybrid or alternate realities, particularly when they combine objects in 3D contexts. An art historian might want to understand how an artifact was perceived in context, or how a built structure looked in earlier eras, or to document an installation or exhibition. An archeologist might use 3D models or prints to complete a broken artifact or to reassemble a ruin. A sociologist might develop agent-based modeling in a 3D space to understand the social dynamics in a given location. A historian might explore 3D viewsheds to determine lines of sight and power. A linguist might construct a virtual environment for language learning. A literary scholar might build out a navigable imagined space as a form of nonlinear literary criticism. A statistician might display data in 3D infographics to aid in interpretation. And of course, artists, architects, and designers of all stripes might use 3D to create new objects and environments as well as use such techniques as a way to study those that already exists. All of these researchers in turn might communicate their work through multimodal, immersive, affective visualizations for public outreach, policy impact, or funding solicitations.
Although the technologies used to create them are daunting at first, these visualizations are becoming increasingly accessible to nonspecialist users, and the underlying conceptual approaches that they highlight are not new to the disciplines where they’re now being used. Designing and representing 3D space and objects in 2D images, text, and other forms comes naturally to us in many fields. Maps, plans, and networks fill the pages of social science research. Where and how people think, live, work, and interact are contextualized in historical and contemporary places, spaces, environments, and geographies. Artists and architects build their maquettes and design their structures and installations. The dimensional space of the stage, performance hall, or theater is a key component of the production. Lighting, acoustics, and movement are all part of the process. Museums and cultural heritage institutions have taken advantage of the rhetorical power of 3D for their interpretative exhibits for years.
Spatial metaphors like point, field, or boundary abound already in scholarly work. The introduction of digital technologies for 3D visualization, however, also means that additional metaphors shift into the foreground, potentially shaping future directions for academic knowledge production as a whole. Techniques like zooming in and out; examining an argument, object, or phenomenon from various perspectives; running a simulation; and exploring change over time become easier and more prominent intellectual and technical approaches. Such visualizations are potentially interactive as well, which leads to a more experiential and even playful approach to knowledge formation and argument building.
There are dangers, however, in taking our visualizations too seriously, as many media scholars and critics would warn. Developments in digital media technologies typically move in the direction of greater similitude, a more perfect representation. Over the last 20 years our online images have become less pixelated, our videos less grainy, and our audio recordings less tinny—at the same time as they have saturated our media landscape through the internet. We are happy about this as consumers, even if we are deeply skeptical of such perfection as critical thinkers. The asymptotic search for completeness is reflected in the ways our new technologies trumpet their advances, and in how our tools themselves are constructed. The asymptotic search for completeness is reflected in the ways our new technologies trumpet their advances, and in how our tools themselves are constructed. Authoring tools for 3D modeling and rendering privilege specificity, exactness, and consistency that may not be possible with our imperfect data, leading to the temptation to fudge for the sake of expediency. In his short story “On Exactitude in Science,” Jorge Luis Borges wrote about a cartographer’s guild that “struck a Map of the Empire whose size was that of the Empire.”1As translated by Andrew Hurley. In subsequent years that dream was in tatters—an object lesson to us all. And yet this dream of perfect reflection of the real—and creation of the convincing alternative—haunts not only cartographers but also those who produce 3D visualizations for film, games, and installations. The endpoint of the visual Turing Test is not knowing where “reality” ends. Higher resolution at scale means greater believability, a more perfect immersion.
For the humanities and social sciences, this combination of a need for perfect data and their convincing reality effects together are what make 3D visualizations potentially disruptive, as well as attractive. Their “scientism” is belied by what we know of how the sausage is made. Similarly, it is not just because they disrupt the focus on text, with its ellipses and habits of long-form narration, but also because they promise a completeness of vision, a holistic view, and, when deployed in immersive environments, a powerful takeover of our senses. The same rhetorical power we see in movies where cities convincingly fall into the ocean and aliens perforate our flesh, in the vacant stare of the video game addict, and in the willingness of otherwise rational people to run into traffic to capture a Pokémon Go creature are also potentially at our disposal as scholars striving to demonstrate the relevance of our work to a wider community. We are well aware we need to be skeptical of these media effects, even as we explore their potential.
Digitization and digital tools make it easier than ever to curate reality, but we have always done so in our narratives and hypotheses. Evidence is selected and presented in the context of a larger argument and narrative flow. Inevitably, we make representational choices based on expediency, priority, bias, and inclination. Data is collected and curated and shaped. Three-dimensional visualization is particularly susceptible to the allure of perfect representation because its tools themselves demand a completeness we must resist if we want to showcase ambiguity, uncertainty, hypothesis, or counter-facticity. On the other hand, the flipside of introducing 3D visualization into the conversation is that its use draws our attention to how we make our arguments, and the lacunae in our sources—correctives that may strengthen the work as a whole.
The complexity of the process of 3D construction contributes to its opacity as well in that it can combine elements from photography, film, and mapping—each of which have their own tricks and biases. When we study photography, we point out that those photos were taken from a vantage point and were manipulated both in the process of their creation and in subsequent edits. The camera-eye of film and video is complemented by the manipulatable camera eye of the software package. Whether we are looking at a virtual world or at a 360-video shoot, the detachment from a singular perspective gives us a tempting illusion of complete authority. The god’s-eye-view of paranoid surveillance culture is the flipside of the desire for perfect reflection and completeness of representation. This awareness of perspective can also become a point of reflection for the researcher.
If 3D visualization introduces a Z-axis into our thinking, then 3D visualizations contextualized, whether through maps, virtual worlds, exhibitions, or narrative description, become sites of exploration and inquiry. Space ventilates our conversations and representations, making them dimensional, encouraging exploration of adjacency and association. And perhaps paradoxically, it is with the introduction of even more dimensions that we can address the biggest concerns about 3D visualizations and their “truthiness,” and at the same time unlock their fuller potentials. When we introduce the fourth dimension of time into our 3D visualizations, we enter the realm of simulation, animation, and potential interactivity, with both reflective and predictive potential. The temporal dimension here is not only the elapsed time of the temporalities being described in our texts and images, but also that of readerly experience with our creations. The temporal dimension here is not only the elapsed time of the temporalities being described in our texts and images, but also that of readerly experience with our creations.If we push forward into the fifth dimension, perhaps we can reintroduce degrees of freedom, which might be expressed as the ambiguity or uncertainty of possible worlds, or perhaps context or the situation of the observer, gravity. However we choose to conceive of it, it is that space beyond the singular representation. The perfect representation of the world cannot exist. What can exist are partial remediations that privilege one aspect of a story over another, taking advantage of the potential forms available, with technical decisions made in dialogue with intellectual goals.
For example, a few years ago our Visualizing Venice art, architectural history, and urbanism research team was studying the cistern network of Venice. We were interested in individual cisterns as physical and aesthetic objects in architected space, cisterns as existing as wells within the environment of a particular campo in which they were placed, and cisterns as part of a larger water network in the city. For us, the engineering details were less important than the fact of their existence in one spot or another, which also tied to human relationships—power, wealth, authority, religion—over time, but other researchers might privilege those aspects when thinking about, for example, the sustainability of the water supply in the city today and tomorrow.
Part of our thinking also related to what we wanted the end point of our representations to be. In this case we were open to various outcomes but knew we would want to be able produce an interactive digital 3D model of a cistern, to place one in a video, and to create a 3D map that included one in situ. For the interactive model that privileged one specific cistern, highlighting its crumbled edges and faded carvings and the flowers growing out of it, we used photogrammetry, a method by which you take a series of individual photographs using camera shots taken from different perspectives (perhaps including drones for bigger objects) and stitch them together to form a 3D object. That 3D object can then be rendered into a mesh with software that enables you to embed it in, say, a web viewer, which in turn allows you to rotate the model virtually and add annotations, or in a mobile app. For these purposes, 3D modeling serves to augment lived experience of the object by giving the user additional information, and free play of the artifact, whether or not they are on site and in front of the original object. The benefits here are a realistic texture that can retain some of the specific nuance of the original. Such objects might include void areas, due to the limits of what the cameras can see, shadows, reflections, etc. The developer then has to decide whether or not to fill in the voids with filler, and how to indicate those lacunae.
Along the way we needed to address the choice of technologies, and appropriate methods. Do we model with polygons or splines—that is, geometrically? Do we fill in the blanks with procedural models (plausible filler) to round out our settings? Do we construct with vectors first, or images? Point clouds or polygons? Amongst modelers the “war” between historical 3D geographical information systems (GIS) and building information modeling (BIM) approaches to 3D visualization inheres in whether it is more important to able to place your models in rich contexts for conducting viewshed analysis or to anatomize your representations for future manipulation and simulation via parametric modeling. Similarly, 3D scanning with point clouds creates a photo-like 3D rendering of objects and places useful for representative purposes, but it is converted to polygons and splines when we want to simulate movement, growth, and change over time.
All of these approaches require a knowledgeable subject-area expert to work alongside the modeler—or increasingly to coexist within the same minds of the scholars who create them— to decide which approaches to privilege. How well the fourth dimension of time and fifth dimension of uncertainty are deployed may also help us determine which choices to make for the first three. In the long run, 3D visualization software packages may become the “word processors” of choice for some future scholars as the novelty of the medium recedes in favor of what it does for us in our work.