Voices From The Field

The TERC team that is hosting the STEM for All Videohall events (this year’s begins on May 15th, at http://stemforall2017.videohall.com;  last year’s event is archived at http://stemforall201.videohall.com) began experimenting with video presentations some years ago in an electronic community for graduate students in the IGERT program (IGERT.org).  The videos started as accompaniments to on-line posters, which the young scientists developed to share their work with their colleagues and  with experienced scientists.

At the time, my curiosity was piqued by how the videos and the posters complemented each other. I think my first sense was that the videos were great at engaging attention, and  supporting the IGERT students in their intent to gain skill in speaking about their work to non-specialists.  The trend of science journals permitting or encouraging video abstracts was under way and attracting attention (see a widely-cited article by Berkowitz on video abstracts  here).

Some people have taken to this opening “multi-modality” easily — at least as viewers/hearers;  others have been slower to embrace it, laden as it is with the need to learn some new techniques, authorial skills, and technology.  But expertise is spreading rapidly, and guidelines and standards are emerging (see for example this study on video abstracts, and the website “The Scientist Videographer,” which has much useful advice and critique, as well as links to other resources; also see TERC’s Videhall.com for diverse examples of videos from recent years).

But then, as I saw more and more videos, I realized that, when a video was able to convey something substantive about the importance, context, or methodology of the research, I was being reached on several channels — the typically text-and-graph heavy poster, and the narrative, visual, and somehow participatory video.  When conversation was added (for example if I posted a question, as a visitor or as a judge for the competition), the exchange was mostly in text (I write a comment, the presenter answers, etc.), it felt like a different kind of text than the text of the poster — conversational, less stylized, often more exploratory or “think aloud,” even if the messages were composed after some reflection.  It is indeed an example of dialogue-as-inquiry, a continually fruitful idea since at least the time of Plato, and something that is enacted every day in classrooms, workplaces, laboratories, and other forums.

And another thing:  When you listen to scientists talk informally about the ideas or phenomena in their areas of interest, you often see gestures and other kinds of body language, hear sounds imitated, see phenomena (including experimental procedures or field experiences) re-enacted — all accompanied by the variety of facial and vocal expressiveness that are inevitably part of animated conversation.  This, too, is part of how science is done, and sometimes it is these dramatic performances (if you will) that engage a learner, make him or her realize that this is something to be excited about and to enjouy — maybe to participate in.    In many videos, that part of the great dialogue that is science is captured, the same excitement stimulated, the invitation to connect is extended.   This, I think, is yet another channel, another mode of exchange — dramatic or mimetic or even theatric.

How do all these work together?  I found myself rereading a piece by Jay Lemke, “The literacies of science,” which I recommend.   Lemke, trained as a physicist, has joined to that background some serious inquiry into semiotics, ethnography, and science education, and the multimodality of his interests and inquiries often produces very interesting reflections.  In this piece, from 2004, Lemke explores  ideas such as those I have been sketching in this blog post, and reinforces me in my sense that more meaning, not just more information, can be conveyed using multimedia, multi-modal methods.

Lemke points out that science (his focus and mine, though he also draws examples from mathematics) as practiced is inherently not a verbal enterprise, however important words are as one component:

It is often said, by scientists, that mathematics is the language of science, but it would be closer to the whole truth to say that the language of science is a unique hybrid: natural language as linguists define it, extended by the meaning repertoire of mathematics, contextualized by visual representations of many sorts, and embedded in a language (or more properly a ‘semiotic’) of meaningful specialized actions afforded by the technological environments in which science is done. The texts of science are not written in any natural language studied by linguists. They are written in as much of this hybridmeaning-making system as can be presented on paper or animated on a computer screen

This is because science is about the world, and the world is a nonverbal thing:

The world makes meanings that go beyond what natural language can say: our proteins, our cells and their membranes do; organisms of other species do; ecosystems do; cosmology does. Science is the great enterprise of paying attention to the kinds of meanings that require us to go beyond natural language.

I suspect that many  students are discouraged from science, get the feeling they can’t do it, or understand it, because so much of the world, and of the investigation of the world, is not included in science education, with honorable exceptions — laege chunks of the “message” just are never made available.  Sometimes those chunks can “engage,” but sometimes they might indeed bring the kind of insight about a subject (phenomenon, puzzle) that excites in ways deeper than is usually meant by “engagement.”   As Lemke writes,

The whole of meaning, the whole of communication is an evolved human capacity for survival in a physical and biological world. The whole of communication includes gestures and posture, facial expressions, mime, nonverbal vocalizations, drawings, and a great deal more. What can you communicate with a gesture that you cannot say in words? What can you represent with a drawing or a map that cannot be said? Even speech is more than language: we vary the timbre and pacing of our voices, the sharpness and force of our articulation in ways that convey emotion, mood, health, seriousness, importance, urgency, surprise, doubt, need, desire, and a host of core human meanings essential to our social cohesion and group survival.

It seems to me that this states very powerfully the ground for a multimodal approach to science education — and I should say that “video” is not “the answer,” either.  The opportunities and affordances that video, or video + discourse, or video+discourse+text, offer — the excitement that comes from every expansion or permutation of our dialogues about the world — serve first and foremost as a reminder that “experience” in our world is the ideal  — not just doing and undergoing, as Dewey would say, but immersion and reflective, inquiring conversation (or quarreling) with the world, and with other people (past and present) who are trying to make some more sense about the Whole, and also who we are as a part of it.    To close with a bit more from Lemke’s article:

There are no names in natural language for all the angles from acute to obtuse….There is no way to describe the shape of a mountain or a cloud or a face. No way to precisely describe the twists and turns of a winding path. There are no words to distinguish degrees of speed, or trajectories of motion. There are no words for all the intervals of time that matter in life. There are not nearly enough words for all the degrees of certainty and doubt, importance and urgency, unexpectedness and surprise, need and desire, that matter to us.

Note:  The opinions expressed here are those of the author, and do not necessarily represent those of MSPnet, TERC, or the National Science Foundation.