“What we have today is a view of knowledge as collections of (verbal and visual) ideas that can arrange themselves into a kaleidoscope of hierarchical and associative patterns-each pattern meeting the needs of one class of readers on one occasion” (Bolter, 91). It has been asserted by Bolter and others that different modes of instruction will produce different characteristics of knowledge. One of the key facets in the debate around the benefits or potential setbacks in multimodal language and literacy instruction revolves around what constitutes the production of knowledge. The debate also frequently touches upon whether that production can be effectively disseminated in progressively non-hierarchical structures (i.e. without the single beginning-to end structure of a standard written text). Within this framework, a contemporary outlook by The New London Group asserts that instructors “are seen as designers of learning processes and environments” (193) which set the stage for the production of this knowledge. There are some localized specifics in some of the propositions regarding multimodal instruction. These may include, for example, how much traditional text is consumed and produced in professional spheres. The majority of the debate, however, is really centered on an ongoing reorganization around what constitutes actual knowledge and the conception of knowledge in the western cultural imagination (Lauer, 23).
The debates on what constitutes knowledge, particularly rhetorical knowledge, often lead to debates on how new forms of knowledge can be measured and categorized. Landow argues that the latest manifestation of knowledge presents itself in something that can be called cross-spatial or hyper-spatial ways of reading. For his example, Landow presents the contemporary scholarly texts as ones “that situate themselves within a field of relations, most of which the print medium keeps out of sight and relatively difficult to follow, because in print technology the referenced… materials lie spatially distant from the references to them” (Landow, 5). Assessing the printed word as the restrictive hierarchical mode of knowledge production, Landow takes something he refers to as “Hypertext” (in the way he formulates his description, the topic brings to mind mostly the internet but not exclusively) and explains how the knowledge produced by engaging with hypertext differs from the printed word.
To Landow, the availability and interconnectedness of texts provided on the network he describes creates “individual references easy to follow and the entire field of interconnections obvious and easy to navigate. Changing the ease with which one can orient oneself within such a context and pursue individual references radically changes both the experience of reading and ultimately the nature of that which is read” (5). Landow then gives a hint of the nature of knowledge produced by the interconnected, intertextual nature of hypertext. This version of knowledge is a knowledge based on navigating between several interconnected texts instead of absorbing information from one, carefully designed long form text in a beginning to end reading.
Landow’s model is only one potential mode of non-traditional textual knowledge production. Bolter characterizes knowledge in the cultural context of the emerging digital age in a different way. Bolter asserts that when readers “refashion the book through digital technology, we are diminishing the sense of closure that belonged to the codex of print. Various electronic devices… pay homage to the printed codex and other paper based materials, while at the same time aiming to supersede them” (Bolter, 79). To Bolter, the chief characteristic of knowledge produced through digital text is in the lack of closure in regards to a certain topic as opposed to a beginning-end design prompted by the long form print format. The defining characteristic of this new non-traditional textual knowledge, in Bolter’s view, is this continued production of knowledge without any significant closure.
Numerous other variations of knowledge can be produced through modes that are visual, textual-visual, textual kinesthetic, textual-visual-kinesthetic etc. All of these variations can potentially produce different skills/paradigms and, hence, slightly differing variations of knowledge. So far, this re-constitution of our definition of textual comprehension has begun to, step by step, integrate other media. There are other elements of how understanding meaning, and to a broader extent, knowledge, is produced through integrating multiple media into a single design. Multimodal textual production can in itself be an emblematic act without any literal assertions. This gets into a reframing of the definition of what constitutes an act as rhetorical. Shipka et. al argue that there exists a necessity of what constitutes a rhetorical act, primarily based on broadening the horizons of mediation (7). The mediation can be found in the distribution of the text, it’s delivery, it’s intended effect on an audience, whether that was accomplished or not etc. Shipka et. al would most likely base multimodal rhetorical practice not so much in learning outcomes but rather effectiveness of the rhetorical act, which could be a paper, an email, a film, a gesture etc.
This framing of the rhetorical act also serves as one of the key issues surrounding the issue of multimodal composition instruction. The argument of skeptics is centered on the question of what sort of textual composition can the students produce after such instruction, can the texts be a traditional five paragraph essay, textual-visual projects, if so how would research be integrated into such projects? This also brings up the issue of the merit of multimodal instruction, its benefit to students entering the varied corners of the labor force and what rhetorical skills will provide students with the acquisition of appropriate cultural capital. To address this, I propose to examine the direct lineage of employment of non-traditional textual forms into language instruction to the point where new variations of knowledge and skills are produced. This is to be a process-focused assessment, zeroing in on how certain designs for language instruction settings and curricula can affect the resulting skill sets acquired. Through tracing the process of the production of knowledge, I hope to manufacture a clearer view of what this new cultural context for rhetorical knowledge and mastery of language constitutes. Is this new knowledge to be more dynamic and less hierarchically subservient, as Landow suggests? If so, how is this manifested in the classroom results (if at all?) and how do the new skills present themselves in the students?
Getting a more concrete view of this process can be derived from three sections. These sections will consist of a selective assembling of theoretical works, classroom designs and a few academic frameworks outside the sphere of multimodal composition. The works outside the field will be kept to a minimum (two at the most). This is very much in line with Shipka et al. referencing Vygotsky, definitely an outsider to the field of rhetoric and composition, in their attempt to reshape the traditional rhetorical cannon. Their use will be to possibly shed light on the use of media not often considered as optimal for multimodal instruction. This field can serve to unite the pragmatic with the conjectural, define points in the linear process from conception of multimodal composition designs to the creation of new modes of knowledge and shed light on what rhetorical knowledge in a new cultural context can constitute. There are numerous steps in comprehending this new form of knowledge derived from multimodal literacy. The key steps have to do with understanding what some working definitions for multimodal composition are, what role these definitions play in designing the mode of composition and what ramifications potential deviations from these definitions may have.
Since the concept of multimodal composition is still relatively open to interpretation and its parameters are up for debate, it is important to define the pedagogue’s present mode of multimodality. Out of the three steps, this one is the first challenging. In his analysis of what constitutes multimodal composition, Kress attempts to encompass the entire sphere of representation. According to Kress, “representation, especially in the linguistic modes of speech and writing, is… bound up with social and ethical values [and] cannot be debated at the level of representation alone. It does, always, have to be seen in the wider framework of economic, political, social, cultural and technological changes… representational changes do not happen in isolation” (6). To Kress, the formal composition or modality of a representation is inseparable from its ideological and ethical perspectives.
Kress also follows the standard Marxist/Constructivist model in assessing the perception of skill in representation as a product of the material circumstances of its time and place. With that Kress asserts that there is an ever evolving paradigm on what constitutes skill in representation. Kress even notes alterations to the lexica describing representation within a span of twelve years. He notes that “there are… revealing changes in the principles of representation and organization: from the densely printed (relatively) mono-modal page to the multimodal screen and the new pages; from the conventions of page production to the mode of layout; from writing as dominant to image as dominant” (11). This perspective can make narrowing down of what constitutes acceptable parameters for modality in composition a possibly overwhelming task. However, this perspective can be used as a reference point for a pedagogue who becomes too stagnant in their conception of skill in representation.
Modality could be studied from the perspective of how the text is designed and what effect the design might have on the consumer of the text. Wysocki looks at textual productions from the perspective of the way designs are construed in the contemporary cultural climate, her first example being an academic paper written in crayons being looked at today as unacceptable. Wysocki firmly asserts that “a particular visual space has become natural to how we now read. Space between words has not always been a function of written texts in the West. Our current practices of spacing text on a page developed over hundreds of years” (56). Wysocki echoes Kress when she acknowledges that not all modes of representation are created equal and that the effects of representation are not all entirely relative. While there are rhetorical benefits from comprehending ever-expanding modes of representation, deviation from the traditional linear textual format presents gain and losses in different forms of knowledge.
The nature of the knowledge produced by multimodal composition could range from hypertextual, as Landow presents, to audio-visual. There is also the question of balancing different types of skills of meaning making in the student, and which ones will be most economically viable. For this, parameters of what constitutes multimodal composition instruction in the context of a specific setting do need to be set. There are numerous issues that could and have arisen from this necessity in the field. One of the more prominent is the differentiation of the use and creation of multimodal texts versus the mastery of multimedia. Lauer sets the parameters between the two in the following way: Multimodal generally refers to several modes of communication produced at once (audio/visual) while multimedia refers to several modes of media working separately (24-28). Both designations leave a great deal to the imagination when it comes to combining different modes of textual production. Nevertheless, clear parameters need to be set for what constitutes multimodal textual production in the present moment.
Defining the set parameters is necessary, primarily, to understand their effect on the production of knowledge. Will a student be able to communicate effectively through images, web pages, both… where does that leave traditional texts etc. Again, this is only a single, general, example of parameters being set by instructors or scholars on what will constitute multimodality in their research or instructions. There can be more specified parameters set as, for example, the design will involve mixing in textual, live visual and performative kinesthetic communication, audio and visual recordings will be omitted. Shipka et al. argue for an even more open approach, introducing conceptions of what constitutes representation and listening to suggestions and rationales about why certain modes of representations would constitute an effective rhetorical act.
After the definitions are set, the second primary determiner of how knowledge is produced will be based on how the parameters manifest themselves in classroom and/or assignment design. This has to do more with specifics of what exactly will be required of students to communicate and through what media. Some argue that this could boil down to the matter of the classroom design in itself. Sirc points out “that simplified programs, programs that ignore the complexity and contradiction of everyday life, result in bland architecture… the spaces of our classrooms should offer compelling environments in which to inhabit situations of writing instruction” (42-43). Sirc goes from discussing how the design of the physical space of the classroom will affect the production of student text and knowledge to how various visual and auditory tones affect student involvement in discussions (49). The design of the classroom’s physical makeup is only one manifestation of certain definitions of multimodal instruction manifesting themselves in practice, however.
Some of the design selections are not as general as to affect the entire space where the learning is supposed to take place. There is a significant difference between asking students to do a conceptual analysis of Gaughin’s poetics in an audio/kinesthetic presentation and to do a brief video on personal identity. Many of the more pragmatically minded theorists will focus on the design of specific, in-class assignments or research projects. Bailey and Carroll, as one example, start by introducing objects that represent certain portions of one of their assigned tasks for a single class, to have a kinesthetic oriented textual analysis assignment (79). This can later be expanded into a research project based on material objects, and that research later will be expressed through verse or song lyrics (Bailey and Carroll, 82). Whether attributed to broad spaces and long term commitments or single lessons, the specifics of the designed course and assignments can actually set the expectations for what modes of knowledge will be produced.
Many of the initial arguments for the benefits of multimodal composition instruction were based on the perpetually changing modes of communication in the professional world and beyond. As the speed, with which new media are constantly being reinvented, increases in late stage capitalism, instruction for textual comprehension and production evolves at a similar pace. This is why it is beneficial to sometimes look outside to related fields and to look at methods being employed using multiple modes of communication that could be utilized in language instruction. An example presents itself in the work of Mark Whale.
Whale uses notable theoretical models from literary theory for his students to view the hermeneutical process of what he calls “encountering” Beethoven’s music (33-34). His articulation of the process can be read in, at least, two ways. On the one hand, the theoretical work’s use in understanding textuality sheds a greater light on the function of some of Beethoven’s pieces in contemporary culture, but also, interpreting Beethoven’s music through the lens of textuality might give students who are otherwise unfamiliar with literary analysis an understanding of how to dissect complex forms of cultural texts, in this case through complex musical forms. This would be an example of students showing a comprehension of the theory of textuality and being able to write from this theoretical standpoint. While they are familiar with the theoretical outline their familiarity would not be with literary, but instead, musical forms. In this sense the students would obtain some of the essential components of writing on this topic by using platforms that are not common in literary analysis.
For the purpose of avoiding intellectual and cultural stagnation, there may be inevitable points of deviation from initially set parameters on the part of the instructor and the students. Some of this deviation can emerge from new media being introduced in the classroom, possibly from other fields. Some of the deviations can emerge from ever evolving cultural definitions of knowledge, necessitating new modes of language rhetoric instruction. The deviation from the parameters set by the initial definition will give insight into how successful the instruction was in producing a skill set, which includes adaptability to new forms of meaning-making in the students. Assuming adaptability to new genres of expression is one of the learning outcomes.
It is through the process of definition, design and deviation from designs, similar to the process advocated by the New Lonon Group, where the characteristics of newly produced knowledge and skills can be identified. The formal characteristics of the skills identified as new forms of knowing in rhetoric and language serve as an overarching characterization of the ever changing definitions of knowledge in an ever evolving cultural context. The comprehension of this process might then produce insight for instructors on how to adapt to this ever-evolving context more consciously. This would be an adaptability very necessary in an increasingly speed-driven, multicultural society with definitions of knowledge morphing, emerging, re-emerging and evolving at a speed, possibly, never before known in history.
Avillez, De, Fisher, Klotz, and Long. “Public Philosophy and Philosophical Publics: Performative Publishing and the Cultivation of Community.” The Good Society 24.2 (2016): 118-45. Print.
The authors chronicle how the practice of philosophy has evolved in the public sphere through analysis from Kant to Dewey to a few contemporary theorists. There is a section on the digital age that covers how new modes of technology can bring elements of philosophical and rhetorical discourse to a continuous debate in the public sphere, there is a discussion of how new modes of communication will affect that discourse.
Bolter, Jay David. Writing space: the computer, hypertext, and the history of writing. Hillsdale N.J.: Erlbaum, 1991. Print.
Bolter makes numerous historical connections centered on the physical make up of text and what effect that makeup has on the text’s meaning. The connections bring examples of setting concrete parameters on the mode of textual production and also make concrete connections to the type of literacy exposure to the mode of text may produce.
Bailey, Nancy M., and Kristen M. Carroll. “Motivating Students’ Research Skills and Interests through a Multimodal, Multigenre Research Project .” The English Journal 99.6 (2010): 78-85. Print.
Bailey and Carroll outline several variations of research projects utilizing different modes of meaning making. They emphasize several methods of divulging meaning from facts and outline a process oriented assignment with several steps particularly focused on analyzing the implications of information found.
Curwood, Jen Scott, and Lora Lee H. Cowell. “IPoetry: Creating Space for New Literacies in the English Curriculum.” Journal of Adolescent & Adult Literacy 55.2 (2011): 110-20. Print.
The authors focus specifically on poetry and creative writing and how to define a text as poetic. They then outline classroom discussions and assignments on what constitutes poetry and discuss several modes they encourage their students to integrate in their creative writing endeavors.
Dalton, Bridget. “Engaging Children in Close Reading: Multimodal Commentaries and Illustration Remix.” The Reading Teacher 66.8 (2013): 642-49. Web.
Dalton argues for several effective uses of multiple modes of meaning making to encourage close reading. She zones in on a particularly diverse array of commentaries, annotations and continuous dialogue possible through hypertext.
The New London Group. “A Pedagogy of Multiliteracies: Designing Social Futures.” Harvard Educational Review 66.1 (1996): 60-93. Print.
The New London Group outlines the process of producing knowledge and how its merits are generally decided by the cultural context in which it is produced. Their analysis is process-oriented, discussing several modes of knowledge production an what sort of possible effects each one of these modes may have.
Whale. “How Universal is Beethoven? Music, Culture, and Democracy.” Philosophy of Music Education Review 23.1 (2015): 25. Web.
Whale discusses how the unique formal amalgamations of Beethoven’s Ninth may sometimes make its music meritorious even for some who are distant from the cultural context in which it was produced. He relies on a great deal of literary theory to analyze the effects of a complex musical text, at the same time using the mode of classical music to effectively understand textuality.
Hundley, Melanie, and Teri Holbrook. “Set in Stone or Set in Motion?: Multimodal and Digital Writing With Preservice English Teachers.” Journal of Adolescent & Adult Literacy 56.6 (2013): 500-09. Print.
This is an empirical study where the authors observed the effect of several multimodal projects and their effects on learning outcomes in the classroom. The authors come away with several conclusions, most notably the value that digital literacy has in language instruction and how those exposed to more visual imagery are more receptive to ambiguity than those gravitating more towards printed text.
Kress, Gunther. “Gains and losses: New forms of texts, knowledge, and learning.” Computers and composition 22.1 (2005): 5-22.
Kress outlines the effects of ideology and material conditions on new modes of communication and their effects. He also describes how the different modal amalgamations may affect the meaning and reception of the texts. Finally, he touches upon how the new mode of knowledge may have, what could be viewed as, both intellectually enriching and impoverishing effects.
Landow, George P. Hypertext: the convergence of contemporary critical theory and technology. Baltimore: Johns Hopkins U Press, 1996. Print.
Landow very accurately foreshadows the dynamic of textual production in the digital age and how it has and may manifest itself in the future. He sets a very effective theoretical foundation for hypertext, where variations of it might have existed historically and what potential effect it might have on textual consumption, comprehension and production going forward.
Lauer, Claire. “Competing with Terms: “Multimodal” and “Multimedia” in the Academic and public Spheres.” Computers and Composition 26 (2009): 225-39. Print.
Lauer traces the history of the two terms in her title and emphasizes the difference between the two. She then argues the ways in which one is utilized in the world outside of academia and one inside academia, and what are the benefits in being skilled in both.
Pandya, Jessica Zacher. “Unpacking Pandora’s Box: Issues in the Assessment of English Learners’ Literacy Skill Development in Multimodal Classrooms.” Journal of Adolescent & Adult Literacy 56.3 (2012): 181-85. Print.
Pandya discusses the benefits and potential problems that may arise in assisting the work of English Language Learners with imagery. She points out how several multimodal designs can potentially assist L2s in language acquisition as well as cultural competency but points to many problematic points in the analysis, particularly assessment of effectiveness.
Prior, Paul, et al. “Re-situating and re-mediating the canons: A cultural-historical remapping of rhetorical activity.” Kairos 11.3 (2007): 1-29.
The authors present a broad overview of how to re-imagine what is presently considered in contemporary western thought as rhetorical knowledge. They present many variations on how to alter the present paradigm while leaving many questions as to how we view rhetorical knowledge.
Selfe, Richard J., and Cynthia L. Selfe. ““Convince me!” Valuing Multimodal Literacies and Composing Public Service Announcements.” Theory Into Practice 47.2 (2008): 83-92. Print.
The authors argue for the professional benefits of students learning numerous skills through multimodal projects in the public sphere. There is a distinct emphasis on students communicating various notions to various publics and learning to effectively use several methods of persuasion through different modes of expression.
Serafini, Frank. “Expanding Perspectives for Comprehending Visual Images in Multimodal Texts.” Journal of Adolescent & Adult Literacy 54.5 (2011): 342-50. Web.
Serafini begins by introducing the notion of acquiring textual comprehension skills, particular ones oriented towards formal literary analysis, using techniques learned to understand visual media. He particularly emphasizes the benefits of these forms of analysis when it comes to understanding metaphors and motifs.
Sirc, Geoffrey. “The Still-Unbuilt Hacienda.” English as a Happening (2002): 1-32. Web.
Sirc chronicles the evolution of “happening” classrooms and discusses various potentials for classroom design, including in the special realm, and how it can be benefitial in producing knowledge. He takes a macro approach outlining the necessity to pay attention to classroom architecture, space and sound amongst, time inside the classroom and many other elements.
Wysocki, Anne Frances. “awaywithwords: On the possibilities in unavailable designs.” Computers and Composition 22.1 (2005): 55-62.
Wysocki reviews how the guidelines for what constitutes appropriate formats for texts in numerous contexts are in fact social constructs. She gives numerous examples of how the appearance and order of the text influences the effect it has on the reader and gives suggestions for expanding away from some of these socially constructed paradigms.