And now, a few theoretical words on Multimodality

Multimodality is nothing new in the world

But until recently, educator types liked to ignore the implications of visual elements other than print

Words, numbers, and symbols ruled.

But then, something happened: The internet

And now, educators are just ga-ga about theorizing how web pages “mean”

And one of the most useful theories comes from someone who lived and died almost 100 years before the Internet was invented.

His name is Charles Sanders Peirce.

Peirce was a philosopher and logician who worked for the US Geological Survey. He wondered how the raw input of the senses becomes sensible to people (and animals and plants…)

He theorized that input comes to “mean” in one of three ways.

You see this 1. Through its resemblance to other things we have already experienced. And recognize it as a “book” because of its resemblance to other objects in a class that in English we call “books”: Peirce called signs (things that we recognize) that “mean” because they resemble other things we’ve experienced before “icons.”

You hear this: 2. Through its contiguity (placement near) another object in time or space. And recognize it as a siren because of its association with the presence of this: Peirce called signs that “mean” through their association in time and space with other signs “indexes”

You see this: 5 3. Through social convention, a sign is associated with a particular meaning Or you hear this: And it represents the concept of “five-ness” to you: Peirce called signs like these “symbols.” This is the part of his theory that partly overlaps with other theories you may have heard of, like the work of Saussure on language.

Peirce argued that all signs are understood through relations that are iconic, indexical, or symbolic (or a combination of these).

But WAIT!! There’s more.

Peirce also argued that the different sign relations convey meaning at different levels of comprehension.

Icons convey “Firstness” A largely unarticulated but felt sense of the presence of meaning (an intuition, an unexplored emotion) Awwwww….

Indices convey “Secondness” An informational, relational sense of presence Note that an index often combines icons or words (symbols). It is the placement of the sign near danger that makes this sign an index, not the figures that are drawn. The figure itself is an icon of how to properly feed a child to a crocodile.

Symbols convey “Thirdness” Ideas that are fully articulated and that are understood through the use of other symbols. “Let’s discuss…”

In other words, different types of signs convey different types of meanings: emotional, intuitive, directive, rational and argument-based.

That’s why multimodal messages often have so much power: they can “hit you” at a lot of different levels of your being and expand your full understanding of something intellectually and emotionally.

But WAIT! There’s still more! Multimodal texts provide a more complete and authentic experience of a topic as well. They not only “hit you” at a lot of different levels of cognition, those levels SUPPORT each other.

They can lend readers vital clues to the meaning of texts and fill-in missing “holes” in their understanding. For this reason, multimodality is a Godsend to: struggling readers students who speak English as a second or third language (ELLs) even to proficient readers who are being presented a complex topic for which they have little or weak prior knowledge.

That’s right. Multimodality is Da Bomb, too.