WHAT IS VISUAL SOCIAL SEMIOTICS?

Semiotics is generally described as the “study of signs.” For a sign to exist, there must be meaning or content (the signified) manifested through some form of expression or representation (the sign). Figure 1 is a well-known painting by Rene Magritte that demonstrates this relationship in a striking and explicit manner. By putting the sentence This is not a pipe below a highly realistic representation of a pipe, Magritte reminds viewers that the image is not reality but artifice–in other words, a representation or sign.
A thoughtful viewer might note that the word “pipe” itself is an arbitrary combination of four letters that conveys the concept of “pipe” through the form of written expression–once again, a sign.
Signs exist within semiotic systems. For example, the green light in a traffic signal is a sign meaning “go” within the semiotic system of traffic control; words are signs in the semiotic system of language; gestures are signs within the semiotic system of nonverbal communication; and so on. Because semiotic systems encompass the entire range of human practices,
Semiotics provides us with a potentially unifying conceptual framework and a set of methods and terms for use across the full range of signifying practices, which include gesture, posture, dress, writing, speech, photography, film, television, and radio…. As David Sless notes, “we consult linguists to find out about language, art historians or critics to find out about paintings, and anthropologists to find out how people in different societies signal to each other through gesture, dress or decoration. But if we want to know what all these different things have in common then we need to find someone with a semiotic point of view, a vantage point from which to survey our world.” (Chandler 2001)
It is this cross-cutting vantage point that allows professional communicators to compare and contrast objects from two different semiotic systems–language and imagery–and make a valid, useful analysis.

SOCIAL SEMIOTICS
Social semiotics is a branch of the field of semiotics. Lemke notes that social semiotics is a synthesis of several modern approaches to the study of social meaning and social action. One of them, obviously is semiotics itself: the study of our social resources for communicating meanings. … Formal semiotics is mainly interested in the systematic study of the systems of signs themselves. Social semiotics includes formal semiotics and goes on to ask how people use signs to construct the life of a community. (1990, p. 183)
Because every community is different, the signs used by one community may be different from those used by another, For example, the color red indicates mourning for people in Ivory Coast, whereas, in contrast, it represents procreation and life for people in India.
Social semioticians apply three important principles when analyzing a semiotic system such as language or imagery–principles that have significance for professional communicators.
1. Semioticians believe all people see the world through signs. As Chandler explains, Although things may exist independently of signs we know them only through the mediation of signs. We see only what our sign systems allow us to see….Semioticians argue that signs are related to the signifieds by social conventions which we learn. We become so used to such conventions in our use of various media that they seem “natural, “and it can be d difficult for us to realize the conventional nature of such relationships. (2001)
Schriver suggests that successful professional communicators use intuition to “imagine the audience and draw on their internal representation of the audience as a guide to writing…”(1997, p. 156). I would add that this intuitive internal representation” includes a highly sensitized understanding of the sign conventions in a communicator’s particular language semiotic system. This sensitivity contributes to the skills that enable writers to replicate their communities’ discourse in ways that attract interest or please readers.
2. The meaning of signs is created by people and does not exist separately from them and the life of their social/cultural community. Therefore, signs have different meanings in different social and cultural contexts–meanings that can range from very different (for example, different languages) to subtle and nuanced (for example, spoken English in U.S. versus spoken English in India). This principle has profound implications for professional communicators who must write for international audiences. The growing number of books and articles on this subject attests to the difficulties writers face when trying to create messages for people whose semiotic systems are different from theirs.
3. Semiotic systems provide people with a variety of resources for making meaning. Therefore, when they make a choice to use one sign, they are not using another. As Lemke adds, These are the contexts of “what might have been”… In the same sentence, what other words could have been used? At the same point in the game, what other plays might have been made? For the same detail in the painting, what other colors could have been used? (1990, p. 188)
The ability to choose gives communicators a certain amount of power to use signs in unconventional ways and, therefore, affect and even alter meanings.

VISUAL SOCIAL SEMIOTICS
Visual social semiotics is a new field of study (originating in the 1990s) and has been defined by Jewitt and Oyama as involving “the description of semiotic resources, what can be said and done with images (and other visual means of communication) and how the things people say and do with images can be interpreted” (2001, p. 136).
Here is an example of how visual social semiotics can be used as a tool in analyzing an image on a Web site to see if it enhances, or detracts from, the text. Figure 2 is a photograph from the Web site home page for the Supreme Court of the U.S.
The visual social semiotician would note a significant aspect of this photograph–its point of view. The photograph is taken from an ant’s-eye perspective, placing the Court building at a high vertical angle from the viewer. This angle allows the photographer to glorify the Court by emphasizing the grandeur of its architecture and its classical elegance. The perspective elongates the columns and makes the portico more imposing. Moreover, the high vertical angle compels the viewer to look up at the building–a statement about the pre-eminent power of the Court.
It’s important to remember that the producer of this Web page had other perspective choices that could have altered this “power statement” about the Court. For example, we could have seen the building from a bird’s-eye viewpoint, making it smaller, putting it within the context of its surroundings, and reminding us that the U.S. government is “of the people, by the people, and for the people.” Or we could have had a photograph of the building taken on its steps so that we would be looking horizontally toward the interior–a perspective that would imply equality between citizens and the judicial system.
Linguistic social semioticians would look at the text that, in addition to the title “Supreme Court of the U.S.,” includes a list of links such as Oral Arguments, Case Handling Guides, Court Rules, and Opinions. They would note that the document really contains ellipsis–a form of cohesive textual tie that acts by assuming that the reader is able to presuppose meaning despite omitted words–in this case, verbs and adjectives. The full thought behind the title and links is: “The Supreme Court of the U.S. hears legal Oral Arguments, sets court Case Handling Guides, establishes Court Rules, and gives legal Opinions.” The linguistic social semiotician would note that this full thought makes heavy use of nominalizations–arguments, guides, rules, and opinions. Nominalization, the process of creating a noun from a verb,
* Deletes the people who do things and the people to whom things are done
* Eliminates tense (that is, past, present, and future)
* Omits modal verbs (for instance, can, might, should)
When these elements are cut Out from a text, the process (for example, to judge) becomes depersonalized and is rendered as an object (for example, a judgment). Such textual entities are not only impersonal, they also take on the quality of being timeless and fixed, even though arguments can change and judgments have been overturned. On this Web page, the nominalizations contribute substantial weight to the importance and influence of what nine people do for a living.
Let us imagine the team putting this Web page together. They have already created the text for the title and links and are now choosing a photograph. Say they have a choice of three photographs: the ant’s-eye view, a bird’s-eye view, and a “level” view. Which would provide them with the best “tight coupling” to make the page’s visual language most effective? For example, how “supreme” would the Court appear if the bird’s-eye view photograph was used? Or how timeless and fixed would the Arguments, Guides, Rules, and Opinions seem if the perspective of equality were used on the page? Clearly, the team would choose the ant’s-eye perspective to enhance the importance of the highest court of the U.S. judicial system, as already implied by the text.
Note that I have not discussed the size or placement of the photograph on the Web page. Although such features–as well as the relationship of the image to other elements of the page, both printed and online–can have a significant impact on visual language, they are part of page design, an aspect of visual communication not covered in this article. To help readers focus on the content of images analyzed here, most of the images I use in this article are excerpted from Web pages.
Professional communicators should be aware that visual social semiotics may not be able to answer all the issues that an image may raise. For example, 10 different creators could be given the same image to produce and would do so in 10 different ways, creating different effects through aesthetics and style–effects that are not addressed through this particular framework. Nor can visual social semiotics help in understanding the rhetorical role that text itself plays as a graphical image through color, typography, and placement. (See Schriver 1997 for a discussion of how people “see” text.) However, as I hope the discussion of the photograph of the U.S. Supreme Court demonstrates, visual social semiotics can be an extremely useful tool for analyzing images and their relationship to text.