A taxonomy of the discourse relations between words and visual

Kenneth Kong*

*Corresponding author for this work

Research output: Journal PublicationsComment / Debate / Erratum

23 Citations (Scopus)

Abstract

A document is mainly composed of words and images, but the complex relationship that binds these two completely different semiotic resources is usually taken for granted as transparent. The simple relations between words and images-'anchorage' and 'relay', identified by Barthes almost 30 years ago-are unable to deal with the complexity of their bond, made even more complex by current printing and computer technology. This paper aims to identify the potential relations that bind texts and images together by arguing for a multilevel description of their logico-semantic relationships. The multiple, evaluative and metaphorical functions of the relations will also be discussed. The data generated from the proposed framework can form an empirical corpus for quantitative analysis. Examples from a variety of sources will be used as examples to show how the framework can be operationalized.

Original languageEnglish
Pages (from-to)207-230
Number of pages24
JournalInformation Design Journal
Volume14
Issue number3
DOIs
Publication statusPublished - 2006
Externally publishedYes

Fingerprint

taxonomy
discourse
semiotics
semantics
resources

Keywords

  • Multimodality
  • Rhetorical structure
  • Systemic-functional linguistics
  • Visual analysis
  • Word-image interface

Cite this

@article{9d004e70dcd240cab28841e76e539e29,
title = "A taxonomy of the discourse relations between words and visual",
abstract = "A document is mainly composed of words and images, but the complex relationship that binds these two completely different semiotic resources is usually taken for granted as transparent. The simple relations between words and images-'anchorage' and 'relay', identified by Barthes almost 30 years ago-are unable to deal with the complexity of their bond, made even more complex by current printing and computer technology. This paper aims to identify the potential relations that bind texts and images together by arguing for a multilevel description of their logico-semantic relationships. The multiple, evaluative and metaphorical functions of the relations will also be discussed. The data generated from the proposed framework can form an empirical corpus for quantitative analysis. Examples from a variety of sources will be used as examples to show how the framework can be operationalized.",
keywords = "Multimodality, Rhetorical structure, Systemic-functional linguistics, Visual analysis, Word-image interface",
author = "Kenneth Kong",
year = "2006",
doi = "10.1075/idj.14.3.04kon",
language = "English",
volume = "14",
pages = "207--230",
journal = "Information Design Journal",
issn = "0142-5471",
publisher = "John Benjamins Publishing Company",
number = "3",

}

A taxonomy of the discourse relations between words and visual. / Kong, Kenneth.

In: Information Design Journal, Vol. 14, No. 3, 2006, p. 207-230.

Research output: Journal PublicationsComment / Debate / Erratum

TY - JOUR

T1 - A taxonomy of the discourse relations between words and visual

AU - Kong, Kenneth

PY - 2006

Y1 - 2006

N2 - A document is mainly composed of words and images, but the complex relationship that binds these two completely different semiotic resources is usually taken for granted as transparent. The simple relations between words and images-'anchorage' and 'relay', identified by Barthes almost 30 years ago-are unable to deal with the complexity of their bond, made even more complex by current printing and computer technology. This paper aims to identify the potential relations that bind texts and images together by arguing for a multilevel description of their logico-semantic relationships. The multiple, evaluative and metaphorical functions of the relations will also be discussed. The data generated from the proposed framework can form an empirical corpus for quantitative analysis. Examples from a variety of sources will be used as examples to show how the framework can be operationalized.

AB - A document is mainly composed of words and images, but the complex relationship that binds these two completely different semiotic resources is usually taken for granted as transparent. The simple relations between words and images-'anchorage' and 'relay', identified by Barthes almost 30 years ago-are unable to deal with the complexity of their bond, made even more complex by current printing and computer technology. This paper aims to identify the potential relations that bind texts and images together by arguing for a multilevel description of their logico-semantic relationships. The multiple, evaluative and metaphorical functions of the relations will also be discussed. The data generated from the proposed framework can form an empirical corpus for quantitative analysis. Examples from a variety of sources will be used as examples to show how the framework can be operationalized.

KW - Multimodality

KW - Rhetorical structure

KW - Systemic-functional linguistics

KW - Visual analysis

KW - Word-image interface

UR - http://www.scopus.com/inward/record.url?scp=43249149796&partnerID=8YFLogxK

U2 - 10.1075/idj.14.3.04kon

DO - 10.1075/idj.14.3.04kon

M3 - Comment / Debate / Erratum

AN - SCOPUS:43249149796

VL - 14

SP - 207

EP - 230

JO - Information Design Journal

JF - Information Design Journal

SN - 0142-5471

IS - 3

ER -