A FORMALIZED VISUAL LANGUAGE

Eleonora Bilotta, MarianoFiorito, Pietro Pantano, Laboratorio Interfacolta' della Comunicazione, Cubo20, Universita' della Calabria, Arcavacata di Rende (CS), 87036.

1) some concepts drawn from mathematical logics;

2) a set of icons on which we have worked applying a grammar and a semantics.

A pratical application of the visual language has been made for a public environment, a Museum. Finally, the authors have mentioned some possible applications of the visual language.

Keywords: Icon, Visual language, Visual Interface.

1. Introduction

The actual technological trends in informatics stresses upon the visual approach through visual programming, visual interfaces, visual language that generally mean the systematic use of visual expression to convey meaning.

In fact many empirical research show that memory and learning are faster, consistent and permanent if it is applied to visual images, instead of written words or sounds; because images are shaped of many dimensions (form, colour, size), of concreteness (that is almost touchable) and of perceptual readiness, they produce some redundance effects that improve the man-computer interaction. We don't know when man begins to think for images but we know that the actual Homo uses essentially mental images to formulate his concepts (Kossilyn, 1987; Arnheim, 1969, 1977). Thought seems to run on two complementary railways: the symbolic verbal language and immagination. The last works like a mental translation of concrete visual perceptions. In this way iconology could be a 'natural and spontaneous' discipline for cognitive processes; on the one hand it has to be based on the logic thought tools (facies mathematica), on the other hand on the analogic and metaphoric peculiarities of language (facies filologica), and on a third part, the expressive and communicative techniques of iconography, of picture and design, that convey to the sensible level of perception the conceptual contents of mind (facies dialogica).

Thus iconology, before translating in iconography, must be founded on a screening on the virtual area that human mind builds like added extensions to physiological organs, to improve the Ego dominion on the environment. We mean of planning extensions, constitued of the same materials of dreams, but organized through logical functions that, guiding the phantasies, lead to the ideation, to the discovery, to the invention.

Some reaserchers mean for visual languages the manipulation of visual objects, then images or pictorial objects; others mean language that are visual because their formal expressions are visual. Chang (1989) defines the first " languages for programming visual information", the other "languages of visual programming". We mean for visual language a systematization of iconographic rapresentation of informatic concepts in a laguage that is like the formalized language of empirical sciences. In fact, this paper deals about the building of a visual language for whom we'll use some particular interpretative tools:

1) the analysis of perceptual/cognitive processes during the mental/psychological elaboration in human vision;

2) a logical/mathematical grid that represents the phisical part of the visual language proposed;

3) some elements of classic semiotics that constitute the semantic part of the visual language.

Normally, when we see, we organize a complex structure of signs that help us to establish communicative contacts with the environments which we are in (Ingle, 1967; Schneider, 1967; Trevarthen, 1968; Held, 1968). Our brain works like an interface between the signs system and the environment orienting us in the space (ambient vision) and enabling us for the detailed examination and the identification of objects (focal vision); we'll based the visual language on this two way of seeing. The logical/mathematical grid is the formalizing elements, the syntax of the visual language and it works like the grammatical rules in natural languages for sentence building. In the visual language presented in this paper it utilizes iconographic elements for the construction of simple icons before and of complex icons or groups of icons after to represent structured visual sentences. The semiotic elements constitute the semantic part we'll utilize for building the visual language or the meaning it is possible to convey by the visual iconic sentences.

The result will be an iconic language that convey meanig in the same way of a natural language that could be utilized in visual programming, visual interfaces reducing the learning time in man-computer interaction and orienting the user in the informatic environments.

2. The formalized language

In order to build a visual language we will use some philosophic and scientific concepts drawn from mathematical logics (Guenot, 1988). Our aim is to create a specialistic communication tool which is specific of the empirical science (eg. a type of code).

In fact a formal language that will be denoted by L, is usually composed of four fundamental elements:

1) a signs system;

2) a grammar;

3) a logics;

4) a semantics.

For describing the L language we use an auxiliary language M called meta-language. In M we define the rules which code L. M is generally the common language.

1) The signs system: in the formal communication, the signs system represents no-ambiguous and easy to identify references. This is very useful for resolving a lot of emphirical science problems.

This set of signs has got all the typical natural features of languages and so it is composed of a limited number of distinct signs. Signs have got a rappresentative or referential function and they are completely arbitrary.

The signs set of a communication system (and also of natural or symbolic languages typical of the emphiric or artificial science) is always characterized by:

a) a limited set of distinct signs. For natural languages this set is composed of sounds and their combinations and coordinations which are in rappresentative samples of messages in a specified language. For the visual language the set is composed of icons and their combinations which are organized in structured messages;

b) reppresentative or referential signs function. The signs system is the expressive function of a specified language. Signs have got a reppresentative or referential function because they show constant corrispondences or relationships with external situations that exist in the objective and phisical reality of the speaker. This set of relationships is the contents, meanings or semantic system of a language. Other signs such as phonemes haven't got a referential function but they can be considered, togheter with other phonemes, signs with a referential meaning. The language is structured like a hierarchical system. As far as the visual language, which has got a reppresentative or referential function, the relationship with the external objective reality or with phisical one of the speaker is immediate and direct because signs and icons visually represent these realities. The visual language is also hierarchically organized by the use of first-level units which are structured in second-level units in accordance with the kind of message and the complexity of the communication system that we have choosen. The referential function, which is always present because of using icons that have got a reppresentative nucleus of the reality, is not present when we use artificial icons builded by particular communication aims such as inclusion, exclusion, etc.;

c) arbitrariness of the signs system. The signs system can be arbitrarily choosen. This is true for natural language and for simbolic logic-mathematical systems as well, except for icons. That is because icons look like reality (Peirce, 1931-1935; Morris, 1946, 1971; Eco, 1975) and the corpus of icons must be choosen in accordance with their natural similarity with the environment objects in which the visual language is applied.

2) The grammar works on the formalized structures of the language and it lets to group signs for obtaining correct sentences. The grammar is composed by morfology and syntax.

3) The Logics will be treated like iconic logics and utilized for the building of the visual language.

4) The Semantics represents the more ambiguous sectors of the formalized language.

3. Iconic Logics

For defining the physical and structural parts of the visual language proposed in this paper we will use particular working tools such as some concepts drawn from mathematical logics.

Through these concepts, after having defined a corpus of icons which represents our visual signs system, choosen in an arbitrarly way, we will define the iconic logics. The iconic logics is the formal part of the visual language that is to say how to use the icons through syntatic rules for building visual sentences.

Like natural languages, the iconic logics uses a grammar that lets us first to join the icons for writing visual words and then to join visual words for making correct phrases.

In fact our iconic syntax is composed of an "iconic dictionary" (like a "graphical vocabulary", useful for uniforming representations for all aspects of visul programming such as code, data, types and structures) (Glinert et al., 1990), and a set of well defined rules which let us to link the icons of the dictionary.

Besides the dictionary is composed of "terms" and "relations". A term can be defined as the following couple: (icon, icon properties set) in which the properties must be compatible with the icon. If the icon is a changeable one in the same meaning of the mathematical term, we have a particular icon called "variable".

An iconic relation is a link between two different terms which have only two different sets of properties visualized by the following structure:

The iconic link is a first-class operator and it must be compatible with the two sets of properties. It is important to note that the number of linked sets of properties could be bigger than two.

We have just defined the model from a morphological point of view. Now we are going to define the iconic grammar.

An iconic "assertion" is a phrase. It is possible to fill the structure visualized before, just putting the compatible icons in the structure relation:

We can distinguish the iconic assertion in "close assertion" (if there are no variables) and "open assertion" (if there are some variables). From this simple assertion it is possible to have more "complex" assertion by using the "connective" and "quantificator" operators.

These operators are called "first-class", "second" and "third-class" operators respectively. The use of these operators is regulated by four grammatical rules.

The aim of the iconic "semantics" is to select meaningful iconic phrases from all the correct iconic phrases (Chomsky, 1955,197O ) and so to allow the dialogue, like an interactive conversation between two or more partners, or simply to convey meaning in particular domain of information. In this case the most important concept is the "boolean" property (true="1" or false="0"). This is a label which is suitable to every assertion; but in the first approach, we can not define the rules of the labels because these rules depend on the kind of operators that we are going to work with.

A different kind of assertion is called "tautology" in which the boolean label is always true.

Till now we have been working on icons, icons properties and links among icons. We have defined terms, relations and then assertions.

Now we are going to define the last step of the iconic logic which is represented by a particular class of assertion "axioms" or "hypothesis" for building a pratical iconic demonstration.

A demonstration is a succession of iconic assertions organized according to well defined rules, in which it is possible to go from one to the next one for analyzing and reading the contents of the phrases.

A "theorem" or "lemma" or "corollay" or "preposition" is an iconic assertion that can be found in a demostration. According to this definition a theorem is a "true" assertion and a "false" one is an assertion whose negation is true.

In the physical realization of a communication system, with particular contents to convey, it is not simple to discover if a complex iconic assertion is true or false. The only way is to use the demonstration rules operating a visual decoding of complex and formalized iconic assertions step by step. It means to know the label of a single assertion in every single step until the last one.

4. Formal aspects

After having defined the iconic logics, we are going to discuss how the icons, that constitues the signs system, can be manipuleted pratically.

At the beginning we will define a set of signs and then we will generalize the model to a set of icons that represents our universe of signs.

To build a "signs system" we use some important and basic principles from the "Mathematical Sets Theory" such as: elements, sets, belongings and functions establishing criterions which make up new sets starting from initial sets.

In this system there are three different kind of signs:

a) element: every sign is an element;

b) set: a set is a collection of elements. There is a hierachical relationship between elements and sets. In fact, an element can be a member of a set. This kind of relatioship is explained by the link of belongings;

c) operator: an operator is a tool which has to produce new elements. It has got a specified number of inputs (inlets) and outputs (outlets). These numbers depend on the kind of operator.

In this signs system we can also identify two different groups of sets; the first one is composed of five sets which are called "basic sets". The second one is composed of three sets and it is called "secondary sets" or "derived sets".

The sets of the first group are:

1) "V"= set of basic signs.

It is composed of a limited number of signs (eg. "alfa"). Every sign of V is an element. V is a set.

2) "W"= set of properties of basic signs.

Every member of W is a property which describes a sign. The most important property in this set is the "boolean label" (true or false). Let "beta" be the number of W members.

There is a link between the V and W sets. This link is really an operator which is called "matching" operator ("psi"). It is a "zero-class" operator which link one V element with one or more W elements.

3) "F"= set of first-class operators.

Every operator has got a particular function which is more difficultous than the "psi" function. Let's assume the number of F members equals to N. Elements of F could be: "equivalence", "presence", "inclusion", "link", "association", "indication" or "development operators".

4) "Y"= set of second-class operators.

The number of Y members is limited (eg. "M"). Examples of operators are: "negation", "disjunction", "junction", "implication" and "double implication".

5) "Q"= set of third-class operator.

In this set we have the highest degree of difficulty. The number of Q members is limited (eg. "H"). Q operators are the "universal" and "existential" quantificators.

In order to understand how the operators act, we have to think of a single operator like a machine which is able to receive signs and to give off different ones:

For using a new operator is necessary to define clearly the number and the kind of signs (inlets and outlets). The number of inlets depends on the class where the operator is placed.

The sets of the second group are:

1) "T"= set of elements created by using the "psi" operator.

The number of T members is limited (eg. "L"). Every elements of T is a first-class n-pla where n can be equal or bigger than one:

2) "A"= set of elements created by using a first-class operator.

The number of A members is still limited. How the new element is created is shown here:

3) "AC"= set of elements created by using a second or third-class operator.

This is the only case in which the number of set members is unlimited because of using third-class operators (quantificators). The reason is that quantificator operators actually work with variables. A variable is still a sign and so we can apply an operator on it. Besides a variable can be substituted by a basic sign in every moment.

This particular sign system allows to work with different signs and combination of signs and with different levels of complexity. There are at least two "indexes" which directly measure the difficulty degree. The first index calculates the complexity from a formal point of view; it means to measure the operating level.

The following is a list of elements going from a simple to a complex degree:

- member of V

- member of T

- member of A

- member of AC

Complexity can be shown by particular symbols using two indexes, eg. AC(i,j) in which i is the index of formal complexity and j the index of cognition complexity.

The "j" value is based on interpretation and it represents the semantics aspects of the signs system (De Sausurre, 1916).

We have already discussed the boolean label defining the W set and the "psi" function or operator. If every sign of V has got a boolean value then it is possible to calculate the boolean value of a new element created by using an operator. This is indipendent from the class in which the same operator is.

The following drawing shows how an operator works:

As a consequence of this way of working, there is the possibility to create special elements of AC set, which have always boolean values equal to "1".

5. An example

The following is a pratical application of the theory of the visual language proposed. In this application we need to create a "signalling system" for a public environment.

The choosen environment is a Museum. In this case the signalling system must create a visual map of the Museum for the visitors, helping them in orienting in the space and improving their competence in learning the Museum contents.

Accoding to the theory, previously developed, a signs system is composed of two groups of sets. First of all we can try to match them in the example related to the Museum.

The sets of the first group are:

- "V"= set of basic icons.

In V there are a limited number of icons. There is a precious corrispondence between an icon and one single element of the Museum. It means we can find icons for physical environments, synthetic environments, showed objects, databases, services, maps, sites, tools, signalling, etc.

In the figure 1, there are some possible V icons.

Fig. 1 A set of icons

- "W"= set of properties of basics icons.

Every element of W identifies a property (type) of the basic icons. It permits to classify the V icons. First, we distinguish physical icons eg.:

1) enviroment (sections, rooms,...);

2) showed objects (pictures, sculptures,...);

3) service tool (workstations, elevators,...).

Then logical icons eg.:

1) service (shopping, exchange office,...);

2) reference (geographycal area, historical period,...);

3) text;

4) shape;

5) color;

6) background;

7) boolean ("1" true and "0" false).

We consider that every V icon is true.

To denote every property in W we need to choose a symbol for it. This symbol is actually an icon. For this reason a precious correspondance (convention) exists.

The icons choosen for visualizing the conventions have to satisfy some properties like simmetry, regularity and order, keeping in mind that we are going to put the signalling system in a Museum.

An important topic is how to use the color. In our case the color means a channel in which a new information can be sent to the receiver. Three factors are very important in the choice of the colours channel:

a) to avoid hotchpotch of colors;

b) contrasts between icons and backgrounds;

c) psychological sensations.

- "F"= set of first-class iconic operators.

Every element of F is a special function and allows to create what we have called link or relationship between icons. F operators are:

1) f1= equivalence, x icon "is equal to" y icon;

2) f2= presence, x icon "is in" y icon;

3) f3= inclusion, x icon "is inclused in" y icon;

4) f4= link, x icon "is linked to" y icon;

5) f5= association, x icon "is in association with" y icon;

6) f6= indication, x icon "indicates" y icon;

7) f7= development, x icon "is developed in or through" y icon (see Fig. 2).

Fig. 2 First class of iconic operators

These seven operators create new elements with the boolean values always equal to "1".

- "Y"= set of second-class iconic operators.

Fig. 3 Second class of iconic operators

There are five operators in Y:

1) y1= negation, it creates an icon that denies another one;

2) y2= disjunction, x icon "or" y icon;

3) y3= conjunction, x icon "and" y icon;

4) y4= implication, "if x icon then y icon";

5) y5= double implication, x icon "that is" y icon (see Fig. 3).

In this set the boolean value of the new element depends on a set of rules; every single operator has got a boolean rule. The rule for the negation operator is shown in the following tables:

For the other operators the rules are the same of the mathematical ones.

- "Q"= set of third-class iconic operators.

Every element of Q operates as a special function with the highest degree of complexity. The following are two Q functions examples:

a) q1= existential operator: x variable icon exists, "so that" the iconic assertion will be true;

b) q2= universal quantificator: for every x variable icon, the iconic assertion will be true.

In the case of Q operators there are boolean rules which control q1 and q2. They are a generalization of the rules for the disjunction (y2) and conjunction (y3) operators respectively but now they must operate with an unlimited number of iconic assertions.

As we have seen for the graphical reppresentation of the properties (types), we need now to define some symbols to represent F, Y and Q operators. We have choosen the convention illustrated in the figures 3 and 4.

We have tryed to use international symbols such as mathematical symbols for conjunction and disjunction operators to make simple the process of learning.

The second group is composed of the following sets:

- "T"= set of icons created by using the "psi" operator.

The "psi" operator generally gives a n-pla which is composed of basic icon and a set of its properties. We assume that n is equal to two: (basic icon, property).

T is a set of iconic terms and together with iconic relations they become the "dictionary". In the figure 4 there are some examples of T elements.

Fig. 4 Some examples of T elements

- "A"= set of icons created by using a first-class operator.

In this set we are using one of the first-class operators (f1,..., f7). This kind of operator works at least with two icon terms. In the figure 5 there are examples of A element.

Fig. 5 Examples of A elements

- "AC"= set of icons created by using a second or third-class operator.

AC is the only set with a n unlimited number of members because of variable icons. At this point it is necessary to choose a convention for the variable icons: for variable icons we'll use small letters; upper letters for assertions . In the figure 6 there is a set of examples of AC elements.

Fig. 6 Examples of AC elements

We have previously discuss the tautology. We have seen that the boolean value of an iconic assertion (builded up by operators) depends on primay values of the initial iconic assertions. So we can introduce a "boolean table" of an iconic assertion without using quantificator operators q1 and q2. Finally, we can say that a tautology is an assertion which has got the final column of its boolean table full of "1" values. In the figure 7 there is an example of tautology without q1 and q2 operators.

Fig. 7 An example of tautology

Ordinarly in the iconic assertion we can find the q1 and/or q2 operators. It is not possible to generalize the concept of boolean table for this case but anyway we can extend the concept of tautology. In fact we can conclude that a tautology is an iconic assertion if every iconic assertion, builded up by starting from this assertion, has got the boolean value "1".

After having visualized step by step the visual language and applied the

demonstration rules, we can build up an iconic dialogue. The figure 8 shows an example of iconic dialogue.

Fig. 8 An example of iconic dialogue

The example builded through the formalized visual language shows that it is possible to apply the model to every public environments, even if it is necessary to create a physical layout of the iconic signalling system. This could raise important problems:

1) the physical layout of the icons (signallings) in the environment which means "where do you need the visual information according to the ambient space or focal vision?"; 2) the complexity level of the single icon defines different levels of perceptive complexity and then of decoding the visual contents that it intends to convey and chooses where a signalling system must be placed; 3) the signalling system have to be read from the first to the last level of complexity for generating a good learning in the users.

6. Applications

The visual language proposed in this paper can be applied in different domains in which it is possible to exchange and obtain information and generate learning processes:

1) Data Base creation for the resolution of orientation problems and for organizing the contents in a hierarchical way, linking the identification problems to the visual language and improving the user ability in building personal path of learning;

2) Information system: actually every domains of informations is deeply articulated in many areas and there are a lot of communication systems that must convey informations (multimedia). These communication systems must dialogue with the users and among them. The visual language can be useful by creating a link that allows the dialogue;

3) Public environments signalling system: today we are living in a multi-cultural era; the difficulty to convey informations in a multi-racial society points the attention on the need to have a universal language that is possible to identify in the visual one. The visual perception processes are the same in every cultures and so the visual language can be a useful tool for the trans-cultural communication;

4) Visual interfaces: the informatic areas could be considered a large land in which the user is inserted in. There are different kinds of users and different ways to convey information. The visual interface, builded through the visual language model proposed in this paper, can be an example to join the man-computer communication to the linguistic biological interaction (Bertacchini et al., 1993);

5) Communication system: visual workstations are composed of different media that if from one hand allow to obtain information in a fast way, reducing the time of learning and improving the users ability in managing different media, from the other hand can be very difficult to use; a visual programmig can solve a lot of problems in this area.

6) Design: Norman (1990) says that the visual planning is necessary for a good design for helping man in using complex objects, expecially computers, domestic appliances, telephones, etc. Through a visual language it is possible to link the needs for a good design and the users demands of interaction.

References

Arnheim R. (1969). Visual Thinking, University of California Press, Berkeley, CA.

Arnheim R. (1977). La dinamica della forma architettonica, Feltrinelli, Milano.

Bertacchini P.A., E. Bilotta, P. Pantano (1993). Contributi per la definizione di un modello in ergonomia, Atti del 5 Congresso Nazionale "Ergonomia e Progetto", Palermo.

Chang S. K. (1989). Principles of Pictorial Information Systems Design, Prentice-Hall, Englewood Cliffs, NJ.

Chomsky N. (1970). La grammatica generativa, Boringhieri. Torino.

Chomsky N. (1955). "Semantics considerations in grammar". In Rep. of Sixth Annual Round Table Meeting on Linguistics and Language Teaching, a cura di R.H. Weinstein , Washington, pp. 144-150.

De Sausurre F. (1916). Course de linguistique Général, a cura di Bally C. e Sechehaye A., Parigi (tr. it. De Mauro, Bari, 1967).

Eco U. (1975). Trattato di semiotica generale, Bompiani, Milano.

Guenot J. (1988). Fondamenti dell'analisi: logica matematica, Mediterranean Press. Rende (CS).

Held J. R. (1968). "Dissociation of visual functions by deprivation and rearrangment", Psychologische Forschung, 31, 338-348.

Ingle D. (1967). "Two visual mechanisms underlying the behaviour of fish", Psychologische Forschung, 31, 44-51.

Kosslyn S. M. (1987). Le immagini della mente, Giunti. Firenze.

Morris C. (1946). Signs, Language and Behavior, New York, Prentice Hall.

Morris C. (1971). Writings on the General Theory of Signs, The Hague, Mouton.

Norman D. A. (1990). La caffettiera del masochista, Giunti. Firenze.

Peirce C. S. (1931-1935). Collected Papers, Cambridge, Harvard Un. Press.

Schneider G. E. (1967). "Contrasting visuomotor functions of tectum and cortex in the golden hamster", Psychologische Forschung, 31, 52-62.

Trevarthen C. B. (1968). "Two mechanisms of vision in Primates", Psychologische Forschung, 31, 229-337.