From Semiotics to Computational Semiotics State University of Campinas UNICAMP - Brazil Ricardo Gudwin

Semiotics and Computational Semiotics zSemiotics ybranch of human sciences, that studies the sciences of signification and representation, involving mainly the phenomena of cognition and communication on living systems. zIntelligent systems ysystems that exhibits behavior that can be considered intelligent ysome of the objectives are the study of the phenomena of cognition and communication, but now explicitly under the scope of artificial systems zComputational semiotics yproposition of a set of methodologies that in some way try to use the concepts and terminology of semiotics, but composing a framework suitable to be used in the construction of artificial systems, in this case, implementable within computers

Computational Semiotics zComputational semiotics ynew inborn science, ybut, there are currently some important contributions that despite still not complete and definitive, help us in understanding the nature of semiotic processes and allow their synthesis and implementation within computational platforms. zIn this work ywe explore one possible pathway along a set of ideas evolving around, which proposes one way of gathering the transition between traditional semiotics and computational semiotics, making it possible to synthesize semiotic systems by means of artificial computing devices.

Semiotic Analysis zSemiotics yTool of Analysis - main goal is to understand the semiotic processing happening in nature ySemiotic Beings (interpreters) are already there xliving organisms (biosemiotics) xhuman beings (human semiotics) yit is easier to create concepts and apply them to things that do already exist and are already working zQuestions yshould it be possible to use the same conceptual background in order to synthesize new beings (systems), performing the same semiotic behavior as living/human beings would do ? yWhat would be the challenges in such an endeavor ?

Semiotic Synthesis zProblem ythings are not already working yso, we should put them to work ! zHidden Problems yspecify the basic entities involved within semiosis xin a way in which it can be produced within a computer yspecify the mechanism by which signs are interpreted xthere are a lot of intermediary steps that are generally not considered within the context of human semiotics How from a scene given by a video camera we discover the objects involved into this scene ? Can we talk about signs if the system is still not aware of objects ? yAre computational devices able to carry on all sign-processing that living/human beings are able to perform ?

Semiotic Synthesis zBasic Foundations yset up a generic scenario in which semiotic synthesis is going to be discussed ytry to get clues on how semiotic processes really happens yallow the implementation of a computational version of semiotic processes zTerminology yrelated to standard semiotic terminology y but we don’t want to limit the meaning of terms to human/bio semiotics zRequirement ybe careful when applying semiotic analysis to our synthesis scenario

Semiotic Synthesis Basic Foundations zRepresentation Spaces

Semiotic Synthesis Basic Foundations zShareable and Non-shareable spaces

Semiotic Synthesis Basic Foundations zInterpreting Fields

Semiotic Synthesis Basic Foundations zMultiple Internal Spaces and Interpreting Fields

Semiotic Synthesis Basic Foundations zInterpreting Field yconcept originated from field theory yfunction (energy function ?) that to each point in space and time determine a unique value ystate zExternal Space yinterpreting field is continuous (that’s the real world) yby definition, is not knowable in its entirety zInternal Spaces yaccommodate a model of external interpreting field yinternal interpreting fields are functions that depend on the type of semiotic synthesis we are trying to model

Semiotic Synthesis Basic Foundations zSign yEverything under the interpreter’s focus of attention (internal or external) that would cause an interpreter action zInterpreter Possible Actions yChange in the focuses of attention (internal and/or external) yDetermination, for the time t = t+1 of a new value for any interpreting field (internal or external), at a point (x,y,z) covered by the focus of attention in that space zInterpretant yany interpreter action caused by a sign yany change in internal and external interpreting fields for time t = t+1, caused by an interpreter action due to the effect of the sign

External Semiosis zInterpretant of signs yhappens at the external space zChange in external interpreting field ychange in environment yshareable with other interpreters ycan act as a sign for the same interpreter or to other interpreters zHappens mainly on interpreters that do not have internal spaces ysemiosis in molecules and chemical reactions yvery simple biological organisms zCan be the final result of a chain of internal semiosis

Internal Semiosis zInterpretant of Signs yhappens within any of the internal spaces zSigns can be yat the external space (semiotic transduction) yat the internal space zA typical semiosis chain ystarts with an external sign ygenerates a set of internal interpretants, that ybecome internal signs ygenerating new internal interpretants, until ysome of them become an internal sign ythat generates an external interpretant

Information, Signs and Knowledge zSignals and Information ysignals - values of parts of interpreting fields that can be differentiated (distinguished) from other values yinformation - meaning of signals zExample ysuppose that  E (x,y,z,t) has a counter-domain like [0,5] ybut, due to sensor limitations, the interpreter is only able to sense values in {0,1,2,3,4,5} ythen, values like 2.3 or 2.2 would equally be understand like 2 yso, the information that those signals convey is tied to only 6 discrete values zSignals only describe states ythey do not cause any actions

Information, Signs and Knowledge zOnce signals are able to cause actions ythey become signs zThe information they carry yassociated to the actions they cause yis then called knowledge zSignals - Information zSigns - Knowledge zRegion under a focus of attention of some space ysign yknowledge unit

Things to Remember zExternal Interpreting Field yis infinite, continuous and probably take values on continuous sets ycan not be known as a whole ycan be known in parts, with approximations zThe only way we are able to know the external interpreting field is due to sensors zThe most basic knowledge units that can be stored into internal interpreting field is of sensorial type zInternal Interpreting Field (Concrete Space) your best model of external interpreting field

Things to Think About zStoring sensorial information is not efficient yWe need better models zBasic mechanism yThe notion of “Entity” zFrom sensorial knowledge units ythe system must try to represent the same external interpreting field as a collection of entities zEntities ymay have attributes that would change in time zOccurrences ymodel the change in entities attributes zSensorial knowledge, entities and occurrences ygrouped to represent situations

A Hierarchy for Knowledge Units

Simplification of the Model zInstead working in general spaces and interpreting fields yrestrict ourselves to memories and places yassign sign processing to micro-interpreters

Micro-Interpreter zMicro-Interpreter’s Responsibilities ychoose the knowledge units that will be used (focus of attention) yeventually destroy them after use ycreate new knowledge units using information contained in earlier ones

Building Intelligent Systems zUsing multiple micro-interpreters yin cooperation to each other yprocessing knowledge units

Conclusions zDyadic or Triadic ? ySome people would say that we are building a dyadic model for a sign yBut, there is some kind of “mediation”, due to: xthe focus of attention mechanism xthe influence that some knowledge units may have over the processing of other knowledge units (catalytic knowledge units) yWe still need to do further reflections in order to have a better position on this issue zThis is not the final word regarding Computational Semiotics yit is only a first exercise in order to get insights to the problem of semiotic synthesis ycomputational implementations of such model indicate that, up to some point, it is worth the value of working on it