Need for Semantics  Now models represent just the object appearance  We need to represent also its  Properties  Roles  Behaviour  Services  …  i.e. its meaning, in a human and machine understandable way  The process has to be as much automatic as possible

Need for Semantics  To exploit fully the potential of of semantic enrichment as many properties of the real world should be represented in their virtual counterpart  We need metaphors for describing objects that are similar to what humans employ  E.g. a table= a rectangular top, 4 cylindrical legs

Shape Annotation  Documenting the 3D shape with contextual knowledge  Knowledge related to the geometry  Knowledge related to the application domain  Knowledge related to the content statue, base Restoration Fracture lines Erosion

Shape Analysis Characterization: Evaluation of scalar functions over the surface Structuring: Extraction of subparts and their spatial arrangement Segmentation: Identification of regions having homogeneous properties (main components or features of interest) …

Ontologies for Shape : Knowledge Modelling  modelling shapes and their associated semantics using knowledge formalisation mechanisms  metadata and ontologies provide the rules for linking semantics to shape or shape parts.

DSW by  Many ontologies, different facets  Annotate whole objects

Geometric Search Engine Based on Geometry and Structure Semantic Search Engine Based on Metadata advanced search engines for digital shapes

Possible scenarios for virtual worlds  semantic search for reuse:  Search among models having large geometry variations (e.g. human characters and objects)  create new VHs, reuse garments, take only interesting parts  Assistive living  Simulation of domestic environments with realistic people (e.g. with disabilities), places and danger conditions

Automatic identification of human body parts Plumber segmentation Tagging human body parts Tailor characterisation Candidate skeletal joints Skeleton extraction IT’S LATE!

Virtual Human Ontology > ObjectAttribute > ObjectAttribute > > Location > > HandPosture > Resource > > SmartObject > > Garment > > VirtualHuman > > > VirtualHumanWithLandmark > > > VirtualHumanWithSkeleton > > > > CompleteVirtualHuman > > VirtualHumanController > > > Skinning > MotionCategory > > Locomotion > > HumanInteraction > > PhysicalActivityAndSport > > InteractionWithEnvironment > > ComunicationAndGesture > PersonalityDimension > Expression > > FacialExpression > > BodyExpression > Node > > Joint > > BodyDefinitionParameter > > Landmark > > > FacialDefinitionParameter > > SegmentLocationHandPostureResourceSmartObjectGarmentVirtualHumanVirtualHumanWithLandmarkVirtualHumanWithSkeletonCompleteVirtualHumanVirtualHumanControllerSkinningMotionCategoryLocomotionHumanInteractionPhysicalActivityAndSportInteractionWithEnvironmentComunicationAndGesturePersonalityDimensionExpressionFacialExpressionBodyExpressionNodeJointBodyDefinitionParameterLandmarkFacialDefinitionParameterSegment > ModelPart > ModelPart > > ObjectPart > > HumanPart > StructureDescription > > SmartObjectStructure > > VirtualHumanStructure > Measurement > FacialAnimationParameter > BodyPartSize > BodyPartMovement > ClothPattern > MorphologyDescription > Geometry > AnimationSequence > > MotionCaptureSequence > > KeyFrameSequence > ArchetypalExpressionProfile > _3DVector > IndividualDescription > > Personality > > EmotionalState > BodyAnimationParameterDefinition > FacialAnimationParameterRange > BodyAnimationParameter > FacialAnimationParameterDefinition ObjectPartHumanPartStructureDescriptionSmartObjectStructureVirtualHumanStructureMeasurementFacialAnimationParameterBodyPartSizeBodyPartMovementClothPatternMorphologyDescriptionGeometryAnimationSequenceMotionCaptureSequenceKeyFrameSequenceArchetypalExpressionProfile_3DVectorIndividualDescriptionPersonalityEmotionalStateBodyAnimationParameterDefinitionFacialAnimationParameterRangeBodyAnimationParameterFacialAnimationParameterDefinition

ShapeAnnotator: general framework Knowledge Base Shape A surface mesh Domain An OWL ontology CG Tools Segmentation Plug-ins Shape Annotator Expert Abstracted Shape A segmented mesh Instance

Shape annotator  Framework for part-based annotation of 3D objects with context-dependent knowledge IT’S LATE!

Semantic-driven segmentations  They capture semantically relevant features automatically  what is relevant in the gaming context?  IMATI segmentation methods for triangle meshes:  Fitting primitives  Reeb Graphs  Tailor  Plumber  They extract specific kind of knowledge suitable also for virtual worlds

Fitting Primitives  Hierarchical face clustering algorithm able to recognise planes, cylinders and spheres

Reeb Graph  The topology of a shape is coded into the Reeb graph respect to f, which captures the evolution of the level sets of f on the shape

Skeletons and grasping regions Plumber (and Tailor) It defines a shape decomposition into connected components that are either tubular features or blob regions Landmarks

Conclusions  3D with semantics would have high impact  Easy indexing, searching and retrieval  Efficient reuse and re-adaptation  Realistic interactions among virtual characters and objects  Analysis, segmentation, annotation, knowledge base  Still a lot of work to do  Manual annotation  No shared conceptualisation  …