Agenda for today 09:30 - 10:00 Overview and Goals of LarKC, Frank van Harmelen 10:00 - 10:30 Introduction to the LarKC Architecture, Spyros Kotoulas 10:30 - 11:00 Coffee break 11:00 - 11:30 Hands-on 1: Working with an existing LarKC workflow, Alexey Cheptsov 11:30 - 12:00 Introduction to the LarKC Data Layer, Vassil/Naso 12:00 - 13:00 Hands-on 2: Building a LarKC decider plug-in to create a workflow from existing plug-ins, Luka Bradesko 13:00 - 14:30 Lunch 14:30 - 15:00 Introduction to Distributed Processing in LarK, Alex Cheptsov 15:00 - 16:00 Hands-on 3: Building a LarKC plug-in and integrating it into an existing workflow, Luka Bradesko 16:00 - 16:30 Coffee break 16:30 - 17:00 Hands-on 4: Understanding and Manipulating the Urban Computing workflow, Emanuele Della Valle 17:00 – 17:15 Demonstrate thread-pooling, Alex Cheptsov 17:15 Closing and Open Discussion, Frank van Harmelen

Welcome to the 3rd LarKC Early Adopters Workshop Frank van Harmelen Vrije Universiteit Amsterdam

Health Warning Today is a WORK shop we first tell you some stuff, then you do stuff (repeat) Goal of today: ours: show LarKC to outsiders <who are we>, yours: <tell us now>

Goals of today At the end of today you will understand the goals of LarKC understand the architecture of LarKC have hands on experience with platform and plugins At the end of the day, you will be able to: roll your own LarKC plugin roll your own LarKC application

LarKC = a platform for large scale reasoning Goals of LarKC LarKC = a platform for large scale reasoning Quote from EU Project Officer: “LarKC's value is as an experimental platform. LarKC is as an environment where people can go to replicate (or extend) their results in an environment where all the infrastructural heavy lifting has already been taken care of” 5 5

LarKC = a platform for large scale reasoning Goals of LarKC LarKC = a platform for large scale reasoning Quote from US high-tech CTO: Semantic web research is stifled by the complexity of writing a large scale engine, with services for data access, storage, aggregation, inference, transport, transformation, etc, Physics research has dealt with a similar problem by providing large scale infrastructure into which experiments can be plugged. The idea behind LarKC, which I found so compelling, is that people who wanted to build small scale plugins, for example, plugins for some non-standard deduction, or transformation of text to triples, or estimating the weights for relational models, could do so, taking advantage of the EU's investment in a platform with significant capabilities.“ 6 6

LarKC = a platform for large scale reasoning Goals of LarKC LarKC = a platform for large scale reasoning Quote from EU Reviewer: “Significant progress is sometimes made not by making something possible that was impossible before, but by substantially lowering the costs of something that was only possible before at high cost” 7 7

LarKC = a platform for large scale reasoning What do we mean by: LarKC = a platform for large scale reasoning reusable components reconfigurable workflows provide infrastructure needed by all users: storage & retrieval registration of plugins communication (plugin2datalayer, plugin2plugins) synchronisation (anytime behaviour) remote execution (abstracts from local/remote storage) remote data-access (abstracts from local/remote invation) (will) provide instrumentation & measuring (will) provide caching & data-locality integration of very heterogeneous components heterogeneous data: unstructured text, (semi)structured data heterogeneous code: Java, scripts, remote services ("wrap & integrate") 8 8

LarKC = a platform for large scale reasoning What do we mean by: LarKC = a platform for large scale reasoning not only from raw large numbers from performant data-layer from parallel computing in plugins from parallel deployment of plugins … but also from interaction of multiple components e.g. avoid reasoning through selection: SELECT + REASON allowing for incompletenes and anytime behaviour 9 9

What do we mean by: LarKC = a platform for large scale reasoning not only: deductive inference over given axioms but also: where do the axioms come from? (IDENTIFY) which part of knowledge & data is required (SELECTion) when is an answer "good enough" or "best possible" (DECIDEr) non-deductive inference (inductive, statistical) (REASONer) “ReaSearch: integrating reasoning and search" 10 10

Overall approach of LarKC Very lightweight platform communication, synchronisation, registration LarKC = “SPARQL endpoint on steroids” The real work happens in the plugins LarKC gives you: very scalable datalayer standardised interfaces for combining components utilities & infrastructure to abstract from remote deployment Three types of LarKC users: people building plugins people configuring workflows people using workflows

How to deploy LarKC All local: Calling remote plugins: Fully remote platform local, plugins local Example: workstation Calling remote plugins: platform local, (some) plugins remote Example: laptop Fully remote platform remote (eg. as a web-service) plugins remote Example: cluster

Why would people (like you) want to use LarKC workflow builders: easier to get some application scenario running Plugin builders: easier integration with components by others, wider take up of your own component by others

What does a workflow look like? Decider Query Transformer Identifier Info Set Transformer Selector Reasoner Data Layer 14

What does a workflow look like? Decider Query Transformer Identifier Info Set Transformer Selector Reasoner Data Layer Data Layer Data Layer Data Layer Data Layer 15

What does a workflow look like? Decider Query Transformer Identifier Info Set Transformer Selector Reasoner 16

What does a workflow look like? Decider Identifier Selector Reasoner 17

What does a workflow look like? Decider Selector Reasoner 18

What does a workflow look like? Decider Selector Reasoner 19

What does a workflow look like? Decider ETCETERA Identifier Info Set Transformer Query Transformer Identifier Selector Reasoner Identifier Info Set Transformer 20

What does a DECIDEr look like? Can be a hardcoded sequence of plugins Can be a self-configuring selection of plugins Can make run-time decisions on progress and resource consumption Coded as: Java a Cyc knowledge base ... as long as it complies with the DECIDEr API

Already any plugins available? existing web-services (e.g. Sindice, Swoogle) another RDF store (geo-queries in Allegrograph) a very large (workflow-based) system (GATE) existing reasoners (Jena, Pellet, Cyc, IRIS) XSLT scripts (XML-2-RDF) spreading activitation (new) RDF-2-weightedRDF (new) 5x IDENTIFY 3x TRANSFORM 10x SELECT 4x REASON 4x DECIDE Sometimes sophisticated, sometimes simple Sometimes novel, sometimes wrapped

Goals of LarKC, and where we are Scalable: > 109 triples, lazy pipes Reconfigurable: plugins with standard API’s Open: Apache license heterogenous: TRANSFORM, wrappers experimentation: wrap & integrate allow incompleteness: IDENTIFY, SELECT enable distribution: plugin containers anytime behaviour: streaming APIs web-enabled: remote plugins & data

What we will not show today Available but not demo’d: lot’s of plugins C-SPARQL: extension of SPARQL to enable stream-querying cognition-based heuristics (e.g. selection rules, stopping rules) very cool data-sets Linked Life Data 2.7B explicit, 4.1B closure, 580M things,2.1M interlinks Milan traffic grid Interest-enhanced DBLP (615k authors + interests) LDSR (1.3B explit + 2.2B closure, 400m things) very large/fast inference engines: MarVIN (P2P) WebPIE (MapReduce) Not yet available (but will be): instrumentation & measuring smart data caching

Agenda for today 09:30 - 10:00 Overview and Goals of LarKC, Frank van Harmelen 10:00 - 10:30 Introduction to the LarKC Architecture, Spyros Kotoulas 10:30 - 11:00 Coffee break 11:00 - 11:30 Hands-on 1: Working with an existing LarKC workflow, Alexey Cheptsov 11:30 - 12:00 Introduction to the LarKC Data Layer, Vassil/Naso 12:00 - 13:00 Hands-on 2: Building a LarKC decider plug-in to create a workflow from existing plug-ins, Luka Bradesko 13:00 - 14:30 Lunch 14:30 - 15:00 Introduction to Distributed Processing in LarK, Alex Cheptsov 15:00 - 16:00 Hands-on 3: Building a LarKC plug-in and integrating it into an existing workflow, Luka Bradesko 16:00 - 16:30 Coffee break 16:30 - 17:00 Hands-on 4: Understanding and Manipulating the Urban Computing workflow, Emanuele Della Valle 17:00 – 17:15 Demonstrate thread-pooling, Alex Cheptsov 17:15 Closing and Open Discussion, Frank van Harmelen