1. CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/i t CF LEMON @ Messaging System Ivan, Omar, Sergio 14 march 2012

2. CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/i t CF Presentation Title - 2 Agenda General architecture Constrains: re-use lemon-agent and already developed sensors Lemon producer Constrains: re-use existing solutions to publish data do not touch agent/core_sensors expensive development backward compatibility with current lemon-server/repository if to develop for lemon, be generic as much as possible expect additional requirements (e.g. cross agent communication, tagging) Message content Constrains tagging metadata included/excluded versioning/source Configuration Constrains for all nodes (quattor, puppet, non-managed)

3. CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/i t CF Presentation Title - 3 General Architecture Lemon Agent Lemon Server Sensor 1 Sensor 2 Lemon Producer Directory Queue msak Messaging Queue Sensor 3 Local Cache Oracle tcp/ip

4. CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/i t CF Presentation Title - 4 Lemon Producer Re-use knowledge (thanks to Lionel): directory queues together with the “msak” tool Lemon Agent Lemon Producer Directory Queue msak tcp/ip More IPCs considered. Why tcp/ip? already implemented (config parameter) cross agent communication Lemon producer functionality receive/aggregate data from lemon-agent(s) logic depends on requirements serializes message into JSON multiple formats/versions add tags provide functionality for additional plugin not to have multiple wrappers add new agent and/or replace lemon-agent

5. CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/i t CF Presentation Title - 5 Message A1 0 lemon-slc5#10002 1331545236 file N/A 1.0.1# lemon-slc5#10003 1331545236 1 2 2# lemon-slc5#10005 1331545236 660# {"body": "{"source": "lemon", "samples": [{"node": "lemon-slc5", "timeStamp": "1331545236", "metricId": "10002", "values": ["file", "N/A", "1.0.1"]}, {"node": "lemon-slc5", "timeStamp": "1331545236", "metricId": "10003", "values": ["1", "2", "2"]}, {"node": "lemon-slc5", "timeStamp": "1331545236", "metricId": "10005", "values": ["660"]} ], "messageType": "0", "protocolVersion": "A1" }", "header": {"destination": "/topic/lemon.test.agent"}, "encoding": {} } Original Message JSON Message Possible tags source schema version whatever registered for schema Meta data include  overhead exclude  metadata dictionary

6. CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/i t CF Presentation Title - 6 Configuration Lemon-agent quattorized/puppetized Lemon producer ? which platform do we want to support (quattor, puppet, no mgmgt)? what will be configured  tags as well? local handling of tags by command-line (like lemon-host-check enable/disable)? runtime change of producer config (e.g. tags) by push mechanism server  client? Large scale test discussion production Apollo broker topics/queue consumer when