Debbie Rogind Nov 11, 2004 SLC Aware IOC Database Service Design

Topics DBS Use Cases Design Considerations DBS Initialization Data Structures Download Sequence ST0 Conversion Receive/Process Setpoints and other messages from Alpha Process/Send SLC IOC udpates to Alpha DB Input/Output Utilities

DB Service Use Cases DBEX - “Look like” a tcp micro Same message formats, traffic, use proxy Req and Accept database (“IPL”) Accept updated setpoints from Alpha Provide updated data from IOC Supertype 3 (ST3) readbacks Supertype 2 (ST2)setpoints (new) Accept and reply to diagnostic inquiries Accept and Process DBEX up/down messages Console Users Utilities to edit, dump, get type of various slc data Application code Utilities to find, get, put, update slc data Coded “data flow dependancies” ??? What else

Design Considerations Port as much existing micro code as possible Messages from VMS control system are little-endian, VMS format, and packed The VMS supertype header structure is packed – no “pad” exists between 4 byte boundaries Support ASCII representations for primary, secondary names and units Slc IOC needs supplementary data definition: primary ASCII to # secondary (secn) ASCII to # mappings for given primary datum width (for I or Z)

Topics DBS Use Cases Design Considerations DBS Initialization Data Structures Download Sequence ST0 Conversion Receive/Process Setpoints and other messages from Alpha Process/Send SLC IOC udpates to Alpha DB Input/Output Utilities

DB Thread Initialization slcExec starts Phase 1 message Q’ed threads dbSend, dbHdlr slcExec starts Phase 2 dbRecv after all Phase 1 actives are T All db threads Create cmlog handle Init resources Message queue Sockets Memory heap Timers, semaphores Set active = T Wait (at message Q or socket) dbHdlr thread initializes shared db thread globals slcExec sends “ DB_DOWNLOAD ” to dbSend

Topics DBS Use Cases Design Considerations DBS Initialization Data Structures Download Sequence ST0 Conversion Receive/Process Setpoints and other messages from Alpha Process/Send SLC IOC udpates to Alpha DB I/0 Utilities

DB Service Globals NameTypeRepresents dbnodes_p[5]dbsuptype_tuPointers to superblocks 0-3 dbRecv mallocs, writes pointers & data while other threads are blocked at db download event. After download event pointers read-only (ptr to ST0 & data deleted; rest never change) dict_pvoid *Pointer to dictionary; new char-based gphash table created for housing supertype 0 data (secondary data & location) and valid ASCII names. gphash has its own mutex protection for add/delete/find dictionary utilities. dbex_up dbVersion epicsBoolean char * Status of dbex up (T) or down (F) Db major / minor version w dbex_up msg dbRWMutexepicsMutexIdActiviated for reads or writes from/to supertype 1-4 data. dbRecv (for Alpha VMS updates), and dbl* I/O utilities access dbhilo_updates [N_JOBS] Dbhilo_job_ts *Each “job service” array contains pairs of offsets to upper and lower bounds of modified ST2/3 data. Acc’ed by utility rte dbhilo_update called from dblput. dbhiloMutex [N_JOBS} epicsMutexIdActiviated for reads or writes from/to dbhilo_updates[ ]. Acced by dbSend, and private utility dbhilo_update called by dblput dbAckEvent [N_JOBS} epicsEventIdJob service threads wait at this event until their updated data has been acknowledged by Alpha downloadEvent, dbExists flag is created by slcExec dbHdlrThread signals downloadEvent, sets dbExists flag after db download / ST 0 conversion Globals for db threads, initialized by dbHdlr:

Supertype 0 Byte Stream to/from Alpha fwd_hdr_ts proxy header dbsuptypehdr_ts supertype header hash length array listhead ptrs array node counts UPS block 0 secn block [0 – nsub]: supn | subn fmt | slen sptr = offset in supn … …. UPS block X secn block [0 – nsub] links – next UPS, unit, collided nodes len - ups + nsub’s catn - prim # unit - unit # nsub - # of secns

Supertype 1-3 Byte Stream fwd_hdr_ts proxy header dbsuptypehdr_ts supertype header data data ….

Supertype Header & Data NameData typeRepresents fwd_hdr_ts ip_port_u len user cmd crc ip_port_tu int4u user_field_ts int1u Lower half of the ip address, and the port number Message bytecount - this fwdheader User defined; chunk count for large buffers; Fwd_server command, e.g. 8 bit crc over header; currently set to 0x55 dbsuptypehdr_ts ordbsuptype_tu = union network netlsn id <<<<< Note len micr blkbeg blkend int2u potential byte int4u Int4u int4u Type of data alignment mistmatch here >>>>> Size of superblock Microname Copy to location in superblk Copy from location in superblk datw[ ]Array of int2utype-specific data; packed; max size is NETBUFLEN minus sizeof(dbsuptypehdr_ts)/2

Buffer/Q Structures NameData typeRepresents dbsndrcvbuf_tsReceive and send buffers proxy_hdr supblk nativeSB fwd_hdr_ts dbsuptype_tu dbsuptypehdr_ts Proxy header Superblock header + data Native superblock header structure added for slc ioc dbSend uses nativeSB Converts prior to sending htoVMSSupHdr() DB Thread Queue message structure NameData typeRepresents dbmsgmail_tsQ messages msgheader nativeSB dbdiag msgheader_ts dbsuptypehdr_ts dbdiagmsg_ts Msg header Native superblk hdr Diagnostic data Rcvbuf, sndbuf get cast to dbsndrcvbuf_ts dbRecv converts to nativeSB -VMStohSupHdr()

Secondary Data The secondary data is discussed in later slides (after ST0 Conversion and the “new” dictionary is discussed)

Topics DBS Use Cases Design Considerations DBS Initialization Data Structures Download Sequence ST0 Conversion Receive/Process Setpoints and other messages from Alpha Process/Send SLC IOC udpates to Alpha DB I/0 Utilities

DB Service Download Block Diagram

dbRecvThread – Download (“IPL”) Sequence Rcvbuf receives Supertype 0-3 messages from DBEX at dbex socket VMStohSupHdr(rcvbuf.nativeSB, rcvbuf.supblk) Sends “DB_DBEX_ACK” to dbSend Q to ack DBEX (if req’ed) Validates data and proper sequence of download (dbmicromail) Allocates super block memory (dbsuperalloc) from heap; populates dbnode_p[0] Copies rcvbuf (dbmsgcopy) to alloc’ed memory; drops proxyhdr Lock / unlock dbRWMutex around memcpy Send “DB_CONVERT” to dbHdlr Upon recv’ing last ST 3 block

dbSendThread – Download (“IPL”) Sequence Receives “DB_DOWNLOAD” message from slcExec Sends “ack” to DBEX socket Formats sndbuf.nativeSB htoVMSSupHdr(sndbuf.supblk, msg.nativeSB) Pre-fixes proxy_hdr Sends (dbdownloadme) Receives “DB_DBEX_ACK” messages from dbRecv for each ST 0-3 block received Sends “ack” to DBEX socket uses msg.nativeSB.id; htoVMSSupHdr(sndbuf.supblk, msg.nativeSB) Pre-fixes proxy_hdr Sends

Topics DBS Use Cases Design Considerations DBS Initialization Data Structures Download Sequence ST0 Conversion ST0 Conversion Receive/Process Setpoints and other messages from Alpha Process/Send SLC IOC udpates to Alpha DB I/0 Utilities

dbHdlrThread dbHdlr is responsible for Initializing the DB Globals upon init Converting the ST0 data to dictionary format Releasing DB resources upon exit Resetting global values upon exit dbExists downloadEvent Thread specific – see General Thread Shutdown Receives DB_CONVERT message to initiate the ST0 Conversion process After conversion Sets dbExists = T Signals downloadEvent Receives TEST_STOP message Refer to General Thread Shutdown

dbHdlr Block Diagram

Secondary Data type NameData typeRepresents GPHENTRYSLC Secondary data type node name pvtid userPvt ELLNODE Const char * Void * Link node “primary unit secondary” Pointer to slcSubblk_ts Not used NameData typeRepresents slcSubblk_tsSLC Secondary data type secn supn len format sptr width int1u int4u int1u Secondary number Supertype number Word length (V=variable) (I,A,R,Z,S,T) Word offset into supn block 1,2,4,8 bytes; data width Gphash find - gphFind (“name”) - returns GPHENTRY*:

dbHdlrThread - ST0 Conversion Sequence Receives “DB_CONVERT” message from dbRecv Replace ST0 num-oriented hash table with string based hash table dictionary Step 1: Walk thru all structures in ST0 to swap words and longs (if arch. is not little endian) Step 2: Read PRIMARY.MAP file and make entries into dictionary, each combining Secondary data from ST0 File data – data width, ASCII names Hash on “primary unit secondary” Hash on “primary” Links ALL* units  For primary name error checking Could also hash on “primary secondary”  For faster secondary name error checking Structure is optimized for run-time operations

dbHdlrThread - ST0 Conversion Sequence Correlates file with ST0 data PRIMARY.MAP: Prim catn Sec secn fmt width # QUAD 1 BMON 135 R 4 1 QUAD 1 KTIM 31 T 8 1 … For each unique primary name in file Make “primary” entry call dbunitsST0 (ld_p, catn) if (status OK and ld_p.len>0) for each unit# in list dbgetUpsST0(uptr, catn, unit) link unit in “primary” (for ALL*) for every secondary in file dbgetSecST0(sptr,uptr,secn) store “primary unit sec” with combined ST0 & file data; + link sec in unit entry (for ALL*)

dbHdlrThread - ST0 Conversion Sequence Delete ST0 Signal downloadEvent, sets dbExists flag Comments Translated dblistu.a38 to C Uses memory manager (freeList) to manage new slcSubblk_ts w combined data Optimized for: dblists in thread’s init: dblist(dblist_p, prim1, unit1, sec1) dblist(dblist_p, prim2, unit2, secn) etc. And “on the fly” access, such as operator

Topics DBS Use Cases Design Considerations DBS Initialization Data Structures Download Sequence ST0 Conversion Receive/Process Setpoints and other messages from Alpha Process/Send SLC IOC udpates to Alpha DB I/0 Utilities

Setpoints from Alpha

dbRecvThread – Alpha Updates to slc ioc Receives Supertype 2 (setpoint) update messages from DBEX Sends “DB_DBEX_ACK” msg to dbSend, if req’ed. Lock dbRWMutex Copies super block recv’ed (dbmsgcopy) to slc ioc ST2 block Unlock dbRWMutex Other DBEX messages Receives diagnostic req from DBEX Sends “DB_DIAG_RPY” to dbSend Q to ack DBEX Receives “DXDN”, “DXUP” messages from DBEX Resets/sets dbex_up flag Stores database version in global dbVersion If DBEX becoming available, send “DB_UPDATE_ALL” message to dbSend Q to update all jobs (TBD)

dbSendThread –Alpha Updates to slc ioc Receives DB_DBEX_ACK messages from dbRecv for Alpha ST2 updates Sends “ack” to DBEX socket (same as “IPL” sequence) Receives DB_DIAG_RPY message from dbRecv uses msg.nativeSB.id VMStoh(sndbuf.diag, msg.diag) Process diagnostic data htoVMSSupHdr(sndbuf.supblk, msg.nativeSB) htoVMS(sndbuf.diag, msg.diag) Pre-fixes proxy_hdr Sends to dbex socket (dbmdiag_sendrpy) Receives TEST_STOP message Refer to General Thread Shutdown

Topics DBS Use Cases Design Considerations DBS Initialization Data Structures Download Sequence ST0 Conversion Receive/Process Setpoints and other messages from Alpha Process/Send SLC IOC udpates to Alpha dblput DB I/0 Utilities

Writing Data from slc ioc (dblput)

Topics DBS Use Cases Design Considerations DBS Initialization Data Structures Download Sequence –ST0 Conversion Receive/Process Setpoints and other messages from Alpha Process/Send SLC IOC udpates to Alpha –Dblput –SLC IOC DB Updates to Alpha DB I/0 Utilities

slc ioc DB Updates to Alpha

dbSendThread – slc ioc updates to Alpha Receives DB_UPDATE messages from various “job threads” to send update Lock dbhilo_mutex[job] – prevent further updates for this job Compress job’s list of pending updates, if possible (microdbsendc) Lock dbRWMutex Read dbhilo_update[job] sptr pair memcpy data from ST block indicated by sptr pair into sndbuf (microdbsendb) Unlock dbRWMutex Increment msg seq # (1-256), write in proxyhdr.user_byte store seq #, “job id” locally Send sndbuf (already in VMS format) to dbex socket

dbSendThread – CTL socket After sending data to dbex socket, loop waiting at CTL socket for Ack from DBEX Further updates for this job are blocked; other job updates q’ed Timeout after ~15 sec (less if !dbex_up) Wait 1 sec at ctl socket Check for slcExec “stop flag” CTL loop exit conditions: 1)Successful Ack (valid msg seq #) Clear dbhilo_update[job] If (! dbex_up), set dbex_up flag 2) Invalid msg seq # If (! dbex_up), set dbex_up flag ; no clear 3) Timeout – no clear 4) Socket error Set “lost connection flag” ; no clear Loop waiting for connection Check for slcExec “stop flag” Unlock dbhilo_mutex[job] Signal ackEvent[job] Upon dbex_up / connection up Send updates for all jobs

Topics DBS Use Cases Design Considerations DBS Initialization Data Structures Download Sequence ST0 Conversion Receive/Process Setpoints and other messages from Alpha Process/Send SLC IOC udpates to Alpha DB Input/Output Utilities

DB I/O Utilities Threads use dballoc(), dbfree(), and db macros for list management dballoc() / dbfree() use mem management Re-entrant Return 4byte unsigned VMS cond code Except dbexists returns epicsBoolean All device args (prim,unit, secn) are 4 chars; blank padded, null-terminated; prim/secn are left justified; unit is right-j? dblist (dblist_p, “prim”,”unit”,”secn”) Unit can = “ALL*” dblunits (dbdata_p, “prim”) dblget (dbdata_p, dblist) dblput (dbdata_p, dblist) dballoc (dbdata_p, numBytes, numElements) dbfree (dbdata_or_list_p) dbunit2string (intUnit, “unit”) dbexists (void)

DB I/O Utilities – cont. micro_dbsend(job) All threads call to send their updates to Alpha Threads wait for ack or timeout DB I/O utils wait on downloadEvent prior to access Calling threads are held off until after db is downloaded; timeout in case of failure Shell commands check dbExists() before calling dbl* utils Error conditions A dblist (dblist_p, prim, unit, secn) of either non-existent primary, unit, or secondary returns an error, except if unit = ALL* A dbunits (dblist_p, prim) of non-existent primary returns an error

DB I/O functionality Dblist (dblist_p, prim, unit, secn) Calls gphFind (“prim unit secn”) Enters ptr to entry into dblist_p Dblist (dblist_p, prim, ALL*, secn) Calls gphFind (“prim”) Traverses linked list of prim/unit; for each entry, traverses linked list of secns to find; gets ptr from matching entry’s data Enters ptr to entry in dblist_p Dblunits (dbdata_p, prim) Calls gphFind (“prim”) Traverses linked list of prim/unit; for each entry, enters (4 char) unit(s) into dbdata_p

DB I/O functionality Dblput (dbdata_p, dblist_p) Converts native typed input dbdata to VMS typed data and stores to dblist location locks / unlocks dbRWMutex around memcpy of data to ST block Calls dbhilo_update (job, hi, lo) inserts hi, lo offset pairs into dbhilo_update[job] locks / unlocks dbhilo_mutex[job] Dblget (dbdata_p, dblist_p) sets/resets dbRWMutex around memcpy from ST block data Converts retrieved data to native Inserts in dblist

Shell commands Shells: EPICS iocsh, RTEMS Cexp, vxWorks All previous dbl* utils can be called from ioc shell after parsing console input. dbexists() will be checked prior to calling underlying dbl* utility - no waiting for user dbgettype (“prim”, “secn”) dbdump (“prim”, “unit”, “secn”, “ ”) dbdumpunits (“prim”, “ ”) dbedit (“prim”, “unit”, “secn”,…) dbdumphash (“ ”) dbdumpdatabase (“ ”)