1. CERN/IT/DB Oracle C++ Call Interface Dirk Geppert IT/DB DB Workshop July 11-13, 2001

2. CERN/IT/DB Outline  Overview  Advanced features  Example  Summary  Advanced example

3. CERN/IT/DB What is it?  API to access Oracle database  OCI Oracle Call Interface  OCCI Oracle C++ Call Interface  OCCI a better OCI: wrapper  scalable, high performance, multi-tiered  ANSI C++, STL  JDBC like

4. CERN/IT/DB What to use it for?  Relational Programming  Associative Access of Relational Data SQL DDL (data definition) SQL DML (data manipulating) SQL queries PL/SQL  Object Programming  Navigational and Associative Access of Objects

5. CERN/IT/DB Advanced features  Thread safety  Connection pooling  Client-side object cache  Complex Object Retrieval  Metadata  Context USER

6. CERN/IT/DB Thread safety  Oracle database server and OCCI in multi-threaded environment  Mechanism to maintain integrity of data accessed by multiple threads  Mutexes are granted on basis of OCCI Environment class  THREADED_MUTEXED Thread safe, mutexed internally managed by OCCI  THREADED_UN_MUTEXED Thread safe, client responsible for mutexing

7. CERN/IT/DB Connection pooling  Multi-threading typically  large number of threads need connections to DB for short duration  Open connection for each thread  Inefficient  Poor performance  OCCI connection pooling  create small number of connections  dynamically select free connection to execute statement  release connection

8. CERN/IT/DB Client-side object cache  Memory buffer for objects  Lookup and memory management support  Stores and tracks objects fetched from server to client side  Maintains references  Manages automatic object swapping  Meta-attributes or type information  Created when OCCI-Environment initialized in object mode  Benefits  Reduces client/server round-trips  Object-level locking

9. CERN/IT/DB Complex Object Retrieval  OO data models  Interrelated objects forming graphs  Navigate/traverse: fetch & pin each object individually  Costly network round-trips (client/server)  Oracle’s Complex Object Retrieval  Pre-fetching (content, boundary) Specific type up to depth level All attribute types up to depth level  Objects only pre-fetched, not pinned  Limited by client-side cache size

10. CERN/IT/DB Metadata  Dynamically discover attributes of database objects  Database itself  Schema objects e.g. tables, columns, types, type attributes, result attributes MetaData evt_md=conn->getMetaData(“Event”); vector attr_list = evt_md.getVector(ATTR_LIST_TYPE_ATTRS);

11. CERN/IT/DB Context USER  user owns  e.g. tables, types, objects  May grant privileges to other users, roles (named groups of privileges)  Access, modify OCCI_STD_NAMESPACE::string Event_C::getSQLTypeName() const {return OCCI_STD_NAMESPACE::string(“SCOTT.EVENT_C");}

12. CERN/IT/DB Type mapping  Oracle database - C++ (host language)  Oracle internal – external  Data conversion of C++ data send/requested to/from database CHARchar NUMBERint, float VARRAYSTL vector REFOCCI Ref

13. CERN/IT/DB create type array as varray(1000) of number / create type atype as object ( n number, a array, p ref atype) / create table atable of atype; insert into atable values (atype(0,null,null)); commit; type atype generate ctype as mytype SQL – OTT – C++ ctype Number n Vector a Ref p mytype PObject SQLOTTOCCI see Peter’s talk

14. CERN/IT/DB Generated C++ classes class ctype : public oracle::occi::Pobject { private: // attributes … public: // set & get attributes … void *operator new(size_t size); void *operator new(size_t size, const oracle::occi::Connection * sess, const OCCI_STD_NAMESPACE::string& table); OCCI_STD_NAMESPACE::string getSQLTypeName() const; ctype(); ctype(void *ctxOCCI_) : oracle::occi::PObject (ctxOCCI_) { }; static void *readSQL(void *ctxOCCI_); virtual void readSQL(oracle::occi::AnyData& streamOCCI_); static void writeSQL(void *objOCCI_, void *ctxOCCI_); virtual void writeSQL(oracle::occi::AnyData& streamOCCI_); } Persistent Object transient persistent SQL

15. CERN/IT/DB Environment *env=Environment::createEnvironment(Environment::OBJECT); RegisterMappings(env); Connection *conn=env->createConnection(“scott”,”tiger”,””); Statement *stmt=conn->createStatement(); ResultSet *resultSet=stmt->executeQuery(“select ref(t) from atable t where n=0”); if (ResultSet::DATA_AVAILABLE == resultSet->next()) { Ref ref=resultSet->getRef(1); } conn->commit(); stmt->closeResultSet(resultSet); conn->terminateConnection(conn); Environment::terminateEnvironment(env); Example Session create/modify/delete objects easy to use

16. CERN/IT/DB Ref ref=resultSet->getRef(1); ref_next=new(conn,”ATABLE”) mytype(); ref->setP(ref_next); // forward ref->markModified(); Creating new objects Ref ref=resultSet->getRef(1); do { ref->dump(); mytype *ptr=ref.ptr(); // if needed } while (!((ref=ref->getP()).isNull())); dereference, pin reference pointer Example navigation persistentmodify

17. CERN/IT/DB Summary  OCCI is C++ API to communicate with Oracle database server  Built on top of OCI  High-performance  scalable  Easy to use  Does it provide the flexibility to use it for HEP data models & access patterns? Tests are on the way…

18. CERN/IT/DB UML Event Track VertexHeader PObject 1 1 1 1 * 1 1 *

19. CERN/IT/DB create type header_o as object (run number,evt number); create type track_o; -- dummy create type track_ref_varray_o as varray(100) of ref track_o; create type vertex_o as object (id number,x number,y number,z number,trk track_ref_varray_o); create type vertex_ref_varray_o as varray(100) of ref vertex_o; create or replace type track_o as object ( id number,px number,py number,pz number,vtx ref vertex_o); create type event_o as object (id number,hdr header,vtx vertex_ref_varray_o,nxt ref event_o); create table event_t of event_o create table vertex_t of vertex_o create table track_t of track_o insert into event_t values (event_o(0,header_o(0,0),null,null)); commit; SQL

20. CERN/IT/DB TYPE Header_O GENERATE Header_C AS Header TYPE Track_O GENERATE Track_C AS Track TYPE Vertex_O GENERATE Vertex_C AS Vertex TYPE Event_O GENERATE Event_C AS Event OTT Oracle C++ User (inherit from C++ class)

21. CERN/IT/DB class Event_C : public oracle::occi::PObject { private: oracle::occi::Number ID; Header * HDR; OCCI_STD_NAMESPACE::vector > VTX; oracle::occi::Ref NXT; public: oracle::occi::Number getId() const; void setId(const oracle::occi::Number &value); Header *getHdr() const; void setHdr(Header *value); OCCI_STD_NAMESPACE::vector > getVtx() const; void setVtx(const OCCI_STD_NAMESPACE::vector > &value); oracle::occi::Ref getNxt() const; void setNxt(const oracle::occi::Ref &value); C++.h set & get attributes

22. CERN/IT/DB void *operator new(size_t size); void *operator new(size_t size, const oracle::occi::Connection * sess, const OCCI_STD_NAMESPACE::string& table); OCCI_STD_NAMESPACE::string getSQLTypeName() const; Event_C(); Event_C(void *ctxOCCI_) : oracle::occi::PObject (ctxOCCI_) { }; static void *readSQL(void *ctxOCCI_); virtual void readSQL(oracle::occi::AnyData& streamOCCI_); static void writeSQL(void *objOCCI_, void *ctxOCCI_); virtual void writeSQL(oracle::occi::AnyData& streamOCCI_); } C++.h cont’d new SQL ctor

23. CERN/IT/DB oracle::occi::Number Event_C::getId() const { return ID; } void Event_C::setId(const oracle::occi::Number &value) { ID = value; } Header_C * Event_c::getHdr() const { return HDR; } void Event_C::setHdr(Header * value) { HDR = value; } OCCI_STD_NAMESPACE::vector > Event_C::getVtx() const { return VTX; } void Event_C::setVtx(const OCCI_STD_NAMESPACE::vector > &value) { VTX = value; } oracle::occi::Ref Event_C::getNxt() const { return NXT; } void Event_C::setNxt(const oracle::occi::Ref &value) { NXT = value; } C++.cpp set & get

24. CERN/IT/DB void *Event_C::operator new(size_t size) { return oracle::occi::PObject::operator new(size); } void *Event_C::operator new(size_t size, const oracle::occi::Connection * sess, const OCCI_STD_NAMESPACE::string& table) { return oracle::occi::PObject::operator new(size, sess, table, (char *) “SCOTT.EVENT_O"); } OCCI_STD_NAMESPACE::string Event_C::getSQLTypeName() const { return OCCI_STD_NAMESPACE::string(“SCOTT.EVENT_O"); } Event_C::Event_C() {HDR = (Header *) 0;} C++.cpp cont’d new ctor

25. CERN/IT/DB void *Event_C::readSQL(void *ctxOCCI_) { Event_C *objOCCI_ = new Event_C(ctxOCCI_); oracle::occi::AnyData streamOCCI_(ctxOCCI_); try { if (streamOCCI_.isNull()) objOCCI_->setNull(); else objOCCI_->readSQL(streamOCCI_); } catch (oracle::occi::SQLException& excep) { delete objOCCI_; excep.setErrorCtx(ctxOCCI_); return (void *)NULL; } return (void *)objOCCI_; } void Event_C::readSQL(oracle::occi::AnyData& streamOCCI_) { ID = streamOCCI_.getNumber(); HDR = (Header *) streamOCCI_.getObject(); getVector(streamOCCI_, VTX); NXT = streamOCCI_.getRef(); } C++.cpp cont’d readSQL

26. CERN/IT/DB void Event_C::writeSQL(void *objectOCCI_, void *ctxOCCI_) { Event_C *objOCCI_ = (Event_C *) objectOCCI_; oracle::occi::AnyData streamOCCI_(ctxOCCI_); try { if (objOCCI_->isNull()) streamOCCI_.setNull(); else objOCCI_->writeSQL(streamOCCI_); } catch (oracle::occi::SQLException& excep) { excep.setErrorCtx(ctxOCCI_); } return; } void Event_C::writeSQL(oracle::occi::AnyData& streamOCCI_) { streamOCCI_.setNumber(ID); streamOCCI_.setObject(HDR); setVector(streamOCCI_,VTX); streamOCCI_.setRef(NXT); } C++.cpp cont’d writeSQL

27. CERN/IT/DB void Event::dump() { cout << "Event::dump()" << endl; cout << " ID= " << int(getId()) << endl; if (getHdr()!=0) getHdr()->dump(); OCCI_STD_NAMESPACE::vector > &vtx=getVtx(); cout << " Vtx length= " << vtx.size() << endl; for (int i=0; i<vtx.size(); i++) { oracle::occi::Ref v=vtx[i]; v->dump(); } if (getNxt().isNull()) cout << " Next is NULL" << endl; else cout << " Next exists " << endl; }; User C++ class Event : public Event_C { public: void dump(); };

28. CERN/IT/DB Ref evt_ref, evt_ref_next; Event *evt_ptr, *evt_ptr_next; unsigned long int eid,vid,tid; int evt,run; Header* hdr; Vertex* vtx; Track* trk; evt_ref = resultSet->getRef(1); for (int ievent=0; ievent<nevent; ievent++) { hdr = new Header(); // and set attributes… Navigation - write Header

29. CERN/IT/DB OCCI_STD_NAMESPACE::vector > vtx_ref_arr; for (int ivertex=0; ivertex<nvertex; ivertex++) { vtx=new (conn,"VERTEX_T") Vertex(); OCCI_STD_NAMESPACE::vector > trk_ref_arr; for (int itrack=0; itrack<2; itrack++) { trk=new (conn,"TRACK_T") Track(); trk->setVtx(vtx); // and other attributes… trk_ref_arr.push_back(trk); } vtx->setTrk(trk_ref_arr); // and other attributes… vtx_ref_arr.push_back(vtx); } evt_ptr_next = new(conn,"EVENT_T") Event(); evt_ptr_next->setHdr(hdr); evt_ptr_next->setVtx(vtx_ref_arr); // and other attributes… evt_ptr->setNxt(evt_ptr_next); // forward pointing evt_ptr->markModified(); conn->commit(); evt_ptr=evt_ptr_next; } Navigation - write cont’d Track Vertex Event

30. CERN/IT/DB Ref evt_ref = resultSet->getRef(1); do { evt_ref->dump(); } while (!((evt_ref=evt_ref->getNxt()).isNull())); Navigation - read

31. CERN/IT/DB MAPFILE fileCPPFILE fileHFILE file OCCI Source Compiler OCCI libraryLinkerObject file Executable OTT INTYPE fileSQL DDL Database Server Type Definitions Object Type Translator 1 4 3 2 5