1. GUS Plugin System Michael Saffitz Genomics Unified Schema Workshop July 6-8th, Philadelphia, Pennsylvania

2. Plugin Overview Small Perl programs that load and manipulate data within GUS Written using the GUS Plugin API and Perl Object Layer Provide automatic support for: Data Provenance Object layer and database connectivity Standardized documentation Command line argument processing Logging Error Handling “Supported” and “Community” Plugins provided with GUS

3. Supported Plugins Have been tested in Oracle and Postgres and are confirmed to work Portable Useful beyond the site that developed them Meet the GUS Plugin Standard

4. Community Plugins Fail to meet one or more of the criteria above Have not been tested Provided as a general resource to the community

5. Plugin Life Cycle Plugin Initialization Documentation Command Line Arguments Data Loading Reading, Parsing, Querying Data Manipulation Insert or Update? Restart Logic Data Submission

6. GUS Supported Plugins InsertArrayDesignControl.pm InsertAssayControl.pm InsertBlastSimilarities.pm InsertExternalDatabase.pm InsertExternalDatabaseRls.pm InsertGOEvidenceCode.pm InsertGeneOntology.pm InsertGeneOntologyAssoc.pm InsertRadAnalysis.pm InsertReviewStatus.pm InsertSecondaryStructure.pm InsertSequenceOntology.pm LoadArrayDesign.pm LoadArrayResults.pm LoadFastaSequences.pm LoadGusXml.pm LoadNRDB.pm LoadRow.pm LoadTaxon.pm

7. Plugin Shell package GUS::Supported::Plugin::LoadRow; @ISA = qw(GUS::PluginMgr::Plugin); use strict; use GUS::PluginMgr::Plugin; sub new { … } sub run { … }

8. Plugin Initialization sub new { my ($class) = @_; my $self = {}; bless($self, $class); $self->initialize({ requiredDbVersion => 3.5, cvsRevision => '$Revision: 2934 $', name => ref($self), argsDeclaration => $argsDeclaration, documentation => $documentation }); return $self; }

9. Declaring Arguments stringArg({name => 'externalDatabaseVersion', descr => 'sres.externaldatabaserelease.version for this instance of NRDB', constraintFunc => undef, reqd => 1, isList => 0 }), fileArg({name => 'gitax', descr => 'pathname for the gi_taxid_prot.dmp file', constraintFunc => undef, reqd => 1, isList => 0, mustExist => 1, format => 'Text' }),

10. Argument Types String Integer Boolean Table Name Float File Enumeration Controlled Vocab Local, Database Term Pairs for “dinky” CVs

11. Declaring Documentation my $tablesDependedOn = [ ['GUS::Model::DoTS::NRDBEntry', 'pulls aa_sequence_id from here when id and extDbId match requested']]; my $documentation = { purposeBrief => $purposeBrief, purpose => $purpose, tablesAffected => $tablesAffected, tablesDependedOn => $tablesDependedOn, howToRestart => $howToRestart, failureCases => $failureCases, notes => $notes };

12. Plugin Initialization sub new { my ($class) = @_; my $self = {}; bless($self, $class); $self->initialize({ requiredDbVersion => 3.5, cvsRevision => '$Revision: 2934 $', name => ref($self), argsDeclaration => $argsDeclaration, documentation => $documentation }); return $self; }

13. Plugin Shell package GUS::Supported::Plugin::LoadRow; @ISA = qw(GUS::PluginMgr::Plugin); use strict; use GUS::PluginMgr::Plugin; sub new { … } sub run { … }

14. Run Method “Entry point” for plugin Concise overview/“table of contents” for plugin: sub run { my ($self) = @_; my $rows = 0; my $rawData = $self->readData(); my @parsedData = $self->parseData($rawData); foreach $data (@parsedData) { $data->submit(); $rows++; } return “Inserted $rows ”; }

15. Accessing Data Command line arguments: $self->getArg(‘nrdbFile); Through Objects: my $preExtAASeq =GUS::Model::DoTS::ExternalAASequence->new ({'aa_sequence_id'=>$aa_seq_id}); $preExtAASeq->retrieveFromDB(); Direct Database Access: my $dbh = $self->getQueryHandle(); my $sth = $dbh->prepare(…);

16. Persisting Data Saving & Updating: $obj->submit(); Will cascade and submit children Delete: $obj->markDeleted(1); $obj->submit();

17. Logging and Error Handling For general logging, use logging functions Printed to STDERR $self->log(“message”) For error handling: Either die() immediately or Write errors to a file (for recoverable errors) Restart functionality Check for object existence Check, but ensure loaded from a valid proper invocation Store data from previous run and use as a filter

18. Clearing the Cache Historical: Perl previously had poor garbage collection support Default capacity of 10000 objects At the bottom of the outermost loop: $self->undefPointerCache();

19. Data Provenance Tracks plugin revisions-- Name, Checksum, Revision Tracks parameters that a specific plugin is executed with Algorithm AlgorithmImplementation AlgorithmInvocation AlgorithmParamKey AlgorithmParamKeyType AlgorithmParam

20. Plugin Evolution Changes abound: Data file formats Schema Be flexible in writing plugins-- command line configuration Be clear about what schema objects you use

21. Plugin Standard See Developer’s Guide: http://gusdb.org/documentation/3.5/developers/developersguide.html