Saving the world through the wonder that is >>> CQL <<< Mike Taylor Index Data – With.

Slides:



Advertisements
Similar presentations
Introduction to Java 2 Programming Lecture 10 API Review; Where Next.
Advertisements

Theo van Veen, Koninklijke Bibliotheek The European Library: opportunities for new services.
SRW/U for DSpace Ralph LeVan Research Scientist. What is SRW/U A Pair of HTTP-based Text Query Protocols – SRW: Search and Retrieve Web Service – SRU:
Search Web Services Ralph LeVan Senior Research Scientist.
OCLC Online Computer Library Center SRW & DSpace Ralph LeVan OCLC Research.
A centre of expertise in digital information management UKOLN is supported by: SRU: An overview of the SRU protocol and how it can be used.
CQL – a Common Query LanguageMike Taylor CQL – a Common Query Language 1. What CQL is 2. Motivation 3. Examples and explanation 4. Applications 5. Implementation.
Searching very large bodies of data using a transparent peer-to-peer proxy Mike Taylor and Marc Cromme, Index Data
Delivering MARC/XML records from the Library of Congress catalogue using the open protocols SRW/U and Z39.50 Mike Taylor, Index Data
?! Advanced CQL and ProfilingMike Taylor Advanced CQL and Profiling 1. Esoteric CQL features: – Word Anchoring – Proximity – Relation.
? CQL – a Common Query LanguageMike Taylor CQL – a Common Query Language 1. What CQL is 2. Motivation 3. Examples and explanation 4.
World Meteorological Organization Working together in weather, climate and water WMO Information System (WIS) Search (with SRU) Timo Pröscholdt (PO-WIS)
XSL eXtensible Stylesheet Language. What is XSL? XSL is a language that allows one to describe a browser how to process an XML file. XSL can convert an.
CS 430 / INFO 430 Information Retrieval
1Key – Report Creation with DB2. DB2 Databases Create Domain for DB2 Test Demo.
Multi-Model Digital Video Library Professor: Michael Lyu Member: Jacky Ma Joan Chung Multi-Model Digital Video Library LYU9904 Multi-Model Digital Video.
The best kind of friend is the kind you
BEAUTIFUL... LIFE! BEAUTIFUL... LIFE! The photos are great, but the philosophy is even better!
Nuggets of Wisdom From an forwarded by a friend.
Memoplex (Browser)++: A Semantic Document Browser Clarence Chan CPSC 533 Nov. 12, 2006.
River Campus Libraries CUIPID Catalog User-Interface Platform for Iterative Development David Lindahl Director of Digital Library Initiatives River Campus.
1 CS 430 / INFO 430 Information Retrieval Lecture 7 String Processing.
Rethinking the library catalogue: making search work for the library user Sally Chambers The European Library
Dynamic Web Pages Bert Wachsmuth. Review  Internet, IP addresses, ports, client-server, http, smtp  HTML, XHTML, XML  Style Sheets, external, internal,
The World Wide Web and the Internet Dr Jim Briggs 1WUCM1.
River Campus Libraries Find Articles A Web Redesign for ENCompass David Lindahl Web Initiatives Manager River Campus Libraries University of Rochester.
XML(EXtensible Markup Language). XML XML stands for EXtensible Markup Language. XML is a markup language much like HTML. XML was designed to describe.
Federated Searching: The ABC’s of HSE, XML, & Z39.50 Harry Samuels Product Manager Linking & Searching August 27, 2004.
Python Web Applications A KISS Introduction. Web Applications with Python Fetching, parsing, text processing Database client – mySQL, etc., for building.
Chapter 6 DOJO TOOLKITS. Objectives Discuss XML DOM Discuss JSON Discuss Ajax Response in XML, HTML, JSON, and Other Data Type.
Struts 2.0 an Overview ( )
Databases with PHP A quick introduction. Y’all know SQL and Databases  You put data in  You get data out  You can do processing on it very easily 
READING, WRITING, BINDING, VALIDATING AND VISUALISING YOUR DATA Business value with Silverlight.
Introduction to InfoSec – Recitation 7 Nir Krakowski (nirkrako at post.tau.ac.il) Itamar Gilad (itamargi at post.tau.ac.il)
Z39.50 for Finding It All William E. Moen School of Library and Information Sciences Texas Center for Digital Knowledge University of North Texas Denton,
Parsing arithmetic expressions Reading material: These notes and an implementation (see course web page). The best way to prepare [to be a programmer]
MapReduce With a SQL-MapReduce focus by Curt A. Monash, Ph.D. President, Monash Research Editor, DBMS2
1 In the good old days... Years ago… the WWW was made up of (mostly) static documents. –Each URL corresponded to a single file stored on some hard disk.
Department of Computer Engineering College of Engineering An-Najah National University Prepared by : Saif Marwan & Osama Nabulsi Supervisor Name: Dr. Loay.
A Web Services Approach for Search and Retrieve The Next Generation Z39.50 Access 2004, October 13-16, 2004, Halifax, Nova Scotia William E. Moen School.
University of North Texas Libraries Building Search Systems for Digital Library Collections Mark E. Phillips Texas Conference on Digital Libraries May.
Linking electronic documents and standardisation of URL’s What can libraries do to enhance dynamic linking and bring related information within a distance.
RELATIONAL FAULT TOLERANT INTERFACE TO HETEROGENEOUS DISTRIBUTED DATABASES Prof. Osama Abulnaja Afraa Khalifah
Introduction to Web Services Eric Lease Morgan University Libraries of Notre Dame June 24, 2005.
1 CS 502: Computing Methods for Digital Libraries Lecture 19 Interoperability Z39.50.
ICDL 2004 Improving Federated Service for Non-cooperating Digital Libraries R. Shi, K. Maly, M. Zubair Department of Computer Science Old Dominion University.
B y T aylor C hase Definition and types of bullying  Bullying is making someone feel bad. Bullying is like abuse, it’s a really bad thing to do.  Types.
OCLC Online Computer Library Center Interoperability Standards & Searching Multiple Repositories Ralph LeVan/OCLC Ray Denenberg/Library of Congress.
Facebook API Kelly Orser. Client Libraries Client libraries will simplify the calls to the platform by reducing the amount of code you have to write.
CNI, 4th April 2006 Slide 1 Key Standards Update: SRU (“Technical” Details) Dr. Robert Sanderson Dept. of Computer Science University of Liverpool
Meta-Server System Software Lab. Overview In the Music Virtual Channel system, clients can’t query for a song initiatively Through the metadata server,
SRW/U: Re-Introduction SRW is a Web Services based Information Retrieval Protocol Motivations: Create an easy to implement protocol with the power of Z39.50.
Lawrence Snyder University of Washington, Seattle © Lawrence Snyder 2004.
Windows 7 WampServer 2.1 MySQL PHP 5.3 Script Apache Server User Record or Select Media Upload to Internet Return URL Forward URL Create.
Feb 24-27, 2004ICDL 2004, New Dehli Improving Federated Service for Non-cooperating Digital Libraries R. Shi, K. Maly, M. Zubair Department of Computer.
Next Generation Z39.50 A Web Services Approach for Search and Retrieve 6 th Annual State GILS Conference, March 31 – April 3, 2004, Raleigh, NC William.
 An essential supporting structure of any thing  A Software Framework  Has layered structure ▪ What kind of functions and how they interrelate  Has.
Z39.50 and the ZING Initiatives: MAVIS Users Conference, 2003 November 6, 2003 Larry E. Dixson Library of Congress.
IST 210: PHP Basics IST 210: Organization of Data IST2101.
XML 1.Introduction to XML 2.Document Type Definition (DTD) 3.XML Parser 4.Example: CGI Gateway to XML Middleware.
Session IV Chapter 14 – Chapter 14 – XSLThttp://
Introduction to Mongo DB(NO SQL data Base)
102 % of everything is crap I know the law is “90%” of everything is crap. Unfortunately, my percentage calculator isn't in the other 10%. This is a true.
OUTLINE Basic ideas of traditional retrieval systems
Introduction to Internet Programming
OAI and Metadata Harvesting
CS 430 / INFO 430 Information Retrieval
XML Problems and Solutions
SDLIP + STARTS = SDARTS A Protocol and Toolkit for Metasearching
Agile testing for web API with Postman
Presentation transcript:

Saving the world through the wonder that is >>> CQL <<< Mike Taylor Index Data – With a little help from BMNH R5937 “The Archbishop”

Saving the world through CQLMike Taylor, Index Data Overview Where we're headed: Why OpenSearch rules so much

Saving the world through CQLMike Taylor, Index Data Overview Where we're headed: Why OpenSearch rules so much Why OpenSearch sucks so hard

Saving the world through CQLMike Taylor, Index Data Overview Where we're headed: Why OpenSearch rules so much Why OpenSearch sucks so hard CQL saves the world!

Saving the world through CQLMike Taylor, Index Data Overview Where we're headed: Why OpenSearch rules so much Why OpenSearch sucks so hard CQL saves the world! Why CQL is hard to implement

Saving the world through CQLMike Taylor, Index Data Overview Where we're headed: Why OpenSearch rules so much Why OpenSearch sucks so hard CQL saves the world! Why CQL is hard to implement Why CQL is easy to implement

Saving the world through CQLMike Taylor, Index Data Overview Where we're headed: Why OpenSearch rules so much Why OpenSearch sucks so hard CQL saves the world! Why CQL is hard to implement Why CQL is easy to implement Now what?

Saving the world through CQLMike Taylor, Index Data Why OpenSearch rules so much OpenSearch is great because: It gives clients a well-defined way to request searches It returns data in a well-known format OpenSearch Metasearcher OpenSearch Server 1 Nice, coherent XML

Saving the world through CQLMike Taylor, Index Data Why OpenSearch rules so much OpenSearch is great because: It gives clients a well-defined way to request searches It returns data in a well-known format OpenSearch Metasearcher OpenSearch Server 1 OpenSearch Server 2 Nice, coherent XMLSame XML

Saving the world through CQLMike Taylor, Index Data Why OpenSearch rules so much OpenSearch is great because: It gives clients a well-defined way to request searches It returns data in a well-known format OpenSearch Metasearcher OpenSearch Server 1 OpenSearch Server 3 OpenSearch Server 2 Nice, coherent XMLSame XMLSame XML again!

Saving the world through CQLMike Taylor, Index Data Why OpenSearch sucks so hard OpenSearch Metasearcher OpenSearch Server 1 OpenSearch Server 3 OpenSearch Server 2 Ghasty proprietary query language #1

Saving the world through CQLMike Taylor, Index Data Why OpenSearch sucks so hard OpenSearch Metasearcher OpenSearch Server 1 OpenSearch Server 3 OpenSearch Server 2 Ghasty proprietary query language #1 Revolting proprietary query language #2

Saving the world through CQLMike Taylor, Index Data Why OpenSearch sucks so hard OpenSearch Metasearcher OpenSearch Server 1 OpenSearch Server 3 OpenSearch Server 2 Ghasty proprietary query language #1 Revolting proprietary query language #2 Ethically horrifying, foul and totally incomprehensible query language #3

Saving the world through CQLMike Taylor, Index Data Why OpenSearch sucks so hard OpenSearch Metasearcher OpenSearch Server 1 OpenSearch Server 3 OpenSearch Server 2 Ghasty proprietary query language #1 Revolting proprietary query language #2 Ethically horrifying, foul and totally incomprehensible query language #3 !!!Aaaargh!!!!

Mike Taylor, Index Data CQL saves the world! Saving the world through CQL OpenSearchSRU Metasearcher SRU Server 1 SRU Server 3 SRU Server 2 Lovely CQL

Mike Taylor, Index Data CQL saves the world! Saving the world through CQL OpenSearchSRU Metasearcher SRU Server 1 SRU Server 3 SRU Server 2 Lovely CQL

Mike Taylor, Index Data CQL saves the world! Saving the world through CQL OpenSearchSRU Metasearcher SRU Server 1 SRU Server 3 SRU Server 2 Lovely CQL

Mike Taylor, Index Data CQL saves the world! Saving the world through CQL OpenSearchSRU Metasearcher SRU Server 1 SRU Server 3 SRU Server 2 Lovely CQL : –)

Mike Taylor, Index Data CQL saves the world: see for yourself Saving the world through CQL European Library SRU Server 1 SRU Server 3 SRU Server 2 Lovely CQL Talk to Theo about this

Mike Taylor, Index Data What if... Saving the world through CQL OpenSearch Metasearcher OpenSearch Server 1 OpenSearch Server 3 OpenSearch Server 2 Lovely CQL Even more lovely CQL! Yum! Lovely CQL

Mike Taylor, Index Data OK, what actually is CQL? Saving the world through CQL dinosaur

Mike Taylor, Index Data OK, what actually is CQL? Saving the world through CQL dinosaur dinosaur and vertebra

Mike Taylor, Index Data OK, what actually is CQL? Saving the world through CQL dinosaur dinosaur and vertebra dinosaur or pterosaur

Mike Taylor, Index Data OK, what actually is CQL? Saving the world through CQL dinosaur dinosaur and vertebra dinosaur or pterosaur vertebra not dorsal

Mike Taylor, Index Data OK, what actually is CQL? Saving the world through CQL dinosaur dinosaur and vertebra dinosaur or pterosaur vertebra not dorsal (dinosaur or pterosaur) and (vertebra not dorsal)

Mike Taylor, Index Data OK, what actually is CQL? Saving the world through CQL dinosaur dinosaur and vertebra dinosaur or pterosaur vertebra not dorsal (dinosaur or pterosaur) and (vertebra not dorsal) title=churchill

Mike Taylor, Index Data OK, what actually is CQL? Saving the world through CQL dinosaur dinosaur and vertebra dinosaur or pterosaur vertebra not dorsal (dinosaur or pterosaur) and (vertebra not dorsal) title=churchill creator=churchill

Mike Taylor, Index Data OK, what actually is CQL? Saving the world through CQL dinosaur dinosaur and vertebra dinosaur or pterosaur vertebra not dorsal (dinosaur or pterosaur) and (vertebra not dorsal) title=churchill creator=churchill subject = (dinosaurs and biomechanics)

Mike Taylor, Index Data OK, what actually is CQL? Saving the world through CQL dinosaur dinosaur and vertebra dinosaur or pterosaur vertebra not dorsal (dinosaur or pterosaur) and (vertebra not dorsal) title=churchill creator=churchill subject = (dinosaurs and biomechanics) Hey! What's the big deal? This is easy!

Mike Taylor, Index Data OK, what actually is CQL? Saving the world through CQL People of a faint disposition might like to look away now.

Mike Taylor, Index Data OK, what actually is CQL? Saving the world through CQL People of a faint disposition might like to look away now. dc.author=(kern* or ritchie) and (bath.title exact "the c programming language" or dc.title=elements prox/distance=4 dc.title=programming) and subject any/relevant "style design analysis" (Yes, that's all one query.)

Mike Taylor, Index Data OK, what actually is CQL? Saving the world through CQL People of a faint disposition might like to look away now. dc.author=(kern* or ritchie) and (bath.title exact "the c programming language" or dc.title=elements prox/distance=4 dc.title=programming) and subject any/relevant "style design analysis" (Yes, that's all one query.) Don't try this at home, kids. Get JavaScript to build it for you.

Mike Taylor, Index Data The CQL learning curve Saving the world through CQL Power of query that can be expressed Effort in learning query language SQL

Mike Taylor, Index Data The CQL learning curve Saving the world through CQL Power of query that can be expressed Effort in learning query language SQL Google

Mike Taylor, Index Data The CQL learning curve Saving the world through CQL Power of query that can be expressed Effort in learning query language SQL Google CQL

Mike Taylor, Index Data Why CQL is hard to implement Saving the world through CQL Are you kidding? Just look at all that power: multi-word terms with wildcards and anchoring indexes (creator, title, subject...) index prefixes (dc for Dublin Core, rec for Record Metadata) relations ( > = <> exact all any) relation modifiers booleans (and, or, not, prox) boolean modifiers (ouch, ouch, ouch) prefix mapping... and much, much more!

Mike Taylor, Index Data Why CQL is easy to implement Saving the world through CQL How do you decode CGI parameters? – use a library! How do you parse XML? – use a library! How do you send an HTTP request and get the response? – use a library!

Mike Taylor, Index Data Why CQL is easy to implement Saving the world through CQL How do you decode CGI parameters? – use a library! How do you parse XML? – use a library! How do you send an HTTP request and get the response? – use a library! Now, who can guess how to parse CQL?

Mike Taylor, Index Data Why CQL is easy to implement (Java) Saving the world through CQL import org.z3950.zing.cql; CQLParser parser = new CQLParser(); CQLNode root; try { root = parser.parse(cql); } catch (CQLParseException ex) { System.err.println("Syntax error: " + ex.getMessage()); }

Mike Taylor, Index Data Why CQL is easy to implement (Java) Saving the world through CQL import org.z3950.zing.cql; CQLParser parser = new CQLParser(); CQLNode root; try { root = parser.parse(cql); } catch (CQLParseException ex) { System.err.println("Syntax error: " + ex.getMessage()); } Super-easy!

Mike Taylor, Index Data Why CQL is easy to implement (Perl) Saving the world through CQL use CQL::Parser; my $parser = new CQL::Parser(); my $node; eval { $node = $parser->parse($query); }; if { die "CQL syntax error: }

Mike Taylor, Index Data Why CQL is easy to implement (Perl) Saving the world through CQL use CQL::Parser; my $parser = new CQL::Parser(); my $node; eval { $node = $parser->parse($query); }; if { die "CQL syntax error: } EmbarrassinglyTrivial!

Mike Taylor, Index Data Why CQL is easy to implement (C) Saving the world through CQL #include CQL_parser cp = cql_parser_create(); struct cql_node *root; r = cql_parser_string(cp, query); if (r) { fprintf(stderr, "Syntax error\n"); } else { root = cql_parser_result(cp); }

Mike Taylor, Index Data Why CQL is easy to implement (C) Saving the world through CQL #include CQL_parser cp = cql_parser_create(); struct cql_node *root; r = cql_parser_string(cp, query); if (r) { fprintf(stderr, "Syntax error\n"); } else { root = cql_parser_result(cp); } A child of three could do it!

Mike Taylor, Index Data Why CQL is easy to implement (Intercal) Saving the world through CQL DO (5) NEXT (5) DO FORGET #1 PLEASE WRITE IN :1 DO.1 <- 'V-":1~'#32768c/#0'"c/#1'~#3 DO (1) NEXT DO :1 <- "'V-":1~'#65535c/#0'"c/#65535' ~'#0c/#65535'"c/"'V-":1~'#0c/#65535'" c/#65535'~'#0c/#65535'" DO :2 <- #1 PLEASE DO (4) NEXT (4) DO FORGET #1 DO.1 <- "V-':1~:2'c/#1"~#3 DO :1 <- "'V-":1~'#65535c/#0'"c/":2~'#65535 c/#0'"'~'#0c/#65535'"c/"'V-":1~'#0

Mike Taylor, Index Data Why CQL is easy to implement (Intercal) Saving the world through CQL DO (5) NEXT (5) DO FORGET #1 PLEASE WRITE IN :1 DO.1 <- 'V-":1~'#32768c/#0'"c/#1'~#3 DO (1) NEXT DO :1 <- "'V-":1~'#65535c/#0'"c/#65535' ~'#0c/#65535'"c/"'V-":1~'#0c/#65535'" c/#65535'~'#0c/#65535'" DO :2 <- #1 PLEASE DO (4) NEXT (4) DO FORGET #1 DO.1 <- "V-':1~:2'c/#1"~#3 DO :1 <- "'V-":1~'#65535c/#0'"c/":2~'#65535 c/#0'"'~'#0c/#65535'"c/"'V-":1~'#0 So easy, it hurts!

Mike Taylor, Index Data Now what? Saving the world through CQL Once you've got your CQL parse tree... Walk the tree Convert it to your database's local query language If a CQL construct is unsupported, throw a diagnostic

Mike Taylor, Index Data Now what? Saving the world through CQL Once you've got your CQL parse tree... Walk the tree Conv ert it to your database's local query language If a CQL construct is unsupported, throw a diagnostic It doesn't matter if you don't implement the whole language (No-one does that.) The goal is: “if you can say it in your DB, you can say it in CQL.”

Mike Taylor, Index Data Boldly facing a bright new dawn Saving the world through CQL It really and truly is not that big a deal to implement CQL.

Mike Taylor, Index Data Boldly facing a bright new dawn Saving the world through CQL It really and truly is not that big a deal to implement CQL. Winners: Content providers: their databases get used

Mike Taylor, Index Data Boldly facing a bright new dawn Saving the world through CQL It really and truly is not that big a deal to implement CQL. Winners: Content providers: their databases get used Vendors: they can concentrate on the interesting problems

Mike Taylor, Index Data Boldly facing a bright new dawn Saving the world through CQL It really and truly is not that big a deal to implement CQL. Winners: Content providers: their databases get used Vendors: they can concentrate on the interesting problems Libraries: they don't need expensive metasearch solutions

Mike Taylor, Index Data Boldly facing a bright new dawn Saving the world through CQL It really and truly is not that big a deal to implement CQL. Winners: Content providers: their databases get used Vendors: they can concentrate on the interesting problems Libraries: they don't need expensive metasearch solutions Users: they find more good stuff!

Mike Taylor, Index Data Boldly facing a bright new dawn Saving the world through CQL It really and truly is not that big a deal to implement CQL. Winners: Content providers: their databases get used Vendors: they can concentrate on the interesting problems Libraries: they don't need expensive metasearch solutions Users: they find more good stuff! Losers: No-one!

Saving the world through the wonder that is >>> CQL <<< Mike Taylor Index Data – With a little help from BMNH R5937 “The Archbishop”, an as-yet undescribed brachiosaurid sauropod from the Upper Jurassic of the Tendaguru formation, Tanzania, Africa.