Feeds Module for Drupal 7 Use Cases, Overview, and Walktroughs

Who Am I? johnbarclay on drupal.org University of Illinois College of Education web developer Pixotech.com (Pixo) Consultant Maintain LDAP module Like drupal community/dev model Need to be interrupted

Who are You? Who has used feeds in drupal 6 Who has used feed in drupal 7 Who knows they have a use case for feeds? Who may have a use case for feeds (importing or synching data into drupal) Be thinking of these for middle part of talk.

Goals In love with feeds. Using like crazy the last couple of months. Want to: sell the module it give leg up and warnings for new users give away a book

Breakdown Terms, Module Features, General Use Cases. 10 minutes. Keep me in check. A simple node import walk through. 10 minutes. Examples of use cases we’ve implemented or need to implement. 20 minutes. Start thinking. Advanced Feeds. Gotchas and Tools.

What is Feeds Module? FeedsFeeds is a pluggable system for importing or aggregating content into Drupal. Investing time into learning it can make user import, node import, etc. obsolete. Very extensible. Class Based with stack of interchangeable plugins + hooks. Supports exportables/features.

Use Cases for Site Builders Data Sources: RSS Feeds, XML Feeds, CSV Files, SOAP, WDSL, LDAP, Databases... Data Targets: Nodes, Users, Taxonomy,...

Use Cases For Developers For data ingestion, only need to write plugins for unique data sources or targets. Can leverage other plugins and core feeds functionality (batch, unique identifiers, user interface). For migrations and many use cases don’t need any code at all Supports exportables/features

Architecture Fetcher Parser Processor Importer mapper

Know the Lingo Feeds Glossary very helpful Key to understanding documentation

Fetcher Fetcher gets data. Specific to transport/storage type crawling, directory, file upload, http, ldap query, sql,

Parser Parser Normalizes Data. Specific to data structure Known Data Structures: CSV, RSS, LDAP Entry, OPML, SQL Recordset Customizable Data Structures: Querypath, Xpath, Xpath HTML

Processor Processor “does stuff”. Generally stores feed items in a data structure. Could send out s or do other obscure use cases. Node, Taxonomy Term, User built in. Data, Commerce Products, Self Node Processor, etc.

(field) Mappers required to know where data from feed goes into a field. basic core field types in Feeds module contrib mappers in contrib modules

Other Terms plugin – fetcher, processor, parser feed – body of data (csv file) feed item – individual item (csv line) feed node – confusing. instance of importer stored in node. mapping – source to target field mapping guid and unique target

Import CSV Example

Examples for Learning In the simpletest folder in feeds are example feeds sources and test configurations

Our Use Cases What are you using feeds for? What do you need to either import on a one time basis or synch continually.

My Common Use Cases MSSQL Data Synch to Nodes & Taxonomy XML Feed to Nodes LDAP Query to User Records CSV to Nodes to Taxonomy

Where are Plugins Processors and Mappers: for core structures such as node, user, and taxonomy in feeds module. Outside core, should be in related contrib module or own module.

7.x version not ready Lots of bugs and RTBC patches If it does the job great. Read issues. When importing large amounts of data, everything has to work or need an alternative approach. my notes: if you use a patch, RTBC!

Feeds Tamper Handy for edge cases trim, regex, explode, etc.

Write Plugins search the sandbox use other plugins as templates extend other plugins rather than rewriting use file logging for debugging because of batch nature of execution. dpm() and tracing problematic.

Add to documentation feeds are confusing

7.x-2.x gotchas