Magento Indexers Ivan Chepurnyi Magento Trainer / Lead Developer

Magento Developers Meetup Agenda Overview of Indexes Functionality Creation of own indexes Magento Developers Meetup

Magento Developers Meetup Let Imagine… … that Magento doesn’t have indexes: The prices in product list are calculated on the fly depending on catalog rules, tier prices for customer groups Stock availability for configurable and bundle products can be calculated only after loading the product collection Layered navigation data is build in real-time for product attributes information Anchor categories recursively collects subcategories for filtering product list Magento Developers Meetup

It’s all about performance… The main goal is minimizing amount of operations to display products to a customer Magento Developers Meetup

Magento Developers Meetup Definitions Indexed Data Aggregated data for entity representation on the frontend lists. Indexer Generates index data on event or manual by process. Index Event The moment when entity or related to it information is changed and that affects its index data. Index Process Wrapper for indexer and contains information about its mode and status Main Controller Forwards events to Index Process Magento Developers Meetup

Magento Developers Meetup Index Workflow Event Main Controller Event Events Manual Invoke Process Indexer Indexed Data Magento Developers Meetup

Magento Developers Meetup Event Types Save When indexed entity or related to it information was changed Delete When indexed entity or related to it one was deleted Mass Update When batch of entities was updated. (Update Attributes on Product Grid) Magento Developers Meetup

Magento Developers Meetup Observed Entities Indexed Entities Product Product Inventory Category Tag Entities Scope Customer Group Website Store Group Store View Magento Developers Meetup

Magento Developers Meetup Index Process Available Statuses Pending Indicates that indexer is up to date Running Index currently in process of full rebuilding index data. Require Reindex Status for notifying admin user, that index is not up to date and should be rebuild. Magento Developers Meetup

Magento Developers Meetup Index Process Indexing Modes Real-time Manual Update Index Data Event Event Require Reindex Magento Developers Meetup

Magento Developers Meetup Indexer Flow Match Event Main Controller Index Process Register Event Data Reindex Data Magento Developers Meetup

Mage_Index Module Main Controller Process Indexer Base Mage_Index_Model_Indexer Process Mage_Index_Model_Process Indexer Base Mage_Index_Model_Indexer_Abstract Magento Developers Meetup

… Indexers Modularity Mage_Index_Model_Indexer_Abstract Index Module Mage_Index_Model_Indexer_Abstract Catalog Module Mage_Catalog_Model_Product_Indexer_Eav Mage_Catalog_Model_Product_Indexer_Flat Mage_Catalog_Model_Product_Indexer_Price Inventory Module Mage_CatalogIndex_Model_Indexer_Stock … Magento Developers Meetup

Indexer Structure Model Resource Model Mage_Index_Model_Indexer_Abstract Resource Model Mage_Index_Model_Mysql4_Abstract Matches event data and runs appropriate method in resource model for re-indexing Works directly with database for generation of the indexed data. Usually all the data operated via MySQL queries. Magento Developers Meetup

What can you use? Mage_Index_Model_Indexer getProcessByCode($indexerCode) getProcessCollection() processEntityAction($entity, $entityType, $eventType) Mage_Index_Model_Process reindexAll() reindexEverything() setMode($mode) Magento Developers Meetup

Magento Developers Meetup What you shouldn’t do… Invoke reindexAll method from index model/resource model, because it is better to let admin user know when the index was rebuild. Process entity events directly with indexer, instead of passing data through the main controller. You never know which index may depend on this event. Magento Developers Meetup

Magento Developers Meetup Creating own indexer Defining indexer in configuration Designing index data table Implementing model Implementing resource model Applying index on the frontend Magento Developers Meetup

Magento Developers Meetup Featured Products There is easier way to create featured products functionality, but it is a simple example on what should be done for creation own indexer. Magento Developers Meetup

Defining index in configuration etc/config.xml <config> <!-- …. module configurtaions --> <global> <index> <indexer> <featured_products> <model>your_module/indexer_featured</model> </featured_products> </indexer> </index> </global> </config> Indexer Code Indexer Model Magento Developers Meetup

Designing index data table Adding new attribute to catalog product entity called is_featured Creating table that will contain product ids of products that are marked as featured products. Magento Developers Meetup

Designing index data table Attribute Code Setup Script $this->addAttribute('catalog_product', 'is_featured', array( 'type' => 'int', 'label' => 'Is featured', 'input' => 'select', 'source' => 'eav/entity_attribute_source_boolean', 'user_defined' => false, 'required' => false )); Yes/No Dropdown Magento Developers Meetup

Designing index data table Setup Script Table Alias $table = new Varien_Db_Ddl_Table(); $table->setName($this->getTable(‘module/featured')); $table->addColumn('product_id', Varien_Db_Ddl_Table::TYPE_INTEGER, 11, array( 'unsigned' => true, 'nullable' => false, 'primary' => true )); $this->getConnection()->createTable($table); Table Column Magento Developers Meetup

Designing index data table Setup Script Table Alias $table = new Varien_Db_Ddl_Table(); $table->setName($this->getTable(‘module/featured')); $table->addColumn('product_id', Varien_Db_Ddl_Table::TYPE_INTEGER, 11, array( 'unsigned' => true, 'nullable' => false, 'primary' => true )); $this->getConnection()->createTable($table); Table Column Magento Developers Meetup

Magento Developers Meetup Implementing Model Defining Matching Events class Your_Module_Model_Indexer_Featured extends Mage_Index_Model_Indexer_Abstract { protected $_matchedEntities = array( Mage_Catalog_Model_Product::ENTITY => array( Mage_Index_Model_Event::TYPE_SAVE, Mage_Index_Model_Event::TYPE_MASS_ACTION ) ); } Entity Type Event Types Event Type Magento Developers Meetup

Magento Developers Meetup Implementing Model Defining Indexer Resource Model class Your_Module_Model_Indexer_Featured extends Mage_Index_Model_Indexer_Abstract { // … other code protected function _construct() $this->_init(‘your_module/indexer_featured'); } Resource model Magento Developers Meetup

Implementing Model Defining Indexer Information class Your_Module_Model_Indexer_Featured extends Mage_Index_Model_Indexer_Abstract { // … other code public function getName() return Mage::helper(‘your_module')->__('Featured Product'); } public function getDescription() return Mage::helper(‘‘your_module')->__('Indexes something'); Indexer Name in the admin Indexer Description in the admin Magento Developers Meetup

Implementing Model Register Event for Processing class Your_Module_Model_Indexer_Featured extends Mage_Index_Model_Indexer_Abstract { // … other code protected function _registerEvent(Mage_Index_Model_Event $event) /* @var $entity Mage_Catalog_Model_Product */ $entity = $event->getDataObject(); if ($entity->dataHasChangedFor('is_featured')) { $event->setData('product_id', $entity->getId()); } elseif ($entity->getAttributesData()) { $attributeData = $entity->getAttributesData(); if (isset($attributeData['is_featured'])) { $event->setData('product_ids', $entity->getProductIds()); } Product Save Registering Mass Action Registering Magento Developers Meetup

Implementing Model Processing Event class Your_Module_Model_Indexer_Featured extends Mage_Index_Model_Indexer_Abstract { // … other code protected function _processEvent(Mage_Index_Model_Event $event) if ($event->getData('product_id') || $event->getData('product_ids')) { $this->callEventHandler($event); } Calling processor in resource model Entity Type Event Type catalogProductSave($event) catalogProductMassAction($event) Magento Developers Meetup

Implementing Resource Model Define resource connection class Your_Module_Model_Mysql4_Indexer_Featured extends Mage_Index_Model_Mysql4_Abstract { protected function _construct() $this->_setResource(‘your_module'); } Your module resource prefix Magento Developers Meetup

Implementing Resource Model Indexing Method class Your_Module_Model_Mysql4_Indexer_Featured extends Mage_Index_Model_Mysql4_Abstract { // … other code protected function _reindexEntity($productId = null) $select = $this->_getReadAdapter()->select(); /* @var $attribute Mage_Catalog_Model_Resource_Eav_Attribute */ $attribute = Mage::getSingleton('eav/config') ->getAttribute('catalog_product', 'is_featured'); $select->from($attribute->getBackendTable(), 'entity_id') ->where('value = ?', 1) ->where('attribute_id = ?', $attribute->getId()); Retrieving only featured product ids Magento Developers Meetup

Implementing Resource Model Indexing Method If it is partial re-index, then delete only related indexed data if ($productId !== null) { if (!is_array($productId)) { $productId = array($productId); } $select->where('entity_id IN(?)', $productId); $this->_getWriteAdapter()->delete( $this->getTable(‘your_module/featured'), array( 'product_id IN(?)' => $productId ) ); } else { $this->_getWriteAdapter()->truncate($this->getTable(‘your_module/featured')); Otherwise clear all indexed data Magento Developers Meetup

Implementing Resource Model Fulfill index data from select we created before Indexing Method $sqlStatement = $select->insertIgnoreFromSelect( $this->getTable(‘your_module/featured'), array('product_id') ); $this->_getWriteAdapter()->query($sqlStatement); } Magento Developers Meetup

Implementing Resource Model Handling Events class Your_Module_Model_Mysql4_Indexer_Featured extends Mage_Index_Model_Mysql4_Abstract { // … other code public function reindexAll() $this->_reindexEntity(); } Full index re-build Magento Developers Meetup

Implementing Resource Model Reindexing Events class Your_Module_Model_Mysql4_Indexer_Featured extends Mage_Index_Model_Mysql4_Abstract { // … other code public function catalogProductSave($event) $this->_reindexEntity($event->getData('product_id')); } public function catalogProductMassAction($event) $this->_reindexEntity($event->getData('product_ids')); Single Save Product Event Mass Save Product Event Magento Developers Meetup

Applying Index for the frontend Observing and event catalog_product_collection_apply_limitations_after Joining index table to product collection select Create sub-select filter for collection Magento Developers Meetup

Magento Developers Meetup Liked it? Checkout our advanced training programs: http://www.ecomdev.org/magento- development-training-programs/advanced Follow our blog posts: http://www.ecomdev.org/blog Magento Developers Meetup

Questions?