1. Magento Indexers
Ivan Chepurnyi
Magento Trainer / Lead Developer

2. Magento Developers Meetup
Agenda
Overview of Indexes Functionality
Creation of own indexes
Magento Developers Meetup

3. Magento Developers Meetup
Let Imagine…
… that Magento doesn’t have indexes:
The prices in product list are calculated on the fly depending on catalog rules, tier prices for customer groups
Stock availability for configurable and bundle products can be calculated only after loading the product collection
Layered navigation data is build in real-time for product attributes information
Anchor categories recursively collects subcategories for filtering product list
Magento Developers Meetup

4. It’s all about performance…
The main goal is minimizing amount of operations to display products to a customer
Magento Developers Meetup

5. Magento Developers Meetup
Definitions
Indexed Data Aggregated data for entity representation on the frontend lists.
Indexer
Generates index data on event or manual by process.
Index Event The moment when entity or related to it information is changed and that affects its index data.
Index Process
Wrapper for indexer and contains information about its mode and status
Main Controller
Forwards events to Index Process
Magento Developers Meetup

6. Magento Developers Meetup
Index Workflow
Event
Main Controller
Event
Events
Manual
Invoke
Process
Indexer
Indexed Data
Magento Developers Meetup

7. Magento Developers Meetup
Event Types
Save
When indexed entity or related to it information was changed
Delete
When indexed entity or related to it one was deleted
Mass Update When batch of entities was updated. (Update Attributes on Product Grid)
Magento Developers Meetup

8. Magento Developers Meetup
Observed Entities
Indexed Entities
Product
Product Inventory
Category
Tag
Entities Scope
Customer Group
Website
Store Group
Store View
Magento Developers Meetup

9. Magento Developers Meetup
Index Process
Available Statuses
Pending
Indicates that indexer is up to date
Running
Index currently in process of full rebuilding index data.
Require Reindex Status for notifying admin user, that index is not up to date and should be rebuild.
Magento Developers Meetup

10. Magento Developers Meetup
Index Process
Indexing Modes
Real-time
Manual
Update Index Data
Event
Event
Require Reindex
Magento Developers Meetup

11. Magento Developers Meetup
Indexer Flow
Match Event
Main Controller
Index Process
Register Event Data
Reindex Data
Magento Developers Meetup

12. Mage_Index Module Main Controller Process Indexer Base
Mage_Index_Model_Indexer
Process
Mage_Index_Model_Process
Indexer Base
Mage_Index_Model_Indexer_Abstract
Magento Developers Meetup

13. … Indexers Modularity Mage_Index_Model_Indexer_Abstract
Index Module
Mage_Index_Model_Indexer_Abstract
Catalog Module
Mage_Catalog_Model_Product_Indexer_Eav
Mage_Catalog_Model_Product_Indexer_Flat
Mage_Catalog_Model_Product_Indexer_Price
Inventory Module
Mage_CatalogIndex_Model_Indexer_Stock …
Magento Developers Meetup

14. Indexer Structure Model Resource Model
Mage_Index_Model_Indexer_Abstract
Resource Model
Mage_Index_Model_Mysql4_Abstract
Matches event data and runs appropriate method in resource model for re-indexing
Works directly with database for generation of the indexed data. Usually all the data operated via MySQL queries.
Magento Developers Meetup

15. What can you use? Mage_Index_Model_Indexer
getProcessByCode($indexerCode)
getProcessCollection()
processEntityAction($entity, $entityType, $eventType)
Mage_Index_Model_Process
reindexAll()
reindexEverything()
setMode($mode)
Magento Developers Meetup

16. Magento Developers Meetup
What you shouldn’t do…
Invoke reindexAll method from index model/resource model, because it is better to let admin user know when the index was rebuild.
Process entity events directly with indexer, instead of passing data through the main controller. You never know which index may depend on this event.
Magento Developers Meetup

17. Magento Developers Meetup
Creating own indexer
Defining indexer in configuration
Designing index data table
Implementing model
Implementing resource model
Applying index on the frontend
Magento Developers Meetup

18. Magento Developers Meetup
Featured Products
There is easier way to create featured products functionality, but it is a simple example on what should be done for creation own indexer.
Magento Developers Meetup

19. Defining index in configuration
etc/config.xml
<config>
<!-- …. module configurtaions -->
<global>
<index>
<indexer>
<featured_products>
<model>your_module/indexer_featured</model>
</featured_products>
</indexer>
</index>
</global>
</config>
Indexer Code
Indexer Model
Magento Developers Meetup

20. Designing index data table
Adding new attribute to catalog product entity called is_featured
Creating table that will contain product ids of products that are marked as featured products.
Magento Developers Meetup

21. Designing index data table
Attribute Code
Setup Script
$this->addAttribute('catalog_product', 'is_featured', array(
'type' => 'int',
'label' => 'Is featured',
'input' => 'select',
'source' => 'eav/entity_attribute_source_boolean',
'user_defined' => false,
'required' => false
));
Yes/No Dropdown
Magento Developers Meetup

22. Designing index data table
Setup Script
Table Alias
$table = new Varien_Db_Ddl_Table();
$table->setName($this->getTable(‘module/featured'));
$table->addColumn('product_id', Varien_Db_Ddl_Table::TYPE_INTEGER, 11, array(
'unsigned' => true,
'nullable' => false,
'primary' => true
));
$this->getConnection()->createTable($table);
Table Column
Magento Developers Meetup

23. Designing index data table
Setup Script
Table Alias
$table = new Varien_Db_Ddl_Table();
$table->setName($this->getTable(‘module/featured'));
$table->addColumn('product_id', Varien_Db_Ddl_Table::TYPE_INTEGER, 11, array(
'unsigned' => true,
'nullable' => false,
'primary' => true
));
$this->getConnection()->createTable($table);
Table Column
Magento Developers Meetup

24. Magento Developers Meetup
Implementing Model
Defining Matching Events
class Your_Module_Model_Indexer_Featured extends Mage_Index_Model_Indexer_Abstract {
protected $_matchedEntities = array(
Mage_Catalog_Model_Product::ENTITY => array(
Mage_Index_Model_Event::TYPE_SAVE,
Mage_Index_Model_Event::TYPE_MASS_ACTION ) ); }
Entity Type
Event Types
Event Type
Magento Developers Meetup

25. Magento Developers Meetup
Implementing Model
Defining Indexer Resource Model
class Your_Module_Model_Indexer_Featured extends Mage_Index_Model_Indexer_Abstract {
// … other code
protected function _construct()
$this->_init(‘your_module/indexer_featured'); }
Resource model
Magento Developers Meetup

26. Implementing Model Defining Indexer Information
class Your_Module_Model_Indexer_Featured extends Mage_Index_Model_Indexer_Abstract {
// … other code
public function getName()
return Mage::helper(‘your_module')->__('Featured Product'); }
public function getDescription()
return Mage::helper(‘‘your_module')->__('Indexes something');
Indexer Name in the admin
Indexer Description in the admin
Magento Developers Meetup

27. Implementing Model Register Event for Processing
class Your_Module_Model_Indexer_Featured extends Mage_Index_Model_Indexer_Abstract {
// … other code
protected function _registerEvent(Mage_Index_Model_Event $event)
$entity Mage_Catalog_Model_Product */
$entity = $event->getDataObject();
if ($entity->dataHasChangedFor('is_featured')) {
$event->setData('product_id', $entity->getId());
} elseif ($entity->getAttributesData()) {
$attributeData = $entity->getAttributesData();
if (isset($attributeData['is_featured'])) {
$event->setData('product_ids', $entity->getProductIds()); }
Product Save Registering
Mass Action Registering
Magento Developers Meetup

28. Implementing Model Processing Event
class Your_Module_Model_Indexer_Featured extends Mage_Index_Model_Indexer_Abstract {
// … other code
protected function _processEvent(Mage_Index_Model_Event $event)
if ($event->getData('product_id') || $event->getData('product_ids')) {
$this->callEventHandler($event); }
Calling processor in resource model
Entity Type
Event Type
catalogProductSave($event)
catalogProductMassAction($event)
Magento Developers Meetup

29. Implementing Resource Model
Define resource connection
class Your_Module_Model_Mysql4_Indexer_Featured extends Mage_Index_Model_Mysql4_Abstract {
protected function _construct()
$this->_setResource(‘your_module'); }
Your module resource prefix
Magento Developers Meetup

30. Implementing Resource Model
Indexing Method
class Your_Module_Model_Mysql4_Indexer_Featured extends Mage_Index_Model_Mysql4_Abstract {
// … other code
protected function _reindexEntity($productId = null)
$select = $this->_getReadAdapter()->select();
$attribute Mage_Catalog_Model_Resource_Eav_Attribute */
$attribute = Mage::getSingleton('eav/config')
->getAttribute('catalog_product', 'is_featured');
$select->from($attribute->getBackendTable(), 'entity_id')
->where('value = ?', 1)
->where('attribute_id = ?', $attribute->getId());
Retrieving only featured product ids
Magento Developers Meetup

31. Implementing Resource Model
Indexing Method
If it is partial re-index, then delete only related indexed data
if ($productId !== null) {
if (!is_array($productId)) {
$productId = array($productId); }
$select->where('entity_id IN(?)', $productId);
$this->_getWriteAdapter()->delete(
$this->getTable(‘your_module/featured'),
array(
'product_id IN(?)' => $productId ) );
} else {
$this->_getWriteAdapter()->truncate($this->getTable(‘your_module/featured'));
Otherwise clear all indexed data
Magento Developers Meetup

32. Implementing Resource Model
Fulfill index data from select we created before
Indexing Method
$sqlStatement = $select->insertIgnoreFromSelect(
$this->getTable(‘your_module/featured'),
array('product_id') );
$this->_getWriteAdapter()->query($sqlStatement); }
Magento Developers Meetup

33. Implementing Resource Model
Handling Events
class Your_Module_Model_Mysql4_Indexer_Featured extends Mage_Index_Model_Mysql4_Abstract {
// … other code
public function reindexAll()
$this->_reindexEntity(); }
Full index re-build
Magento Developers Meetup

34. Implementing Resource Model
Reindexing Events
class Your_Module_Model_Mysql4_Indexer_Featured extends Mage_Index_Model_Mysql4_Abstract {
// … other code
public function catalogProductSave($event)
$this->_reindexEntity($event->getData('product_id')); }
public function catalogProductMassAction($event)
$this->_reindexEntity($event->getData('product_ids'));
Single Save Product Event
Mass Save Product Event
Magento Developers Meetup

35. Applying Index for the frontend
Observing and event catalog_product_collection_apply_limitations_after
Joining index table to product collection select
Create sub-select filter for collection
Magento Developers Meetup

36. Magento Developers Meetup
Liked it?
Checkout our advanced
training programs:
development-training-programs/advanced
Follow our blog posts:
Magento Developers Meetup

37. Questions?