Indexing overview

Contents

• Introduction to indexing
• How Magento implements indexing
• Magento indexers
• Adding a custom indexer

Introduction to indexing

Indexing is how Magento transforms data such as products, categories, and so on, to improve the performance of your storefront. As data changes, the transformed data must be updated—or reindexed. Magento has a very sophisticated architecture that stores lots of merchant data (including catalog data, prices, users, stores, and so on) in many database tables. To optimize storefront performance, Magento accumulates data into special tables using indexers.

For example, suppose you change the price of an item from $4.99 to $3.99. Magento must reindex the price change to display it on your storefront.

Without indexing, Magento would have to calculate the price of every product on the fly—taking into account shopping cart price rules, bundle pricing, discounts, tier pricing, and so on. Loading the price for a product would take a long time, possibly resulting in cart abandonment.

Indexing terminology:

Dictionary

Original data entered to the system. Dictionaries are organized in normal form to facilitate maintenance (updating the data).

Index

Representation of the original data for optimized reading and searching. Indexes can contain results of aggregations and various calculations. Index data can be always re-created from a dictionary using a certain algorithm.

Indexer

Object that creates an index.

Create custom indexers

Magento contains several indexers out of the box, but you might want to add your own if your customization requires data searches, which are not optimized by the Magento default indexers.

This topic provides a high level description of how indexing is implemented from a developer’s point of view, and practical advice of how to add your own indexer.

How Magento implements indexing

The following components are involved in the indexing process:

Component	Description
Magento_Indexer	Implements: indexer declaration indexer running indexer running mode configuration indexer status
Magento\Framework\Mview	Allows tracking database changes for a certain entity (product, category and so on) and running change handler. Emulates the materialized view technology for MySQL using triggers and separate materialization process (provides executing PHP code instead of SQL queries, which allows materializing multiple queries).

Indexing types

Each index can perform the following types of reindex operations:

• Full reindex, which means rebuilding all the indexing-related database tables.

  Full reindexing can be caused by a variety of things, including creating a new web store or new customer group.

  You can optionally fully reindex at any time using the command line.

• Partial reindex, which means rebuilding the database tables only for the things that changed (for example, changing a single product attribute or price).

The type of reindex performed in each particular case depends on the type of changes made in the dictionary or in the system. This dependency is specific for each indexer.

The following figure shows the logic for partial reindexing.

Indexer status

Depending on whether an index data is up to date, an indexer status value is one of the following:

• valid: data is synchronized, no reindex required
• invalid: the original data was changed, the index should be updated
• working: indexing is in progress

The Magento indexing mechanism uses the status value in reindex triggering process. You can check the status of an indexer in the Admin panel under System > New Index Management or manually using the command line.

Indexing modes

Reindexing can be performed in two modes:

• Update on Save: index tables are updated immediately after the dictionary data is changed.
• Update by Schedule: index tables are updated by cron job according to the configured schedule.

To set these options:

1. Log in to the Magento Admin.
2. Click System > Index Management.
3. Select the check box next to each type of indexer to change.
4. From the Actions list, click the indexing mode.
5. Click Submit.

You can also reindex from the command line

The following figure shows an example of setting indexers to Update by Schedule.

How to reindex

You can reindex in any of the following ways:

• Using a cron job (preferred because indexing runs every minute)
• Using the magento indexer:reindex [indexer] command, which reindexes selected indexers, or all indexers, one time only

Magento indexers

The Magento application implements the following indexers:

Indexer name	Indexer method name	Indexer class	Description
Category products	catalog_category_product	Magento\Catalog\Model\Indexer\Category\Product	Creates category/products association
Product categories	catalog_product_category	Magento\Catalog\Model\Indexer\Product\Category	Creates category/products association
Product price	catalog_product_price	Magento\Catalog\Model\Indexer\Product\Price	Pre-calculates product prices
Product entity attribute value	catalog_product_attribute	Magento\Catalog\Model\Indexer\Product\Eav	Reorganizes the EAV product structure to flat structure
Stock	cataloginventory_stock	Magento\CatalogInventory\Model\Indexer\Stock
Catalog rule product	catalogrule_rule	Magento\CatalogRule\Model\Indexer\Rule\RuleProductIndexer
Catalog product rule	catalogrule_product	Magento\CatalogRule\Model\Indexer\Product\ProductRuleIndexer
Catalog search	catalogsearch_fulltext	Magento\CatalogSearch\Model\Indexer\Fulltext