3.2.2

Documentation updated

3.2.1

Self-filtering option for individual filter (disabled by default).

Feature Request
Advanced configuration for AggregationQuery, if enabled, then result for filter can contain only filter values.
Useful for cases with ValueIntersectionFilter (AND condition).

$filters[] = (new ValueIntersectionFilter('size', [12,32]))->selfFiltering(true);

3.2.0

• ValueIntersectionFilter added Feature Request
• Enable self-filtering option for AggregationQuery added

ValueIntersectionFilter

Default Filters example:

Find phones with memory sizes ANY OF (12, 32, 64) AND camera 12m

$filters[] = new ValueFilter(‘size’, [12,32,64]);
$filters[] = new ValueFilter(‘camera’, [12]);

New functionality example:

Search brand "Digma" OR "Pony" where the recommended usage is for portraits AND wildlife.
Can be used for items with multiple field values

<?php 
$data = [
 ['id'=>1, 'brand'=>'Digma', 'usage'=>['portraits', 'wildlife']],
 ['id'=>2, 'brand'=>'Pony', 'usage'=>['streetphoto', 'weddings','portraits']],
];

// ...

$filters[] = new ValueFilter('brand', ['Digma', 'Pony']); // ANY OF
$filters[] = new ValueIntersectionFilter('usage', ['portraits', 'wildlife']); // portraits AND wildlife

Self-filtering

Aggregates disables property self-filtering by default. It allow the user to choose another option in the interface.

Example:
User wants a phone with 32GB memory, checks the box for the desired option from (16, 32, 64).
If self-filtering is enabled, then all other options in the UI will disappear and only 32 will remain.
Thus, user will not be able to change his choice.

During aggregation field filter value is used to limit values only other fields.
Example: the "size" filter condition uses to limit the list of "brand" field variations.

All depends on your use case of the library.
Initially, the library was developed to simplify the construction of a search UI.
If you want to use the library at the level of technical analysis, statistics, etc. , then enabling self-filtering can help you to get expected results.

$query = (new AggregationQuery())->filters($filters)->countItems()->sort()->selfFiltering(true);

3.1.0

Exclude Filters

Feature Request

• ExcludeValueFilter and ExcludeRageFilter added
• Updated examples demonstrates the use of most of the library's functions
• Added sorting inside RangeIndexer's sets

New filters allows to select values ​​and ranges to exclude from search results.
In some cases, such feature is convenient for users, especially when their search comes from understanding what they definitely do not want to see.

Usage is similar to regular filters:

// ....
<?php
$query = (new AggregationQuery())
          ->filters([
                new ExcludeValueFilter('color', ['green']), // remove products with green color from results
                new ValueFilter('size', [41,42]),
          ])
          // Count items for each acceptable filter value (slower)
          ->countItems()
          // Sort results by fields and values
          ->sort();
// ...

Notes

ExcludeValueFilter slightly slows down the search (Query), but speeding up the construction of aggregates. Considering that aggregates are much slower than searches, the functionality in general has a positive effect on performance.

In order to organize user-friendly behavior, additional sorting is needed within the RangeIndexer ranges. Sorting requires additional resources during index construction, and does not affect the performance of subsequent use of indexes.

Sorting within ranges is possible only during the initial creating of index, since the connection with the real value is lost. Therefore, when using the RangeIndexer, you should not use adding new single values after a complete rebuild. As a workaround new values will be added to the end of range and be sorted only inside new values. This is relevant only for cases with sorting by field indexed by RangeIndexer.

3.0.0

• Removed deprecated methods.
• The code has been refactored, the complexity has been reduced.
• The library API has been slightly changed.
• Documentation and demo updated according to the new API.
• Improved performance of FixedArrayStorage.
• The new version fully supports data exported from 2.2.x indexes.

Api changes

<?php
use KSamuel\FacetedSearch\Index\Factory;

// Index creation moved to factory method
$search = (new Factory)->create(Factory::ARRAY_STORAGE);

// The data storage is moved to a separate object
$storage = $search->getStorage();

$data = [
    ['id'=>7, 'color'=>'black', 'price'=>100, 'sale'=>true, 'size'=>36],   
    ['id'=>9, 'color'=>'green', 'price'=>100, 'sale'=>true, 'size'=>40], 
    // ....
];

foreach($data as $item){ 
   $recordId = $item['id'];
   unset($item['id']);

   // Data and indexers are now passed to the storage
   $storage->addRecord($recordId, $item);
}
$storage->optimize();

// Data export is now performed by a separate method
$indexData = $storage->export();

file_put_contents('./first-index.json', json_encode($indexData));

v3.0.0 Bench ArrayIndex PHP 8.2.3 + JIT + opcache (no xdebug extension)

Items count	Memory	Find	Get Filters (aggregate)	Get Filters & Count (aggregate)	Sort by field	Results Found
10,000	~3Mb	~0.0008 s.	~0.001 s.	~0.002 s.	~0.0001 s.	907
50,000	~20Mb	~0.002 s.	~0.005 s.	~0.010 s.	~0.0006 s.	4550
100,000	~40Mb	~0.004 s.	~0.012 s.	~0.023 s.	~0.0012 s.	8817
300,000	~95Mb	~0.010 s.	~0.036 s.	~0.079 s	~0.004 s.	26891
1,000,000	~329Mb	~0.039 s.	~0.134 s.	~0.287 s.	~0.015 s.	90520
1,000,000 UB	~324Mb	~0.103 s.	~0.225 s.	~0.406 s.	~0.032 s.	179856

v3.0.0 Bench FixedArrayIndex PHP 8.2 + JIT + opcache (no xdebug extension)

Items count	Memory	Find	Get Filters (aggregate)	Get Filters & Count (aggregate)	Sort by field	Results Found
10,000	~2Mb	~0.0012 s.	~0.001 s.	~0.005 s.	~0.0004 s.	907
50,000	~12Mb	~0.004 s.	~0.006 s.	~0.022 s.	~0.001 s.	4550
100,000	~23Mb	~0.007 s.	~0.015 s.	~0.048 s.	~0.002 s.	8817
300,000	~70Mb	~0.020 s.	~0.046 s.	~0.142 s.	~0.005 s.	26891
1,000,000	~233Mb	~0.081 s.	~0.172 s.	~0.517 s.	~0.021 s.	90520
1,000,000 UB	~233Mb	~0.149 s.	~0.260 s.	~0.682 s.	~0.039 s.	179856

2.2.1

Added the ability to update index data without a complete rebuild.
New methods added:

use KSamuel\FacetedSearch\Index\ArrayIndex;

$index = new ArrayIndex();
$index->setData($dataFromStorage);

// delete record from index
$index->deleteRecord($recordId);
// replace record data with new values
$index->replaceRecord($recordId,['newField'=>'newValue'/* .... */]);

FixedArrayIndex also implements new methods

use KSamuel\FacetedSearch\Index\FixedArrayIndex;

$index = new ArrayIndex();
$index->writeMode();
$index->setData($dataFromStorage);

// delete record from index
$index->deleteRecord($recordId);
// replace record data with new values
$index->replaceRecord($recordId,['newField'=>'newValue'/* .... */]);

$index->commitChanges();

2.2.0

Global update

• New Query API
• More efficient result sorting using SearchQuery
• Ability to sort aggregation results
• Performance improvements
• Index optimization using $searchIndex->optimize()
• FilterInterface changed

New Query API

<?php
use KSamuel\FacetedSearch\Index\ArrayIndex;
use KSamuel\FacetedSearch\Search;
use KSamuel\FacetedSearch\Query\SearchQuery;
use KSamuel\FacetedSearch\Query\AggregationQuery;
use KSamuel\FacetedSearch\Query\Order;
use KSamuel\FacetedSearch\Filter\ValueFilter;

// load index
$searchIndex = new ArrayIndex();
$searchIndex->setData($someIndexData);
// create search instance
$search = new Search($searchIndex);

// Find results
$query = (new SearchQuery())
    ->filters([
        new ValueFilter('color', ['black','white']),
        new ValueFilter('size', [41,42])
    ])
    // It is possible to set List of record id to search in. 
    // For example list of records id that found by external FullText search.
    ->inRecords([1,2,3,19,17,21/*..some input record ids..*/])
   // Now results can be sorted by field value.
   // Note! If result item has not such field then item will be excluded from results
    ->order('price', Order::SORT_DESC);

$results = $search->query(query);   

// Find Acceptable filters for user selected input
$query = (new AggregationQuery())
          ->filters([
                new ValueFilter('color', ['black','white']),
                new ValueFilter('size', [41,42])
          ])
          // Count items for each acceptable filter value (slower)
          ->countItems()
          // Sort results by fields and values
          ->sort();

$results = $search->aggregate(query);            

New aggregation API has changed result format for $search->aggregate()
With countItems:

 [
    'field1' => [
        'value1' => 10,
        'value2' => 20
    ]
 ]

Without countItems:

 [
    'field1' => [
        'value1' => true,
        'value2' => true
    ]
 ]

The change was necessary to unify the results structure.
Old API produces results as before in slightly different formats for:

$search->findAcceptableFilters();
$search->findAcceptableFiltersCount();

Backward compatibility

The version is fully backward compatible if you haven't used own filters implementations.

The old API format is available but marked as deprecated.

FilterInterface changed. You need to take this into account if you implemented your own versions of filters

//Interface 
use KSamuel\FacetedSearch\Filter\FilterInterface;
//changed
public function filterResults(array $facetedData, ?array $inputIdKeys = null): array;
//replaced with
public function filterInput(array $facetedData,  array &$inputIdKeys): void;

Performance

Added index optimization method.

<?php
use KSamuel\FacetedSearch\Index\ArrayIndex;

$searchIndex = new ArrayIndex();
/*
 * Getting products data from DB
 */
$data = [
    ['id'=>7, 'color'=>'black', 'price'=>100, 'sale'=>true, 'size'=>36],   
    // ....
];
foreach($data as $item){ 
   $recordId = $item['id'];
   // no need to add faceted index by id
   unset($item['id']);
   $searchIndex->addRecord($recordId, $item);
}

// You can optionally call index optimization before using (since v2.2.0). 
// The procedure can be run once after changing the index data. 
// Optimization takes a few seconds, you should not call it during the processing of user requests.
$searchIndex->optimize();

// save index data to some storage 
$indexData = $searchIndex->getData();
// We will use file for example
file_put_contents('./first-index.json', json_encode($indexData));

Unbalanced Dataset added to Benchmark test

v2.2.0 Bench ArrayIndex PHP 8.2 + JIT + opcache (no xdebug extension)

Items count	Memory	Find	Get Filters (aggregate)	Get Filters & Count (aggregate)	Sort by field	Results Found
10,000	~3Mb	~0.0004 s.	~0.001 s.	~0.002 s.	~0.0001 s.	907
50,000	~20Mb	~0.001 s.	~0.005 s.	~0.010 s.	~0.0004 s.	4550
100,000	~40Mb	~0.003 s.	~0.013 s.	~0.023 s.	~0.0009 s.	8817
300,000	~95Mb	~0.009 s.	~0.034 s.	~0.077 s	~0.003 s.	26891
1,000,000	~329Mb	~0.039 s.	~0.131 s.	~0.281 s.	~0.014 s.	90520
1,000,000 UB	~324Mb	~0.099 s.	~0.218 s.	~0.401 s.	~0.028 s.	179856

v2.2.0 Bench FixedArrayIndex PHP 8.2 + JIT + opcache (no xdebug extension)

Items count	Memory	Find	Get Filters (aggregate)	Get Filters & Count (aggregate)	Sort by field	Results Found
10,000	~2Mb	~0.0007 s.	~0.001 s.	~0.003 s.	~0.0002 s.	907
50,000	~12Mb	~0.003 s.	~0.007 s.	~0.017 s.	~0.0009 s.	4550
100,000	~23Mb	~0.006 s.	~0.017 s.	~0.039 s.	~0.001 s.	8817
300,000	~70Mb	~0.020 s.	~0.056 s.	~0.120 s.	~0.005 s.	26891
1,000,000	~233Mb	~0.073 s.	~0.207 s.	~0.447 s.	~0.021 s.	90520
1,000,000 UB	~233Mb	~0.162 s.	~0.271 s.	~0.609 s.	~0.035 s.	179856

Previous version

v2.1.5 Bench ArrayIndex PHP 8.2 + JIT + opcache (no xdebug extension)

Items count	Memory	Find	Get Filters (aggregate)	Get Filters & Count (aggregate)	Sort by field	Results Found
10,000	~3Mb	~0.0004 s.	~0.001 s.	~0.002 s.	~0.0001 s.	907
50,000	~20Mb	~0.001 s.	~0.006 s.	~0.011 s.	~0.0005 s.	4550
100,000	~40Mb	~0.003 s.	~0.014 s.	~0.024 s.	~0.001 s.	8817
300,000	~95Mb	~0.010 s.	~0.042 s.	~0.082 s	~0.003 s.	26891
1,000,000	~329Mb	~0.046 s.	~0.164 s.	~0.306 s.	~0.015 s.	90520
1,000,000 UB	~324Mb	~0.102 s.	~0.238 s.	~0.446 s.	~0.031 s.	179856

v2.1.5 Bench FixedArrayIndex PHP 8.2 + JIT + opcache (no xdebug extension)

Items count	Memory	Find	Get Filters (aggregate)	Get Filters & Count (aggregate)	Sort by field	Results Found
10,000	~2Mb	~0.0006 s.	~0.001 s.	~0.003 s.	~0.0002 s.	907
50,000	~12Mb	~0.003 s.	~0.007 s.	~0.017 s.	~0.0009 s.	4550
100,000	~23Mb	~0.006 s.	~0.017 s.	~0.040 s.	~0.001 s.	8817
300,000	~70Mb	~0.019 s.	~0.056 s.	~0.120 s.	~0.006 s.	26891
1,000,000	~233Mb	~0.077 s.	~0.202 s.	~0.455 s.	~0.023 s.	90520
1,000,000 UB	~233Mb	~0.146 s.	~0.292 s.	~0.586 s.	~0.044 s.	179856

2.1.6

Bug Fix

• issue#8 Version 2.1.5 does not allow integer names for data fields. Reported by pixobit.

2.1.5

Performance update

Aggregate method now up to 33 % faster.

PHPBench v2.1.5 ArrayIndex PHP 8.1.10 + JIT + opcache (no xdebug extension)

Items count	Memory	Find	Get Filters (aggregate)	Get Filters & Count (aggregate)	Sort by field	Results Found
10,000	~6Mb	~0.0004 s.	~0.001 s.	~0.002 s.	~0.0001 s.	907
50,000	~40Mb	~0.001 s.	~0.005 s.	~0.010 s.	~0.0005 s.	4550
100,000	~80Mb	~0.003 s.	~0.016 s.	~0.029 s.	~0.001 s.	8817
300,000	~189Mb	~0.011 s.	~0.044 s.	~0.091 s	~0.004 s.	26891
1,000,000	~657Mb	~0.047 s.	~0.169 s.	~0.333 s.	~0.018 s.	90520

PHPBench v2.1.5 FixedArrayIndex PHP 8.1.10 + JIT + opcache (no xdebug extension)

Items count	Memory	Find	Get Filters (aggregate)	Get Filters & Count (aggregate)	Sort by field	Results Found
10,000	~2Mb	~0.0007 s.	~0.001 s.	~0.003 s.	~0.0002 s.	907
50,000	~12Mb	~0.003 s.	~0.007 s.	~0.018 s.	~0.0009 s.	4550
100,000	~23Mb	~0.006 s.	~0.017 s.	~0.040 s.	~0.002 s.	8817
300,000	~70Mb	~0.020 s.	~0.059 s.	~0.118 s.	~0.006 s.	26891
1,000,000	~233Mb	~0.079 s.	~0.206 s.	~0.448 s.	~0.026 s.	90520

2.1.4

Performance updates

• aggregate method 2x faster for cases without values count

 $search->findAcceptableFilters()

• some optimisations of sorting method

PHPBench v2.1.4 ArrayIndex PHP 8.1.9 + JIT + opcache (no xdebug extension)

Items count	Memory	Find	Get Filters (aggregate)	Get Filters & Count (aggregate)	Sort by field	Results Found
10,000	~6Mb	~0.0004 s.	~0.001 s.	~0.002 s.	~0.0001 s.	907
50,000	~40Mb	~0.001 s.	~0.007 s.	~0.013 s.	~0.0005 s.	4550
100,000	~80Mb	~0.003 s.	~0.015 s.	~0.028 s.	~0.001 s.	8817
300,000	~189Mb	~0.012 s.	~0.057 s.	~0.097 s	~0.004 s.	26891
1,000,000	~657Mb	~0.047 s.	~0.233 s.	~0.385 s.	~0.017 s.	90520

PHPBench v2.1.4 FixedArrayIndex PHP 8.1.9 + JIT + opcache (no xdebug extension)

Items count	Memory	Find	Get Filters (aggregate)	Get Filters & Count (aggregate)	Sort by field	Results Found
10,000	~2Mb	~0.0007 s.	~0.002 s.	~0.005 s.	~0.0002 s.	907
50,000	~12Mb	~0.003 s.	~0.012 s.	~0.024 s.	~0.0009 s.	4550
100,000	~23Mb	~0.006 s.	~0.025 s.	~0.047 s.	~0.002 s.	8817
300,000	~70Mb	~0.019 s.	~0.083 s.	~0.149 s.	~0.006 s.	26891
1,000,000	~233Mb	~0.077 s.	~0.306 s.	~0.550 s.	~0.025 s.	90520