Scout's simplicity • Elasticsearch's power • Your rules
```php // Add a trait. Search your models. User::search('loves espressos'); ``` ```php // Except you're not stuck with basic text search. User::viaIndex() ->searchPhrase('loves espressos') ->where('status', 'active') ->where('logs.country', 'Norway') ->orderByDesc('created_at') ->paginate(10); ``` ```php // Geo queries. Fuzzy matching. Regex. Aggregations. On your Laravel models. User::viaIndex() ->whereGeoDistance('home.location', '5km', [40.7128, -74.0060]) ->orderByGeo('home.location', [40.7128, -74.0060]) ->get(); ``` Scout gives you a search box behind a black box. ElasticLens gives you a search engine you can open up. Every index is a real Eloquent model you own. You define the field mappings. You define the Elasticsearch schema. You see exactly what's indexed and how. No magic, no guessing, no driver abstraction sitting between you and your data. --- ## The Query Boundary `viaIndex()` is the gateway. Everything after it runs against the full Elasticsearch query builder. Results come back as your Laravel models. [See diagram in the online documentation] ```php User::viaIndex() // Cross into Elasticsearch ->searchTerm('max') // full-text search ->where('state', 'active') // familiar Eloquent syntax ->where('logs.country', 'Norway') // query embedded relations with dot notation ->orderByDesc('created_at') // ordering, pagination, aggregations ->paginate(10); // Returns: Paginator of User models ``` Everything between `viaIndex()` and `get()`/`paginate()` is the Elasticsearch query builder from [Laravel-Elasticsearch](https://elasticsearch.pdphilip.com). Full-text search, geo queries, fuzzy matching, regex, nested objects, aggregations, highlighting. All of it. [Full search capabilities ->](https://elasticlens.pdphilip.com/full-text-search) --- ## Embed Relationships Into Your Index Here's the magic. Flatten your relational data into Elasticsearch and search across everything with dot notation. [See diagram in the online documentation] Define what gets embedded in your index model: ```php class IndexedUser extends IndexModel { protected $baseModel = User::class; public function fieldMap(): IndexBuilder { return IndexBuilder::map(User::class, function (IndexField $field) { $field->text('name'); $field->text('email'); $field->embedsMany('profiles', Profile::class)->embedMap(function ($field) { $field->text('bio'); $field->array('tags'); }); $field->embedsBelongTo('company', Company::class)->embedMap(function ($field) { $field->text('name'); $field->text('industry'); }); $field->embedsMany('logs', UserLog::class, null, null, function ($query) { $query->orderBy('created_at', 'desc')->limit(10); })->embedMap(function ($field) { $field->text('action'); }); }); } } ``` Then search across all of it: ```php User::viaIndex()->where('profiles.bio', 'like', '%elasticsearch%')->get(); ``` Related models are observed too. Update a `Profile` and the parent `IndexedUser` rebuilds automatically. [Full Embedded Relations guide ->](https://elasticlens.pdphilip.com/embedded-relations) --- ## Features - [**Auto-sync**](https://elasticlens.pdphilip.com/getting-started) - Every create, update, and delete on your base model is reflected in the index. Automatically. - [**Embedded relations**](https://elasticlens.pdphilip.com/embedded-relations) - Flatten hasMany, hasOne, belongsTo into searchable nested objects - [**Full Elasticsearch query builder**](https://elasticlens.pdphilip.com/full-text-search) - Term, phrase, fuzzy, regex, geo, nested, aggregations, highlighting. All returning your Laravel models. - [**Index migrations**](https://elasticlens.pdphilip.com/index-model-migrations) - Define your Elasticsearch mapping with a Blueprint, just like database migrations - [**Conditional indexing**](https://elasticlens.pdphilip.com/conditional-indexing) - Control which records get indexed with `excludeIndex()` - [**Soft delete support**](https://elasticlens.pdphilip.com/soft-deletes) - Configure whether soft-deleted records keep their index - [**CLI tools**](https://elasticlens.pdphilip.com/artisan-cli-tools) - Status, health, build, migrate, and make commands --- ## Requirements | | Version | |---|---| | PHP | 8.2+ | | Laravel | 10 / 11 / 12 / 13 | | Elasticsearch | 8.x +| [ Get Started -> ](https://elasticlens.pdphilip.com/getting-started) --- ## Getting Started Trait on. Index built. Searching in under a minute. > **Note:** > **ElasticLens is not Laravel-Elasticsearch.** Laravel-Elasticsearch is Eloquent for Elasticsearch. ElasticLens *uses* it to create and sync a searchable index of your SQL models. ## Installation - Laravel 10/11/12 - Elasticsearch 8.x You'll need your Elasticsearch connection configured first. See the [Laravel-Elasticsearch Getting Started Guide](https://elasticsearch.pdphilip.com/getting-started/) if you haven't done that yet. ```bash composer require pdphilip/elasticlens ``` Publish the config file and create migration files: ```bash php artisan lens:install ``` Run the migrations: ```bash php artisan migrate ``` ## Optional Configuration `lens:install` publishes `config/elasticlens.php` and creates migrations for the build state and migration log indexes. ```php // config/elasticlens.php return [ 'database' => 'elasticsearch', 'queue' => null, // Set queue to use for dispatching index builds 'index_soft_deletes' => false, // See: Soft Delete Support 'watchers' => [ // \App\Models\Profile::class => [ // \App\Models\Indexes\IndexedUser::class, // ], ], 'index_build_state' => [ 'enabled' => true, 'log_trim' => 2, ], 'index_migration_logs' => [ 'enabled' => true, ], 'namespaces' => [ 'App\Models' => 'App\Models\Indexes', ], 'index_paths' => [ 'app/Models/Indexes/' => 'App\Models\Indexes', ], ]; ``` ## How It Works The **Index Model** is a separate Elasticsearch model that ElasticLens manages for you. It's still a real model you can query and inspect directly, just like any other Eloquent model. On top of that, ElasticLens gives you: - Full-text search on your **Base Models** using the entire [Laravel-Elasticsearch](https://elasticsearch.pdphilip.com) query builder - Field mapping with embedded relationships during builds - **Index Model** migrations for schema control - [Conditional indexing](https://elasticlens.pdphilip.com/conditional-indexing) to skip specific records - [Soft delete support](https://elasticlens.pdphilip.com/soft-deletes) to keep or remove index records on soft delete - CLI tools for status, health, builds, and migrations So a **User** model syncs with an **IndexedUser** model in Elasticsearch. Every search feature from Laravel-Elasticsearch is now available on your **User** models. --- ## What's New in V4 Three things: a cleaner search API, conditional indexing, and soft delete support. ## viaIndex Scope Flag The big one. `viaIndex()` now returns **Base Models** by default. In v3, `viaIndex()->get()` returned **Index Models** and you had to call `getBase()` to get your actual models back. That's backwards. In v4, `get()` and `paginate()` return base models when called through `viaIndex()`: ```php // v3 - extra step User::viaIndex()->searchTerm('david')->getBase(); // v4 - just works User::viaIndex()->searchTerm('david')->get(); ``` When you *do* want index models, ask for them explicitly: - `getIndex()` / `paginateIndex()` / `firstIndex()` - Returns **Index Models** - `getBase()` / `paginateBase()` / `firstBase()` - Still available, now just aliases for the default ## Conditional Indexing New `excludeIndex()` method on your **Base Model**. Skip records that don't belong in the index: ```php class User extends Model { use Indexable; public function excludeIndex(): bool { return $this->status === 'banned'; } } ``` Excluded records show up as **skipped** in build state and health checks. See [Conditional Indexing](https://elasticlens.pdphilip.com/conditional-indexing) for details. ## Soft Delete Support ElasticLens now handles soft deletes properly: - **Global config:** `index_soft_deletes` in `config/elasticlens.php` - **Per-model override:** `$indexSoftDeletes` property on your **Index Model** - **Restore detection:** Automatically re-indexes when a model is restored ```php // config/elasticlens.php 'index_soft_deletes' => true, // Keep index records when soft deleting // Or per-model class IndexedUser extends IndexModel { protected ?bool $indexSoftDeletes = true; } ``` See [Soft Delete Support](https://elasticlens.pdphilip.com/soft-deletes) for the full configuration guide. ## Errors Command New `lens:errors` command to inspect build failures for a specific model: ```bash php artisan lens:errors User ``` Shows each failed record with error message, details, source, and timestamp. Paginated with a prompt to view more. ## Bulk Migrate `lens:migrate` now uses the same bulk insert engine as `lens:build`. Previously it rebuilt records one at a time. Same result, significantly faster on large datasets. ## LensBuilder Under the hood, ElasticLens now uses a dedicated `LensBuilder` that extends the Elasticsearch Builder. This powers the viaIndex scope flag and gives the search API a cleaner architecture. `IndexModel::query()` returns a `LensBuilder` instance. --- ## Upgrading from V3 One breaking change. Two new opt-in features. Here's the full picture. ## Update Composer ```bash composer require pdphilip/elasticlens:^4.0 ``` ## Breaking Change: viaIndex() Return Type In v3, `viaIndex()->get()` returned **Index Models**. In v4, it returns **Base Models**. ### If you used `getBase()` / `paginateBase()` They still work. They're now aliases for the default behavior: ```php // v3 (still works in v4, just redundant now) User::viaIndex()->searchTerm('david')->getBase(); // v4 equivalent User::viaIndex()->searchTerm('david')->get(); ``` No action needed. ### If you used `get()` / `paginate()` directly If you relied on `viaIndex()->get()` returning index models, switch to the explicit methods: ```php // v3 - returned IndexedUser models User::viaIndex()->searchTerm('david')->get(); // v4 - use getIndex() for index models User::viaIndex()->searchTerm('david')->getIndex(); ``` Same for pagination: ```php // v3 - paginator of IndexedUser User::viaIndex()->searchTerm('david')->paginate(10); // v4 - use paginateIndex() User::viaIndex()->searchTerm('david')->paginateIndex(10); ``` ### If you used `asBase()` on collections Still works: ```php // v3 (still works) User::viaIndex()->searchTerm('david')->getIndex()->asBase(); // v4 - simpler User::viaIndex()->searchTerm('david')->get(); ``` ## New Features (Opt-in) ### Conditional Indexing Override `excludeIndex()` on your base model to skip records: ```php public function excludeIndex(): bool { return $this->status === 'banned'; } ``` ### Soft Delete Support Configure in `config/elasticlens.php`: ```php 'index_soft_deletes' => true, ``` Or per-model on your **Index Model**: ```php protected ?bool $indexSoftDeletes = true; ``` ### New Config Option The config file has a new `index_soft_deletes` key. Re-publish to get it: ```bash php artisan vendor:publish --tag=elasticlens-config ``` Or add it manually: ```php // config/elasticlens.php 'index_soft_deletes' => false, ``` --- --- ## Indexing your Base-Model Two steps. Trait on your **Base Model**, create your **Index Model**. Done. --- ## Setup ### 1. Add the `Indexable` Trait to Your Base-Model ```php //App\Models\Profile.php + use PDPhilip\ElasticLens\Indexable; class Profile extends Model { + use Indexable; ``` ### 2 (a) Create an Index-Model for your Base-Model ElasticLens expects the **Index Model** to be named `Indexed` + `BaseModelName` in the `App\Models\Indexes` directory. ```php //App\Models\Indexes\IndexedProfile.php namespace App\Models\Indexes; use PDPhilip\ElasticLens\IndexModel; class IndexedProfile extends IndexModel{} ``` ### 2 (b) Or create with artisan ```bash php artisan lens:make Profile ``` That's it. Your `Profile` model now syncs with `IndexedProfile` automatically on every create, update, and delete. Search it: ```php Profile::viaIndex()->searchTerm('running')->orSearchTerm('swimming')->get(); ``` ## Build Indexes for existing data ```bash php artisan lens:build Profile ``` Builds (or rebuilds) all `IndexedProfile` records from your existing `Profile` data. --- ## Index-model field mapping Define what gets indexed. Map your fields. Embed your relationships. Your index, your rules. ## Defining a field Map By default, the **Index Model** indexes every field it finds on the **Base Model** during sync. To be explicit about what gets indexed, define a `fieldMap()`: ```php + use PDPhilip\ElasticLens\Builder\IndexBuilder; + use PDPhilip\ElasticLens\Builder\IndexField; class IndexedUser extends IndexModel { protected $baseModel = User::class; + public function fieldMap(): IndexBuilder + { + return IndexBuilder::map(User::class, function (IndexField $field) { + $field->text('first_name'); + $field->text('last_name'); + $field->text('email'); + $field->bool('is_active'); //See attributes as fields + $field->type('state', UserState::class); //Maps enum + $field->text('created_at'); + $field->text('updated_at'); + }); + } ``` > **Note:** > When `fieldMap()` is defined, *only* the fields listed within it are indexed. > > The `id` is handled automatically: `$user->id` maps to `$indexedUser->id`. > > When mapping enums, cast them in the **Index Model** too. > > If a value isn't found during the build, it's stored as `null`. ## Attributes as fields If your **Base Model** has computed attributes you want searchable, map them like regular fields. > For example, `$field->bool('is_active')` could come from a custom attribute > on the **Base Model**: ```php //App\Models\User.php // @property-read bool is_active public function getIsActiveAttribute(): bool { return $this->updated_at >= Carbon::now()->modify('-30 days'); } ``` These values are stored in Elasticsearch as they were at sync time. They're snapshots, not live calculations. --- ## Relationships as embedded fields Here's the standout feature. Embed relationships into your **Index Model** and search across them as nested objects. No JOINs, no multiple queries. Consider these relationships around the `User` model: [See diagram in the online documentation] ### EmbedsMany `User` has many `Profiles`: ```php use PDPhilip\ElasticLens\Builder\IndexBuilder; use PDPhilip\ElasticLens\Builder\IndexField; class IndexedUser extends IndexModel { protected $baseModel = User::class; public function fieldMap(): IndexBuilder { return IndexBuilder::map(User::class, function (IndexField $field) { $field->text('first_name'); $field->text('last_name'); $field->text('email'); $field->bool('is_active'); $field->type('type', UserType::class); $field->type('state', UserState::class); $field->text('created_at'); $field->text('updated_at'); + $field->embedsMany('profiles', Profile::class)->embedMap(function (IndexField $field) { + $field->text('profile_name'); + $field->text('about'); + $field->array('profile_tags'); + }); }); } ``` ### EmbedsOne `Profile` has one `ProfileStatus`: ```php use PDPhilip\ElasticLens\Builder\IndexBuilder; use PDPhilip\ElasticLens\Builder\IndexField; class IndexedUser extends IndexModel { protected $baseModel = User::class; public function fieldMap(): IndexBuilder { return IndexBuilder::map(User::class, function (IndexField $field) { $field->text('first_name'); $field->text('last_name'); $field->text('email'); $field->bool('is_active'); $field->type('type', UserType::class); $field->type('state', UserState::class); $field->text('created_at'); $field->text('updated_at'); $field->embedsMany('profiles', Profile::class)->embedMap(function (IndexField $field) { $field->text('profile_name'); $field->text('about'); $field->array('profile_tags'); + $field->embedsOne('status', ProfileStatus::class)->embedMap(function (IndexField $field) { + $field->text('id'); + $field->text('status'); + }); }); }); } ``` ### EmbedsBelongTo `User` belongs to an `Account`: ```php use PDPhilip\ElasticLens\Builder\IndexBuilder; use PDPhilip\ElasticLens\Builder\IndexField; class IndexedUser extends IndexModel { protected $baseModel = User::class; public function fieldMap(): IndexBuilder { return IndexBuilder::map(User::class, function (IndexField $field) { $field->text('first_name'); $field->text('last_name'); $field->text('email'); $field->bool('is_active'); $field->type('type', UserType::class); $field->type('state', UserState::class); $field->text('created_at'); $field->text('updated_at'); $field->embedsMany('profiles', Profile::class)->embedMap(function (IndexField $field) { $field->text('profile_name'); $field->text('about'); $field->array('profile_tags'); $field->embedsOne('status', ProfileStatus::class)->embedMap(function (IndexField $field) { $field->text('id'); $field->text('status'); }); }); + $field->embedsBelongTo('account', Account::class)->embedMap(function (IndexField $field) { + $field->text('name'); + $field->text('url'); + }); }); } ``` ### Embedding without observing `User` belongs to `Country`, but countries don't change often enough to justify observing: ```php use PDPhilip\ElasticLens\Builder\IndexBuilder; use PDPhilip\ElasticLens\Builder\IndexField; class IndexedUser extends IndexModel { protected $baseModel = User::class; public function fieldMap(): IndexBuilder { return IndexBuilder::map(User::class, function (IndexField $field) { $field->text('first_name'); $field->text('last_name'); $field->text('email'); $field->bool('is_active'); $field->type('type', UserType::class); $field->type('state', UserState::class); $field->text('created_at'); $field->text('updated_at'); $field->embedsMany('profiles', Profile::class)->embedMap(function (IndexField $field) { $field->text('profile_name'); $field->text('about'); $field->array('profile_tags'); $field->embedsOne('status', ProfileStatus::class)->embedMap(function (IndexField $field) { $field->text('id'); $field->text('status'); }); }); $field->embedsBelongTo('account', Account::class)->embedMap(function (IndexField $field) { $field->text('name'); $field->text('url'); }); + $field->embedsBelongTo('country', Country::class)->embedMap(function (IndexField $field) { + $field->text('country_code'); + $field->text('name'); + $field->text('currency'); + })->dontObserve(); // Don't observe changes in the country model }); } ``` ### EmbedsMany with a query `User` has many `UserLogs` and you only want the last 10: ```php use PDPhilip\ElasticLens\Builder\IndexBuilder; use PDPhilip\ElasticLens\Builder\IndexField; class IndexedUser extends IndexModel { protected $baseModel = User::class; public function fieldMap(): IndexBuilder { return IndexBuilder::map(User::class, function (IndexField $field) { $field->text('first_name'); $field->text('last_name'); $field->text('email'); $field->bool('is_active'); $field->type('type', UserType::class); $field->type('state', UserState::class); $field->text('created_at'); $field->text('updated_at'); $field->embedsMany('profiles', Profile::class)->embedMap(function (IndexField $field) { $field->text('profile_name'); $field->text('about'); $field->array('profile_tags'); $field->embedsOne('status', ProfileStatus::class)->embedMap(function (IndexField $field) { $field->text('id'); $field->text('status'); }); }); $field->embedsBelongTo('account', Account::class)->embedMap(function (IndexField $field) { $field->text('name'); $field->text('url'); }); $field->embedsBelongTo('country', Country::class)->embedMap(function (IndexField $field) { $field->text('country_code'); $field->text('name'); $field->text('currency'); })->dontObserve(); // Don't observe changes in the country model + $field->embedsMany('logs', UserLog::class, null, null, function ($query) { + $query->orderBy('created_at', 'desc')->limit(10); // Limit the logs to the 10 most recent + })->embedMap(function (IndexField $field) { + $field->text('title'); + $field->text('ip'); + $field->array('log_data'); + }); }); } ``` ## `IndexField` Methods ### Field types: - `text($field)` - `integer($field)` - `array($field)` - `bool($field)` - `type($field, $type)` - Set own type (like Enums) - `embedsMany($field, $relatedModelClass, $whereRelatedField, $equalsLocalField, $query)` - `embedsBelongTo($field, $relatedModelClass, $whereRelatedField, $equalsLocalField, $query)` - `embedsOne($field, $relatedModelClass, $whereRelatedField, $equalsLocalField, $query)` > **Note:** > For embeds, `$whereRelatedField`, `$equalsLocalField`, and `$query` are all optional. > > `$whereRelatedField` is the foreign key, `$equalsLocalField` is the local key. Both are inferred from the relationship if not provided. > > `$query` is a closure for customizing the related model query. ### Embedded field type methods: - `embedMap(function (IndexField $field) {})` - Define the mapping for the embedded relationship - `dontObserve()` - Don't observe changes in the related model --- --- ## Index-model migrations Elasticsearch auto-indexes new fields, but it doesn't always guess right. Take control with `migrationMap()`. ## Define the `migrationMap()` Elasticsearch indexes fields automatically, but it may not type them the way you need. A field you want as `keyword` might get mapped as `text`, or vice versa. The `migrationMap()` gives you explicit control. Since the **Index Model** is built on Laravel-Elasticsearch, use the [Index Blueprint](https://elasticsearch.pdphilip.com/schema/index-blueprint/) to define your schema: ```php use PDPhilip\Elasticsearch\Schema\Blueprint; class IndexedUser extends IndexModel { //...... public function migrationMap(): callable { return function (Blueprint $index) { $index->text('name'); $index->keyword('first_name'); $index->text('first_name'); $index->keyword('last_name'); $index->text('last_name'); $index->keyword('email'); $index->text('email'); $index->text('avatar')->indexField(false); $index->keyword('type'); $index->text('type'); $index->keyword('state'); $index->text('state'); //...etc }; } ``` ## Run the Migration ```bash php artisan lens:migrate User ``` This deletes the existing index, runs the migration, and bulk rebuilds all records. --- --- ## Embedded Relations The main event. Embed your relationships into Elasticsearch and search across everything at once. ## The Problem You've got a User model with data spread across six tables. You want to search "find all active users who work in tech companies and have a profile mentioning elasticsearch." In SQL, that's JOINs across three tables, WHERE clauses stacked up, and still no full-text search. [See diagram in the online documentation] ElasticLens flattens everything into a single searchable document at index time. Six tables become one query. --- ## Building the fieldMap Start with the **Index Model** and define what gets indexed. We'll build it up step by step. ### Base fields ```php use PDPhilip\ElasticLens\Builder\IndexBuilder; use PDPhilip\ElasticLens\Builder\IndexField; class IndexedUser extends IndexModel { protected $baseModel = User::class; public function fieldMap(): IndexBuilder { return IndexBuilder::map(User::class, function (IndexField $field) { $field->text('first_name'); $field->text('last_name'); $field->text('email'); $field->type('state', UserState::class); }); } } ``` This indexes the User's own fields. Now let's embed the relationships. ### embedsMany - User has many Profiles ```php return IndexBuilder::map(User::class, function (IndexField $field) { $field->text('first_name'); $field->text('last_name'); $field->text('email'); $field->type('state', UserState::class); + $field->embedsMany('profiles', Profile::class)->embedMap(function (IndexField $field) { + $field->text('profile_name'); + $field->text('about'); + $field->array('tags'); + }); }); ``` Each User's profiles are now embedded as nested objects in Elasticsearch. The `Profile` model is automatically observed: update a profile and the parent IndexedUser rebuilds. ### embedsOne - nested within an embed Profile has one ProfileStatus. Embeds nest inside embeds: ```php $field->embedsMany('profiles', Profile::class)->embedMap(function (IndexField $field) { $field->text('profile_name'); $field->text('about'); $field->array('tags'); + $field->embedsOne('status', ProfileStatus::class)->embedMap(function (IndexField $field) { + $field->text('status'); + }); }); ``` When a `ProfileStatus` changes, ElasticLens traces the chain: ProfileStatus -> Profile -> User -> rebuilds IndexedUser. ### embedsBelongTo - User belongs to Account ```php return IndexBuilder::map(User::class, function (IndexField $field) { $field->text('first_name'); $field->text('last_name'); $field->text('email'); $field->type('state', UserState::class); $field->embedsMany('profiles', Profile::class)->embedMap(function (IndexField $field) { $field->text('profile_name'); $field->text('about'); $field->array('tags'); $field->embedsOne('status', ProfileStatus::class)->embedMap(function (IndexField $field) { $field->text('status'); }); }); + $field->embedsBelongTo('account', Account::class)->embedMap(function (IndexField $field) { + $field->text('name'); + $field->text('industry'); + }); }); ``` ### dontObserve - static reference data For data that rarely changes (like countries), skip the observer. No point rebuilding indexes every time someone corrects a currency symbol: ```php + $field->embedsBelongTo('country', Country::class)->embedMap(function (IndexField $field) { + $field->text('name'); + $field->text('currency'); + })->dontObserve(); ``` ### Query-limited embeds Embed only the 10 most recent logs instead of all of them: ```php + $field->embedsMany('logs', UserLog::class, null, null, function ($query) { + $query->orderBy('created_at', 'desc')->limit(10); + })->embedMap(function (IndexField $field) { + $field->text('action'); + $field->text('ip'); + }); ``` --- ## The Complete fieldMap ```php class IndexedUser extends IndexModel { protected $baseModel = User::class; public function fieldMap(): IndexBuilder { return IndexBuilder::map(User::class, function (IndexField $field) { $field->text('first_name'); $field->text('last_name'); $field->text('email'); $field->type('state', UserState::class); $field->embedsMany('profiles', Profile::class)->embedMap(function (IndexField $field) { $field->text('profile_name'); $field->text('about'); $field->array('tags'); $field->embedsOne('status', ProfileStatus::class)->embedMap(function (IndexField $field) { $field->text('status'); }); }); $field->embedsBelongTo('account', Account::class)->embedMap(function (IndexField $field) { $field->text('name'); $field->text('industry'); }); $field->embedsBelongTo('country', Country::class)->embedMap(function (IndexField $field) { $field->text('name'); $field->text('currency'); })->dontObserve(); $field->embedsMany('logs', UserLog::class, null, null, function ($query) { $query->orderBy('created_at', 'desc')->limit(10); })->embedMap(function (IndexField $field) { $field->text('action'); $field->text('ip'); }); }); } } ``` --- ## What Gets Stored When User #1 is indexed, ElasticLens builds a single Elasticsearch document: ```json { "id": 1, "first_name": "Max", "last_name": "Philip", "email": "max@example.com", "state": "active", "profiles": [ { "profile_name": "dev", "about": "Loves Elasticsearch and Laravel", "tags": ["php", "laravel", "elasticsearch"], "status": { "status": "available" } }, { "profile_name": "music", "about": "Jazz piano and vinyl collecting", "tags": ["jazz", "piano"], "status": { "status": "active" } } ], "account": { "name": "Acme Inc", "industry": "Technology" }, "country": { "name": "South Africa", "currency": "ZAR" }, "logs": [ { "action": "login", "ip": "127.0.0.1" }, { "action": "profile_update", "ip": "127.0.0.1" } ] } ``` Six SQL tables. Zero JOINs. One searchable document. --- ## Searching Embedded Fields Now search across everything. Embedded fields use dot notation, just like you'd expect: ### Dot-notation queries ```php // Users with profiles mentioning "elasticsearch" User::viaIndex()->where('profiles.about', 'like', '%elasticsearch%')->get(); ``` ### Combine with other queries ```php // Active users at tech companies with recent login activity User::viaIndex() ->where('state', 'active') ->where('account.industry', 'Technology') ->where('logs.action', 'login') ->where('logs.ip', '!=', '127.0.0.1') ->get(); ``` ### Full-text search across embedded data ```php // Searches ALL fields, base and embedded User::viaIndex()->searchTerm('elasticsearch')->get(); ``` `searchTerm` searches across every indexed field, including embedded ones. A match in `profiles.about` surfaces the parent User. > **Note:** > If you've defined explicit `nested` mappings in your `migrationMap()`, use `whereNestedObject()` instead. See [Nested Queries](https://elasticsearch.pdphilip.com/eloquent/nested-queries/) for when and why you'd want that. --- ## The Observer Chain When embedded models change, ElasticLens traces back to the base model and rebuilds automatically. [See diagram in the online documentation] No manual intervention needed: - **Save a Profile** -> IndexedUser for that profile's user rebuilds - **Delete a UserLog** -> IndexedUser for that log's user rebuilds - **Update a ProfileStatus** -> traces through Profile -> User -> rebuilds Models marked with `dontObserve()` (like Country in our example) skip this chain. Changes to country data require a manual `lens:build`. --- ## Advanced Patterns ### Independent watchers with `HasWatcher` If you want to observe a related model independently (outside the fieldMap chain), use the `HasWatcher` trait: ```php // App\Models\ProfileStatus.php use PDPhilip\ElasticLens\HasWatcher; class ProfileStatus extends Eloquent { use HasWatcher; } ``` Then register it in `config/elasticlens.php`: ```php 'watchers' => [ \App\Models\ProfileStatus::class => [ \App\Models\Indexes\IndexedUser::class, ], ], ``` See [Model Observers](https://elasticlens.pdphilip.com/model-observers) for the full configuration reference. ### Index migrations for embedded fields When embedding relationships, define nested mappings in your `migrationMap()`: ```php public function migrationMap(): callable { return function (Blueprint $index) { $index->text('first_name'); $index->keyword('first_name'); $index->text('email'); $index->keyword('state'); $index->nested('profiles'); $index->nested('logs'); }; } ``` See [Index Migrations](https://elasticlens.pdphilip.com/index-model-migrations) for the full migration reference. --- ## Summary | Embed Type | Method | Observes by default | |---|---|---| | Has Many | `embedsMany()` | Yes | | Has One | `embedsOne()` | Yes | | Belongs To | `embedsBelongTo()` | Yes | | Any + skip observer | `.dontObserve()` | No | | Any + query constraint | 5th parameter closure | Yes | Full API reference: [Field Mapping](https://elasticlens.pdphilip.com/field-mapping). --- ## Full-text Search Not a search driver. The full Elasticsearch query builder, returning your Laravel models. `viaIndex()` gives you access to every query method from [Laravel-Elasticsearch](https://elasticsearch.pdphilip.com). Full-text search, geo queries, fuzzy matching, regex, nested objects, aggregations, highlighting. The works. ## Quick Search The `search()` convenience method runs a phrase prefix search across all fields: ```php User::search('loves espressos'); ``` > Returns a Collection of `User` models. Good for search bars and quick lookups. ## viaIndex() For everything else, `viaIndex()` drops you into the Elasticsearch query builder: ```php User::viaIndex()->searchTerm('suvi')->get(); // Collection of User models User::viaIndex()->where('state', 'active')->paginate(); // Paginator of User models User::viaIndex()->searchPhrase('ice bathing')->first(); // User model or null ``` Results come back as your **Base Models** by default. You can request **Index Models** directly when you need them: | Method | Returns | |---|---| | `get()` / `paginate()` / `first()` | Base models (User) | | `getBase()` / `paginateBase()` / `firstBase()` | Base models (explicit) | | `getIndex()` / `paginateIndex()` / `firstIndex()` | Index models (IndexedUser) | Use `getIndex()` when you need access to embedded fields or search highlights on the index model itself. Convert back anytime with `->asBase()`. --- ## Full-text Search Multi-match queries across all indexed fields. ```php // Term search - finds "nara" across all fields User::viaIndex()->searchTerm('nara')->get(); // Phrase search - exact word sequence User::viaIndex()->searchPhrase('ice bathing')->get(); // Fuzzy search - handles typos User::viaIndex()->searchFuzzy('quikc brwn foks')->get(); // Phrase prefix - autocomplete-style matching User::viaIndex()->searchPhrasePrefix('loves espr')->get(); ``` Boost specific fields to control relevance: ```php User::viaIndex()->searchTerm('suvi', ['first_name^3', 'last_name^2', 'bio'])->first(); ``` Also available: `searchBoolPrefix()`, `searchTermMost()`, `searchTermCross()`, `searchQueryString()` [Full reference: Search Queries ->](https://elasticsearch.pdphilip.com/eloquent/search-queries/) --- ## Field-level Queries Target specific fields with Elasticsearch-native query types. ```php // Regex on a single field User::viaIndex()->whereRegex('favorite_color', 'bl(ue)?(ack)?')->get(); // Fuzzy match on a single field User::viaIndex()->whereFuzzy('last_name', 'phillp')->get(); // Match with operator control User::viaIndex()->whereMatch('bio', 'laravel elasticsearch', 'and')->get(); ``` Also available: `whereTerm()`, `whereExact()`, `wherePrefix()`, `wherePhrase()`, `wherePhrasePrefix()`, `whereScript()` [Full reference: Elasticsearch Queries ->](https://elasticsearch.pdphilip.com/eloquent/es-queries/) --- ## Eloquent Queries All the familiar Laravel query methods. They work exactly as you'd expect. ```php User::viaIndex()->where('state', 'active')->get(); User::viaIndex()->where('orders', '>=', 10)->get(); User::viaIndex()->whereIn('state', ['active', 'pending'])->get(); User::viaIndex()->whereBetween('created_at', [$start, $end])->get(); User::viaIndex()->whereNull('deleted_at')->get(); User::viaIndex()->where('name', 'like', '%max%')->get(); ``` Chain them with search methods for filtered searches: ```php User::viaIndex() ->searchTerm('elasticsearch') ->where('state', 'active') ->whereBetween('created_at', [now()->subYear(), now()]) ->orderByDesc('created_at') ->paginate(10); ``` [Full reference: Eloquent Queries ->](https://elasticsearch.pdphilip.com/eloquent/eloquent-queries/) --- ## Geo Queries Filter and sort by geographic distance. ```php // Find users within 5km of a point User::viaIndex() ->filterGeoPoint('home.location', '5km', [40.7128, -74.0060]) ->orderByGeo('home.location', [40.7128, -74.0060]) ->get(); // Bounding box filter User::viaIndex() ->filterGeoBox('home.location', [41.0, -74.5], [40.5, -73.5]) ->get(); ``` [Full reference: Geo Queries ->](https://elasticsearch.pdphilip.com/eloquent/es-queries/#geo-queries) --- ## Embedded Field Queries Query [embedded relations](https://elasticlens.pdphilip.com/embedded-relations) with dot notation. No special syntax needed: ```php // Users with profiles mentioning "elasticsearch" User::viaIndex()->where('profiles.about', 'like', '%elasticsearch%')->get(); // Combine embedded fields with other filters User::viaIndex() ->where('state', 'active') ->where('logs.action', 'login') ->where('logs.created_at', '>=', now()->subWeek()) ->get(); ``` If you've set up explicit `nested` mappings in your `migrationMap()`, use `whereNestedObject()` for [nested queries](https://elasticsearch.pdphilip.com/eloquent/nested-queries/). --- ## Aggregations Run analytics directly on your indexed data. ```php User::viaIndex()->where('state', 'active')->avg('orders'); User::viaIndex()->where('state', 'active')->sum('revenue'); User::viaIndex()->min('created_at'); User::viaIndex()->max('last_login'); // Distinct values User::viaIndex()->distinct('state'); // Group by with aggregations User::viaIndex()->groupBy('state')->get(); ``` Also available: `stats()`, `percentiles()`, `cardinality()`, `extendedStats()`, `matrix()`, `boxplot()`, `medianAbsoluteDeviation()` [Full reference: Aggregations ->](https://elasticsearch.pdphilip.com/eloquent/aggregation/) --- ## Highlighting Get search highlights showing exactly where terms matched. Use `getIndex()` to access highlights on the index model. ```php $results = User::viaIndex() ->searchTerm('espresso') ->withHighlights() ->getIndex(); foreach ($results as $indexedUser) { $indexedUser->getHighlights(); // ['bio' => ['Loves espresso and jazz']] } ``` Custom highlight tags: ```php User::viaIndex() ->searchTerm('espresso') ->withHighlights('', '') ->getIndex(); ``` --- ## Full Reference Every query method available through `viaIndex()` is documented in the Laravel-Elasticsearch package: - [**Search Queries**](https://elasticsearch.pdphilip.com/eloquent/search-queries/) - searchTerm, searchPhrase, searchFuzzy, searchPhrasePrefix, searchBoolPrefix, searchQueryString - [**Elasticsearch Queries**](https://elasticsearch.pdphilip.com/eloquent/es-queries/) - whereMatch, whereRegex, whereFuzzy, whereTerm, wherePrefix, wherePhrase, geo queries - [**Eloquent Queries**](https://elasticsearch.pdphilip.com/eloquent/eloquent-queries/) - where, whereIn, whereBetween, whereNull, whereDate, like, orWhere - [**Nested Queries**](https://elasticsearch.pdphilip.com/eloquent/nested-queries/) - whereNestedObject, whereNotNestedObject, filterNested, orderByNested - [**Aggregations**](https://elasticsearch.pdphilip.com/eloquent/aggregation/) - avg, sum, min, max, stats, percentiles, distinct, groupBy, cardinality - [**Ordering & Pagination**](https://elasticsearch.pdphilip.com/eloquent/ordering-and-pagination/) - orderBy, orderByGeo, orderByNested, paginate, skip, take --- ## Base Model Observers Your **Base Model** is observed for saves and deletions out of the box. When the base model is deleted, its **Index Model** goes with it. > **Note:** > Soft delete behavior is configurable. See [Soft Delete Support](https://elasticlens.pdphilip.com/soft-deletes) for details. ### Handling Embedded Models When your `fieldMap()` includes embedded fields, the related models are observed too. > A save or delete on `ProfileStatus` triggers a chain: find the related `Profile`, then `User`, then rebuild that user's index. > One catch: embedded model observers are only loaded when the base model class has been referenced. You need to touch the User model first: ```php //This alone will not trigger a rebuild $profileStatus->status = 'Unavailable'; $profileStatus->save(); //This will new User::class $profileStatus->status = 'Unavailable'; $profileStatus->save(); ``` ### `HasWatcher` trait If you want ElasticLens to observe an embedded model independently (without needing to reference the base model first), use `HasWatcher`: ### 1. Add the `HasWatcher` Trait to the Embedded Model: ```php //App\Models\ProfileStatus.php +use PDPhilip\ElasticLens\HasWatcher; class ProfileStatus extends Eloquent { + use HasWatcher; ``` ### 2. Register the Watcher in Config: ```php // config/elasticlens.php 'watchers' => [ + \App\Models\ProfileStatus::class => [ + \App\Models\Indexes\IndexedUser::class, + ], +], ``` The watcher maps the observed model to the index model that should rebuild when it changes. ### Disabling Base Model Observation If you don't want the **Base Model** observed at all (you'll handle syncing manually), disable it: ```php class IndexedUser extends IndexModel { protected $baseModel = User::class; + protected $observeBase = false; ``` --- --- ## Conditional Indexing Not everything deserves an index entry. Tell ElasticLens what to skip. ## The `excludeIndex()` Method By default, every record gets indexed. Override `excludeIndex()` on your **Base Model** to change that: ```php class User extends Model { use Indexable; public function excludeIndex(): bool { return $this->status === 'banned'; } } ``` Return `true` and the record is skipped. This applies to both bulk builds (`lens:build`) and automatic syncs via observers. ## How It Works During builds, ElasticLens calls `excludeIndex()` on each record. When it returns `true`: - The record is skipped, no index entry created - Build state is marked as **skipped** (not failed) - If the record was previously indexed, the stale entry remains until the next full build ## Cleaning Up Stale Records When you add or change `excludeIndex()` conditions, previously indexed records that now qualify for exclusion still have entries. Run a full rebuild: ```bash php artisan lens:build User ``` The build skips excluded records and removes stale entries for records that no longer qualify. ## Health Check Accounting `lens:health` accounts for exclusions. Skipped records are tracked separately from errors, so your health report stays accurate. ```bash php artisan lens:health User ``` --- --- ## Soft Delete Support Soft delete a model. Keep the index, or don't. Your call. ## Default Behavior When a **Base Model** is soft-deleted, its **Index Model** record is **deleted**. Soft-deleted records aren't searchable by default. ## Keeping Index Records on Soft Delete Want soft-deleted records to stay in the index (with `deleted_at` synced)? Enable `index_soft_deletes`: ### Global Configuration ```php // config/elasticlens.php return [ 'index_soft_deletes' => true, // ... ]; ``` When enabled, soft-deleting a model triggers an index **rebuild** instead of a deletion, syncing `deleted_at` to the index. ### Per-Model Override Override the global setting on individual **Index Models**: ```php class IndexedUser extends IndexModel { protected $baseModel = User::class; // Override global config for this index protected ?bool $indexSoftDeletes = true; } ``` Per-model takes priority over global: - `$indexSoftDeletes = true` - Always keep index records on soft delete - `$indexSoftDeletes = false` - Always remove index records on soft delete - `$indexSoftDeletes = null` (default) - Fall back to global config ## Restore Behavior When a soft-deleted model is restored, ElasticLens triggers an index rebuild regardless of config. The index is always current after a restore. ## Observer Flow The observer handles soft deletes through three events: - **`deleting`** - If soft delete with index retention: skips deletion, lets `deleted` handle it. Otherwise deletes the index record. - **`deleted`** - If the model is trashed and `shouldIndexSoftDeletes()` is true: triggers a rebuild to sync `deleted_at`. - **`restored`** - Always triggers a rebuild to sync the restored state. --- --- ## Artisan CLI Tools Six commands. Everything you need to manage and monitor your indexes from the terminal. ## 1. Status Command See the state of all your indexes at a glance: ```bash php artisan lens:status ``` ## 2. Health Command Drill into a specific index: ```bash php artisan lens:health User ``` ## 3. Migrate Command Delete the existing index, run the migration, and rebuild all records: ```bash php artisan lens:migrate User ``` ## 4. Make Command Generate a new **Index Model**: ```bash php artisan lens:make Profile ``` ## 5. Build Command Bulk build (or rebuild) an index: ```bash php artisan lens:build Profile ``` ## 6. Errors Command View build errors for a specific model, paginated with full error details: ```bash php artisan lens:errors User php artisan lens:errors User --per-page=20 ``` Shows each failed build record with the error message, details, source, and timestamp. Results are paginated with a prompt to view the next batch. --- --- ## Build and Migration States ElasticLens tracks the state of every build and migration in Elasticsearch. You can query these models directly. ## `IndexableBuild` Model Tracks the state of each index build, giving you visibility into the sync process.