YetiSearch Pro

v2.3.1

2 days ago

1. • The simple modal is the default everywhere. The shipped yetisearch-pro.yaml has used it since 1.0.3, but the plugin settings form, the code's fallback when no type is set and the README still said full. A site that sets modal.type keeps its choice
   • The plugin's description, shown in GPM and the admin's plugin list, mentions stemming instead of facets, which YetiSearch Pro does not offer

v2.3.0

4 days ago

1. • Geo search. Give a page a location with geo: { lat: 51.5074, lng: -0.1278 } in its frontmatter, or with the new Location group on the YetiSearch tab of the page editor (Admin Next and the classic admin), and search by distance. A page gets a location only when both numbers are valid (latitude -90 to 90, longitude -180 to 180), never 0,0 from a missing value. PDFs and chunks are found at their page's location, and changing a location is picked up by the next index and by saving the page. It works without SQLite's R-Tree module
   • The /ys endpoint, the Twig yetisearch() function and the onYetisearchProBeforeSearch event's options take lat and lng (both needed), radius (without it every page with a location is returned), units (km, mi or m) and sort=distance. With a location, an empty query lists the nearest pages (as if sort=distance were given, on the endpoint, in Twig and in the CLI). That rule is applied after the onYetisearchProBeforeSearch event, so a listener can supply a location for an empty query, and an empty query with no valid location after the event returns an empty result. Invalid coordinates are ignored and the search runs as a normal one. Every hit has distance (meters) and distance_text (for example 1.3 km or 0.8 mi, translatable), and the response has a geo entry with the location used
   • A Near me button for the full modal, off by default. Turn it on with modal.near_me: true: the visitor's browser is asked for their location, the modal lists nearby results (also with an empty search box), offers a radius menu (modal.near_me_radii, plus the starting modal.near_me_radius when it is not in the list), and says so when the location is denied or unavailable. The coordinates stay in the browser and are sent only with the search request. Both modals and the built-in search page show each result's distance
   • Analytics never store a visitor's coordinates. A recent search only records whether it used a location
   • New settings: indexes.pages.geo (lat and lng, dot paths into the page header, default geo.lat and geo.lng) replaces geo_fields, which nothing read, so a site that already keeps coordinates under other keys can point at them; indexes.pages.query_defaults.geo.units (default km) sets the units when a request names none. All of them, and the Near me options, are in the plugin settings form
   • bin/plugin yetisearch-pro query takes --lat, --lng, --radius, --units and --sort distance, and shows a distance column. The query can be left out when there is a location
   • To set a page's location from code, an onYetisearchBuildDocument listener must assign the changed document back with $event['doc'] = $doc; editing $event['doc']['geo'] in place is not kept. The README has a tested example
   • Stemming, per index and off by default. With indexes.pages.stemming: true, or Stemming (Pages) in the plugin settings, a search also matches other forms of a word: runs finds "running", configured finds "configuration", utilisateur finds "utilisateurs". A per-language index stems in its own language. A single index stems each page in its own language, and a search in the language it names, else the site's default language. English, French, German and Spanish are supported. Matches on the word as typed rank first, search.stem_weight (default 0.5) sets how much a stem match counts, and highlights include stem matches. It costs indexing time and index size: on the Helios demo the database was about a third bigger, and the library measured indexing about 2.5 times slower and the database about 50% larger on 32,000 documents. The README's new Stemming section has the details
   • Changing the stemming setting switches each existing index with one rebuild of its full-text index from the documents it holds, without indexing the pages again. It happens when the plugin settings are saved in either admin, with the settings the site runs with after the save (an environment's override included), or else at the next index run, page save or page delete. Saving another environment's settings file leaves this environment's indexes alone; that environment switches at its own next index run or page save. While any GRAV_CONFIG__ environment variable is set, whichever settings it is for, a save leaves the indexes alone too, and the next index run, page save or page delete switches them with the settings the site runs with. bin/plugin yetisearch-pro index prints a line for each index it rebuilt, and the Grav log records it
2. • Updated the bundled YetiSearch library to 2.6.1. With 2.5.x, deleting a page that has chunks or PDFs, or saving one that is left out of search (an unpublished page, for one), added the field names of the stored documents (title, route, content, url, ...) to the index as words, so a search for title or route returned every document until the next page save rebuilt the index. YetiSearch Pro rebuilds the full-text index of each existing index once after the upgrade, before anything is written to it or deleted from it: at the first index run, page save, page delete, PDF change or settings save, and records it in the index so it never runs again. This matters, as with 2.6.0 a delete from an index still holding those words breaks every later search for one of them (2.6.1 also rebuilds such an index itself on its first write). If the rebuild fails, the Grav log says so and nothing is written to that index until a later request rebuilds it. On the Helios demo sites it took a hundredth of a second per index, and nothing needs to be reindexed by hand. A page indexed again with less content no longer keeps the chunks of its longer version, which were still found by words the page no longer has. Stop words now follow the language code, so a French search leaves out French stop words rather than English ones, and a language with no list, such as Italian, leaves out none. From the 2.5.x releases since 2.2.2: a search never returns results from before a write made in the same process, geo search works on PHP builds whose SQLite has no R-Tree module, with exact radius searches, totals and paging, and a page whose location is removed drops out of location searches the next time it is indexed
   • An index run reported success when an index failed: bin/plugin yetisearch-pro index, the reindex in either admin, the API and the scheduler all said the run had completed. They now report the failure, naming the index and the reason: the command exits with status 1, the admins show the error, and the result's success is false. A stale document (one whose page is gone) that a run could not remove, for instance while its index waits for the one-time cleanup, was dropped from the saved index state anyway, so it stayed searchable and no later run would remove it; it now stays recorded, the next run removes it, and only documents actually removed are counted. The same goes for a PDF whose old pages could not be cleared before it was indexed again. A page save that could not clear the page's old chunks or PDF pages saved the new fingerprints anyway, so the next run skipped the page and its PDFs as unchanged and the old chunks and PDF pages stayed searchable; the page and its PDFs are now recorded as not yet indexed, and the next run indexes them again and removes their old chunks and PDF pages first
   • The built-in search page (built_in_search_page: true) works. Its template called the Twig functions yetisearch() and yetisearch_form(), which were never registered, so the page failed to render. Both exist now, and yetisearch(query, options) runs the same search as the /ys endpoint, so language handling, the onYetisearchProBeforeSearch event and analytics apply to it too. The template reads the fields hits really have, takes q and page from the query string, shows the search time in seconds, and escapes result text while keeping the <mark> highlights
   • The Type filter in the admin Index Browser now applies when you search by text. It was only applied when browsing without a query
   • Search results are no longer able to inject markup. The library's highlights escape nothing and titles are stored as typed, so a page titled <img src=x onerror=...> came back with live markup in the full modal's preview, its breadcrumb line, the simple modal and the built-in search page. Every place that outputs a highlight now turns each < that does not start <mark> or </mark> into <, and leaves page text's own entities alone, so code samples in your pages still read as code
   • The built-in search page prints the search text and the "did you mean" suggestion escaped, so it is safe with Twig autoescaping off, and a result's link is only used when it is a relative path or an http(s) URL: a custom document with url: javascript:... gets #
   • Paged results sorted by distance keep their order with unique_by_route on (the default), and unique_by_route=0, semantic=0 and fuzzy=0 in a request are no longer read as absent (0 is falsy, so unique_by_route=0 left de-duplication on). fuzzy=true and fuzzy=false work too.
   • Editing a document in the admin Index Browser keeps its location and its type, and the classic admin editor's save works again: it sent the document without the content object the update requires, so every save failed with "Document payload must include a content object"
   • The full modal's radius menu always contains the starting radius, sorted and without duplicates. With near_me_radius: 17 and the default list, nothing was selected and the browser picked "Any distance"
   • The search modals send their requests to your configured query_route instead of always using /ys (the site's base URL is still included for sites in a subfolder). Themes that build their own modal still need to read query_route themselves
   • A search request with an array q (?q[]=x) no longer fails with a server error on the search page, the /ys endpoint or any page; it counts as no query
   • A request with ?q= on an ordinary page (/blog?q=anything, a tracking link, a theme's own search form, a bot) no longer runs a YetiSearch search whose results nothing read. The search now runs only on the query_route page (the /ys result without ajax or a JSON header) and on a page of the site's own at search_route (one that reads yetisearch_results); the built-in search page calls yetisearch() instead. yetisearch_query is still set everywhere
   • The query_route page without ajax or a JSON header (opening /ys?q=coffee in a browser) returned its JSON HTML-escaped (" for every quote) with Twig autoescaping on, which is the default, so it could not be parsed. It is valid JSON now, and without a query it returns the empty result envelope instead of []
   • The full modal marked a search word twice in a title (<mark><mark>coffee</mark></mark>) and could match inside markup or an entity: a search for mark broke the highlight tag and showed it as text, and a search for 39 matched inside Tom's. Titles and breadcrumb lines are now highlighted in one pass over the plain text, using the search words and the words the search engine highlighted, and escaped afterwards
3. • Saving a page no longer rebuilds the whole full-text index afterwards. YetiSearch 2.6.0 keeps the index's words in step on every write and delete, so a page save no longer takes longer as the index grows
   • Removed settings that nothing read: Indexed Fields (Pages) (indexes.pages.fields), indexes.pages.facets, indexes.pages.options.typo_tolerance, and the README's claim that options.search can be set per index. They are gone from the plugin settings form, the default yetisearch-pro.yaml and the README. Values already saved in your own config are ignored, as they were before. Search settings come from engine.search and indexer settings can still be set per index with options.indexer
   • README: the Filter DSL section now shows the two forms the /ys endpoint accepts, filter[field][operator]=value and field:operator:value joined with AND. The old field[op]value example was never accepted and its filter was dropped without an error. The onYetisearchPageSkip event is documented with the arguments it receives, page and config, and setting skip to true on the event leaves a page out
   • bin/plugin yetisearch-pro query -F help and the README said field[op]value, but the filter is written without brackets: type=content, price>=10, taxonomy.category=plugins. A filter the command cannot read, such as type[=]content, now prints a warning naming it on stderr (so --raw JSON on stdout stays valid) instead of being dropped without a word, or read as a filter on a field called type[

v2.2.2

6 days ago

1. • Updated the bundled YetiSearch library to 2.5.3, which fixes facets: range facets now work on any numeric field (they were only read by the distance facet), YetiSearch::search() no longer drops the facets option, a facet on a list field such as tags no longer fails the search, and float values are no longer cut to integers. YetiSearch Pro's own search does not request facets, so this matters to sites that call the bundled library directly

v2.2.1

1 week ago

1. • Updated the bundled YetiSearch library to 2.5.2, which ships its SQLite feature check again. Run php user/plugins/yetisearch-pro/vendor/yetidevworks/yetisearch/scripts/check_sqlite_features.php to see whether your PHP has the FTS5 and R-tree support that search needs

v2.2.0

1 week ago

1. • PDF indexing. The text inside PDFs is now searchable. Turn on PDF Indexing in the plugin settings and reindex: every PDF in an indexed page's folder becomes a search result of its own, and a hit links to the page of the PDF where the words were found (file.pdf#page=12). Text is extracted in PHP by the bundled YetiPDF library, so there is nothing to install on the server
   • PDF indexing needs PHP 8.3 or newer. On older PHP versions the setting is ignored, a notice goes to the Grav log, and everything else works as before
   • Largest PDF (default 50MB) and Most Pages per PDF (default 1000) settings keep very large files from slowing down indexing
   • Extracted text is cached, so a reindex only reads the PDFs that changed. Uploading or deleting a PDF in the admin updates the index straight away when real-time indexing is on
   • Index this page's PDFs on the page editor's YetiSearch tab (yetisearch-pro: index-pdfs: false in frontmatter) keeps the page in the index but leaves its PDFs out
   • PDF hits carry doc_type: pdf, pdf_file, pdf_page, pdf_pages, page_route and page_title in their metadata (result._meta) for use in templates
2. • The YetiSearch tab on the page editor works again. It saved its settings under yetisearch-pro: in the page's frontmatter while indexing only read yetisearch:, so Index this page, Index child pages and the extra fields list had no effect when set in the admin. Both keys are now read, yetisearch-pro: is the documented one, and it wins if a page sets both. Pages that were switched off in the admin will leave the index on the next reindex
   • Saving a page in the admin no longer removes its child pages from the index. With real-time indexing on, saving /blog deleted the entries for every page under /blog, and because smart indexing still saw them as unchanged, only a reindex with --flush brought them back. Saving the home page did the same to the whole site
   • A page removed, unpublished or excluded outside the admin no longer leaves its sections behind in search results. A reindex removed the page's own entry but not the separate entries for each of its headings
   • Changes made by real-time indexing are now recorded in the smart indexing state, so the next reindex knows about pages added or removed since the last one
   • The plugin runs on PHP 7.4 as its requirements say. Several files used PHP 8 syntax (match, static return types, mixed and union parameter types), which stopped the search client, the indexer and the CLI commands from loading on PHP 7.4

v2.1.2

2 weeks ago

1. • Plugins that change documents through onYetisearchBuildDocument now see pages saved in the admin too. Before, the event fired only during a full reindex, so a page saved in the admin was stored as built until the next reindex

v2.1.1

3 weeks ago

1. • Noise calibration. Semantic search now works out for itself where nonsense stops and real searches start on your content. After each full embedding run it embeds a set of meaningless test phrases, measures how closely they match your pages, and sets the cutoff just above that. A small site and a large one need very different cutoffs, and one fixed number either let nonsense through on the large site or turned real searches away on the small one. Set Noise Calibration to Off to keep the 2.1.0 behavior
   • Calibration Strictness setting (default 2.0) moves the cutoff up or down without re-embedding. Lower it if good matches are missing, raise it if nonsense gets through
   • The YetiSearch Pro page in Admin Next shows each index's noise cutoff, whether it was measured or is the default, and any test phrases that were skipped because they match your own content. bin/plugin yetisearch-pro embed --status shows the cutoff too
2. • Requires yetidevworks/yetisearch 2.5.1
   • bin/plugin yetisearch-pro embed measures the cutoff even when every page is already embedded, so an upgraded site gets calibrated without re-embedding anything

v2.1.0

3 weeks ago

1. • Semantic search. Search can now find pages by meaning as well as keywords: "can't log in" finds the password reset page, "hire an advisor" finds the consulting page. Turn it on under Semantic Search in the plugin settings and point it at an embedding API: OpenAI, OpenRouter, a self-hosted Ollama server, or anything else that speaks the OpenAI embeddings API. Keyword and meaning rankings are merged, so pages that match on both rank first. Without it enabled nothing changes
   • Pages are embedded after each save once the response has been sent, after a reindex, by a new Semantic Search Embedding scheduler job (every 15 minutes, on by default, runs only while semantic search is enabled), or with the new bin/plugin yetisearch-pro embed command (--status, --test, --reset). Only new or changed pages are sent to the provider, so reindexing unchanged content costs nothing. bin/plugin yetisearch-pro index embeds afterwards unless given --no-embed, and bin/plugin yetisearch-pro schedule --task embed runs the job by hand
   • The YetiSearch Pro page in Admin Next shows embedding coverage with a progress bar, and has Embed, Test connection and Re-embed all buttons
   • The search modals show keyword results while the visitor types and replace them with results ranked by meaning once typing pauses, so no keystroke waits on the embedding provider. The /ys endpoint accepts semantic=0 for keyword ranking only and reports semantic in its response
   • Settings for provider, API base URL, API key (or env:NAME to read it from the environment), model, dimensions, the balance between meaning and keywords, minimum similarity, minimum query length, search timeout and query/document prefixes, plus a Semantic Search toggle per index. If the provider is slow or unreachable, searches fall back to keywords
2. • Requires yetidevworks/yetisearch 2.4, under its correct package name (the plugin asked for yetisearch/yetisearch, a name that does not exist on Packagist)
3. • Pages saved in Admin Next are indexed again right away. Admin Next saves pages through the API plugin, which fires onAdminAfterSave rather than onFlexAfterSave, so realtime indexing never ran for those saves
   • On a multi-language site, a page saved with realtime indexing on no longer ends up in the index twice. The realtime update built the document id without the page's language, so it added a second copy next to the one a full index had written instead of replacing it

v2.0.3

1 month ago

1. • A page reached through version fallback no longer has its Twig processed twice, which could abort the whole index run. A page that documents Twig inside {% verbatim %} threw "Unexpected end of template" on the second pass, and because that happened outside the guard around normal rendering, one such page stopped the run instead of falling back to raw markdown

v2.0.2

1 month ago

1. • An API key limited to specific permissions can no longer use YetiSearch Pro's tools beyond what it was given, even when the account that created it is a super admin. A key limited to YetiSearch Pro keeps working at the level it was given