What's Changed

✨ Enhancement

🔬 Cross-index document hydration

Add a new ForeignKeys setting allowing to hydrate documents with documents coming from other indexes.

📓 Note: This implementation doesn't support a remote sharding environment

foreignKeys experimental feature

A new experimental feature, foreignKeys, has been added to the /experimental-feature route that must be set to true to activate the hydration.

curl -X PATCH 'http://127.0.0.1:7700/experimental-features' \
  -H 'Content-Type: application/json' \
  --data-binary '{"foreignKeys": true}'

foreignKeys index setting

A new index setting, foreignKeys, has been added to the /indexes/{index_uid}/settings:

// new setting, an array of foreign keys that allows multiple foreign relationships between indexes
"foreignKeys":  [
	{
		// the path in the JSON document containing foreign document ids
		"fieldName": "actors",
		
		// the UID of the foreign index containing the documents to fetch during hydration
        "foreignIndexUid": "actors"
	}
]

Example

With this new feature, a document shaped as follows:

{
  "id": 1,
  "title": "Forrest Gump",
  // Document IDs in foreign index
  "actors": [
    1
  ]
}

Will be hydrated as follows in a search response:

{
  "id": 1,
  "title": "Forrest Gump",
  "actors": [
	{
	  "id": 1,
	  "name": "Tom",
	  "familyName": "Hanks",
	  "birthDate": "1956-07-09"
	}
  ]
}

By @ManyTheFish in #6047

Disable proxy response buffering on Server-Sent Events (SSE)

Add X-Accel-Buffering: no on POST /chats/{workspace_uid}/chat/completions when the streaming mode is activated. By @YoEight in #6228

🪲 Bug fixes

• Fix a memory leak in the indexation pipeline by @Kerollmops in #6212

  We fixed an important memory leak caused by an invalid use of bumpalo. If you've seen Meilisearch using more and more memory over time, this issue is no longer visible in the latest version. If you want to read more about the bumpalo-trap we felt in, here is a good summary. It looks like this leak was introduced in v1.12, so approximately a year ago...

• Avoid losing tasks by applying deltas on the wtxn view by @Kerollmops in #6222

  We addressed a small performance regression introduced in v1.38.1 that affected sending updates to the engine during task deletion. We've restored the task deletion performance to match v1.38.0 levels and ensured no race conditions occur while still allowing updates during deletion.

🔩 Miscellaneous

• Ignore flaky test on windows by @dureuill in #6226
• Fix small documentation mistakes by @ManyTheFish in #6205
• Improve messaging when field is missing in the settings indexing pipeline by @Kerollmops in #6213

Full Changelog: https://github.com/meilisearch/meilisearch/compare/v1.38.0...v1.39.0