DocsAPI & clients

Elasticsearch API

Search, aggregate and write collections with Elasticsearch’s clients, queries and bulk format, under the collections’ rules.

With the elasticsearch plugin, an app answers Elasticsearch’s REST API at /api/es. A collection is an index and a record is a document, so Elasticsearch’s own clients, search libraries and tools work against Sluurp with only the address changed. There is no second store to keep in step: a search reads the collection itself, through its full-text index.

To add it:

sluurp plugin add elasticsearch
import { Client } from "@elastic/elasticsearch";

const es = new Client({ node: "https://example.com/api/es", auth: { apiKey: process.env.SLUURP_TOKEN } });

const { hits, aggregations } = await es.search({
  index: "notes",
  query: { bool: { must: [{ match: { title: "trip" } }], filter: [{ range: { day: { gte: "now-30d" } } }] } },
  highlight: { fields: { title: {} } },
  aggs: { tags: { terms: { field: "tag" } } },
});
POST /api/es/notes/_search
Content-Type: application/json

{ "query": { "match": { "title": "trip" } }, "size": 20 }

Sign in

Send a Sluurp token (a person’s or an API key’s) in any of the forms Elasticsearch clients use:

  • Authorization: ApiKey <token>
  • Authorization: Bearer <token>
  • Authorization: Basic …, with any user name and the token as the password.

Every call is the same as the record API’s for that caller. The collection’s rules decide which documents are found, counted and aggregated, hidden fields are left out of _source, and writes run the collection’s hooks and reach realtime subscribers. A document the caller may not see is not found, and it is not counted either.

Queries

QueryBecomes
match, multi_match, query_string, simple_query_stringa full-text search of the searchable fields, ranked by BM25
match_phrase, or match on a field that is not searchablethe field contains the text
term, terms, idsequals, or is one of
range (gt, gte, lt, lte)compared; dates take now, now-7d, now+1h
existshas a value
prefix, wildcard (* at the start or end)starts with, ends with, contains
bool (must, filter, should, must_not), constant_score, match_all, match_nonecombined as Elasticsearch combines them

A dotted field follows relations: { "match": { "org.name": "acme" } } finds the members whose organisation’s name has “acme”, using the organisations’ own full-text index. Each search covers one index, so this is how you look into related records. Elasticsearch would have you copy the organisation into each member; here nothing is copied. field.keyword is read as the field itself, and _id as id. A query Sluurp cannot answer exactly, such as fuzzy or a script, is refused with a 400 rather than answered approximately.

Besides query, a search takes size and from (up to 10,000 together), sort (["title", { "day": "desc" }] or ?sort=day:desc), _source (false, a list of fields, or includes/excludes), highlight (with pre_tags and post_tags; <em> by default) and the URI search ?q=tag:home.

Each hit has its _score when words were searched. With no sort, the best matches come first.

Aggregations

terms (with size), date_histogram (calendar_interval of year, month or day) and filter make buckets. avg, sum, min, max, stats, value_count and cardinality measure them, or the whole result. A bucket can hold metrics and one more level of buckets:

{
  "size": 0,
  "aggs": {
    "per_month": {
      "date_histogram": { "field": "created_at", "calendar_interval": "month" },
      "aggs": { "total": { "sum": { "field": "amount" } } }
    }
  }
}

Aggregations count the same documents as the hits: the query and the rules apply to both.

Documents

PUT    /api/es/notes/_doc/{id}       creates the document, or changes the one with that id
POST   /api/es/notes/_doc            creates one with a new id
PUT    /api/es/notes/_create/{id}    creates it, or answers 409 if it exists
POST   /api/es/notes/_update/{id}    { "doc": { … }, "doc_as_upsert": true }
GET    /api/es/notes/_doc/{id}
DELETE /api/es/notes/_doc/{id}

Bulk

POST /api/es/_bulk (or /api/es/{index}/_bulk) takes Elasticsearch’s newline-delimited format, up to 5,000 actions:

{"index":{"_index":"notes","_id":"a1b2c3d4e5f6g7h"}}
{"title":"Museum trip","tag":"may"}
{"delete":{"_index":"notes","_id":"z9y8x7w6v5u4t3s"}}

Each action is answered on its own, in order, with errors: true if any failed. The actions are not one transaction; for writes that must all happen or none, use batch.

Also answered

GET /api/es (the cluster’s version, for clients that check it), _count, _msearch, _mapping (each field’s Elasticsearch type: searchable text is text with a keyword sub-field), HEAD and GET on an index, and _refresh, which does nothing because writes are seen at once.

Indexes are created and changed as collections, in the admin or with migrations, not through PUT /{index}.

View as Markdown