Indexes and sets

A set is a sorted group of immutable data from a collection. An index is a group of sets within a collection. An index is also a database entity that facilitates data lookup.

Indexes

Indexes allow for the organization and retrieval of documents by attributes other than their refs. They are defined as documents within the indexes collection.

An index cannot be created and read in the same transaction.

When an index is added, it is immediately available for reads, but returns incomplete results until it is built. FaunaDB builds the index asynchronously by scanning over relevant documents. Upon completion, the index’s active field is set to true.

You can set active: true during index creation. If you do so, the index is assumed to be built and relevant documents are not included in the index creation. As you create, update, or delete relevant documents, the index is updated to reflect those modifications.

As such, we recommend only using active: true during index creation for empty collections. For collections with relevant documents, it is better to not set active: true and await the completion of its build, to avoid queries returning unexpectedly small result sets.

You can query whether an index is built:

Get(Index("index_name"))

If you see active: true in the output, the index is ready for your queries. Otherwise, you should wait and check again later, until active: true appears in the output.

It is possible to rename an index by updating its name field. Renaming an index changes its ref, but preserves inbound references to the index. Index data is not rebuilt.

An index’s terms, values, or partitions fields may not be changed. If you update the unique field, existing duplicate items are not removed from the index.

When an index is deleted, it becomes inaccessible, and its data is deleted asynchronously.

Field Value Definition and Requirements

name

String

Cannot be events, sets, self, collections, documents, or _.

source

VarArgs

One or more source objects describing source collections and (optional) field bindings.

terms

Array

An array of term objects describing the fields to be indexed. Optional.

values

Array

An array of value objects describing the fields to be covered. Optional.

unique

Boolean

If true, maintains a uniqueness constraint on combined terms and values. Optional.

serialized

Boolean

If true, writes to this index are serialized with concurrent reads and writes. Optional.

partitions

Integer

The number of sub-partitions within each term. Optional.

permissions

Object

Optional.

Reading indexes

Entries in an index have zero or more terms, zero or more covered values.

Entries are partitioned into sets by their terms. Within each set, entries are sorted by their covered values.

The match function is used to refer to a set of entries within an index:

curl https://db.fauna.com/ \
    -u fnACrVIrDDACAFiX3FN4PhwADpl7dtPRhWObP08j: \
    -d '{ "match": { "index": "spells_by_element" }, "terms": "fire" }'
client.Query(Match(Index("spells_by_element"), "fire"));
System.out.println(
        client.query(
           Match(
              Index(Value("spells_by_element")),
              Value("fire"))
        ).get());
result, _ := client.Query(f.MatchTerm(f.Index("spells_by_element"), "fire"))

fmt.Println(result)
client.query(Match(Index("spells_by_element"), "fire"))
client.query(q.match(q.index("spells_by_element"), "fire"))
$client.query do
  match index('spells_by_element'), 'fire'
end
client.query(
    Match(index: Index("spells_by_element"), terms: "fire")
)
client.query(q.Match(q.Index("spells_by_element"), "fire"))
  .then((ret) => console.log(ret))
{
  @set = {
    match: ref(id = "spells_by_element", collection = ref(id = "indexes")),
    terms: "fire"
  }
}
{map[match:{spells_by_element 0xc4203be2c0 <nil>} terms:fire]}
SetRef({"match":{"@ref":{"id":"spells_by_element","class":{"@ref":{"id":"indexes"}}}},"terms":"fire"})

If the index is configured to index multiple terms (i.e. a compound index), match should be passed an Array with the required number of term values:

curl https://db.fauna.com/ \
    -u fnACrVIrDDACAFiX3FN4PhwADpl7dtPRhWObP08j: \
    -d '{
          "match": { "index": "spells_by_element_and_name" },
          "terms": [ "fire", "Fire Beak" ]
        }'
client.Query(
  Match(
    Index("spells_by_element_and_name"),
    Arr("fire", "Fire Beak")));
System.out.println(
        client.query(
           Match(
             Index(Value("spells_by_element_and_name")),
             Arr(Value("fire"), Value("Fire Beak"))
           )
        ).get());
result, _ := client.Query(
    f.MatchTerm(
        f.Index("spells_by_element_and_name"),
        f.Arr{"fire", "Fire Beak"},
    ),
)

fmt.Println(result)
client.query(
  Match(
    Index("spells_by_element_and_name"),
    Arr("fire", "Fire Beak")))
client.query(
  q.match(
    q.index("spells_by_element_and_name"),
    ["fire", "Fire Beak"]
  ))
$client.query do
  match index('spells_by_element_and_name'),
        ['fire', 'Fire Beak']
end
client.query(
    Match(
        index: Index("spells_by_element_and_name"),
        terms: "fire", "Fire Beak"
    )
)
client.query(
  q.Match(
    q.Index("spells_by_element_and_name"),
    ["fire", "Fire Beak"]))
.then((ret) => console.log(ret))
{
  @set = {
    match: ref(id = "spells_by_element_and_name", collection = ref(id = "indexes")),
    terms: ["fire", "Fire Beak"]
  }
}
{map[terms:[fire Fire Beak] match:{spells_by_element_and_name 0xc4203b46c0 <nil>}]}
SetRef({"match":{"@ref":{"id":"spells_by_element_and_name","class":{"@ref":{"id":"indexes"}}}},"terms":["fire","Fire Beak"]})

Or conversely, if the index is configured with no terms, then only the index ref need be provided:

curl https://db.fauna.com/ \
    -u fnACrVIrDDACAFiX3FN4PhwADpl7dtPRhWObP08j: \
    -d '{ "match": { "index": "all_spells" } }'
client.Query(Match(Index("all_spells")));
System.out.println(
    client.query(Match(Index(Value("all_spells"))))
    .get());
result, _ := client.Query(f.Match(f.Index("all_spells")))

fmt.Println(result)
client.query(Match(Index("all_spells")))
client.query(q.match(q.index("all_spells")))
$client.query do
  match index('all_spells')
end
client.query(Match(index: Index("all_spells")))
client.query(q.Match(q.Index("all_spells")))
  .then((ret) => console.log(ret))
HTTP/1.1 200 OK
{
  "resource": { "@set": { "match": { "@ref": "indexes/all_spells" } } }
}
{ "@set": { "match": { "@ref": "indexes/all_spells" } } }
{ @set = {match: ref(id = "all_spells", collection = ref(id = "indexes"))} }
{map[match:{all_spells 0xc420399180 <nil>}]}
{ "@set": { "match": { "@ref": "indexes/all_spells" } } }
{ "@set": { "match": { "@ref": "indexes/all_spells" } } }
{ "@set": { "match": { "@ref": "indexes/all_spells" } } }
{ "@set": { "match": { "@ref": "indexes/all_spells" } } }
SetRef({"match":{"@ref":{"id":"all_spells","class":{"@ref":{"id":"indexes"}}}}})

Index terms are always scalar values, and match interprets Arrays as tuples. For single-term indexes, the following expressions are equivalent:

curl https://db.fauna.com/ \
    -u fnACrVIrDDACAFiX3FN4PhwADpl7dtPRhWObP08j: \
    -d '{ "match": { "index": "spells_by_element" }, "terms": "fire" }'
client.Query(Match(Index("spells_by_element"), "fire"));
System.out.println(
    client.query(
      Match(Index(Value("spells_by_element")), Value("fire"))
    ).get());
result, _ := client.Query(f.MatchTerm(f.Index("spells_by_element"), "fire"))

fmt.Println(result)
client.query(Match(Index("spells_by_element"), "fire"))
client.query(q.match(q.index("spells_by_element"), "fire"))
$client.query do
  match index('spells_by_element'), 'fire'
end
client.query(
    Match(index: Index("spells_by_element"), terms: "fire")
)
client.query(q.Match(q.Index("spells_by_element"), "fire"))
  .then((ret) => console.log(ret))
HTTP/1.1 200 OK
{
  "resource": {
    "@set": {
      "match": { "@ref": "indexes/spells_by_element" },
      "terms": "fire"
    }
  }
}
{
  "@set": {
    "match": { "@ref": "indexes/spells_by_element" },
    "terms": "fire"
  }
}
{
  @set = {
    match: ref(id = "spells_by_element", collection = ref(id = "indexes")),
    terms: "fire"
  }
}
{map[match:{spells_by_element 0xc4203b0680 <nil>} terms:fire]}
{
  "@set": {
    "match": { "@ref": "indexes/spells_by_element" },
    "terms": "fire"
  }
}
{
  "@set": {
    "match": { "@ref": "indexes/spells_by_element" },
    "terms": "fire"
  }
}
{
  "@set": {
    "match": { "@ref": "indexes/spells_by_element" },
    "terms": "fire"
  }
}
{
  "@set": {
    "match": { "@ref": "indexes/spells_by_element" },
    "terms": "fire"
  }
}
SetRef({"match":{"@ref":{"id":"spells_by_element","class":{"@ref":{"id":"indexes"}}}},"terms":"fire"})
curl https://db.fauna.com/ \
    -u fnACrVIrDDACAFiX3FN4PhwADpl7dtPRhWObP08j: \
    -d '{
          "match": { "index": "spells_by_element" },
          "terms": [ "fire" ]
        }'
client.Query(Match(Index("spells_by_element"), Arr("fire")));
System.out.println(
        client.query(
           Match(Index(Value("spells_by_element")), Arr(Value("fire")))
        ).get());
result, _ := client.Query(
    f.MatchTerm(f.Index("spells_by_element"), f.Arr{"fire"}),
)

fmt.Println(result)
client.query(Match(Index("spells_by_element"), Arr("fire")))
client.query(q.match(q.index("spells_by_element"), ["fire"]))
$client.query do
  match index('spells_by_element'), ['fire']
end
client.query(
    Match(index: Index("spells_by_element"), terms: "fire")
)
client.query(q.Match(q.Index("spells_by_element"), ["fire"]))
  .then((ret) => console.log(ret))
HTTP/1.1 200 OK
{
  "resource": {
    "@set": {
      "match": { "@ref": "indexes/spells_by_element" },
      "terms": "fire"
    }
  }
}
{
  "@set": {
    "match": { "@ref": "indexes/spells_by_element" },
    "terms": "fire"
  }
}
{
  @set = {
    match: ref(id = "spells_by_element", collection = ref(id = "indexes")),
    terms: "fire"
  }
}
{map[match:{spells_by_element 0xc42038d940 <nil>} terms:fire]}
{
  "@set": {
    "match": { "@ref": "indexes/spells_by_element" },
    "terms": "fire"
  }
}
{
  "@set": {
    "match": { "@ref": "indexes/spells_by_element" },
    "terms": "fire"
  }
}
{
  "@set": {
    "match": { "@ref": "indexes/spells_by_element" },
    "terms": "fire"
  }
}
{
  "@set": {
    "match": { "@ref": "indexes/spells_by_element" },
    "terms": "fire"
  }
}
SetRef({"match":{"@ref":{"id":"spells_by_element","class":{"@ref":{"id":"indexes"}}}},"terms":"fire"})

Source objects

Source objects describe the source collection of index entries and, optionally, bindings. A binding must be a pure lambda function that emits values to be used as a term and/or value.

The collection field can be a single collection ref or set of refs. Documents within collections matching the collection field apply the associated bindings to be used in the index’s terms or values. A collection ref can only exist in one source object. If the collection field is a wildcard (_), the index evaluates all collections. Bindings associated with a wildcard are only used if the collection is not matched by any other source object.

An index cannot be created in the same transaction that creates its source collection(s).
Field Type Definition and Requirements

collection

Ref

The collection or collections to be indexed, or a wildcard.

fields

Object

An object mapping a binding name to a transformer.

Binding objects

A binding object contains field names bound to pure, single-argument lambda functions. The function must take the document to be indexed and emit either a single scalar value or an array of scalar values. Binding functions are not permitted to perform reads or writes.

curl https://db.fauna.com/ \
    -u fnACrVIrDDACAFiX3FN4PhwADpl7dtPRhWObP08j: \
    -d '{
          "object": {
            "binding1": {
              "query": {
                "lambda": "document",
                "expr": {
                  "select": [ "data", "field" ],
                  "from": { "var": "document" }
                }
              }
            }
          }
        }'
client.Query(
  Obj(
    "binding1", Query(document => Select(Arr("data", "field"), document))
  ));
System.out.println(
  client.query(
    Obj(
      "binding1", Query(
        Lambda(
          Value("document"),
          Select(
            Arr(Value("data"), Value("field")),
            Var("document"))))
    )
  ).get());
result, _ := client.Query(
    f.Obj{
        "binding1": f.Query(
            f.Lambda(
                "document",
                f.Select(f.Arr{"data", "field"}, f.Var("document")),
            ),
        ),
    },
)

fmt.Println(result)
client.query(
  Obj(
    "binding1" -> Query(
      Lambda { document =>
        Select(Arr("data", "field"), document)
      })
  ))
client.query(
  {
    "binding1": q.query(
      lambda document: q.select(["data", "field"], document)
    )
  })
$client.query do
  {
    binding1: query(do |document| select(['data', 'field'], document) end)
  }
end
client.query(
    Obj(
        "binding1" => Query(
            { document in
                Select(path: "data", "field", from: document)
            }
        )
    )
)
client.query({binding1: q.Query(
  q.Lambda("document", q.Select(["data", "field"], q.Var("document"))))})
.then((ret) => console.log(ret))
HTTP/1.1 200 OK
{
  "resource": {
    "binding1": {
      "@query": {
        "lambda": "document",
        "expr": {
          "select": [ "data", "field" ],
          "from": { "var": "document" }
        }
      }
    }
  }
}
{
  "binding1": {
    "@query": {
      "lambda": "document",
      "expr": {
        "select": [ "data", "field" ],
        "from": { "var": "document" }
      }
    }
  }
}
{
  binding1: QueryV({
    lambda=document,
    expr={
      select=[data, field],
      from={var=document}
    }
  })
}
map[binding1:{[123 34 108 97 109 98 100 97 34 58 34 105 110 115 116 97 110 99 101 34 44 34 101 120 112 114 34 58 123 34 115 101 108 101 99 116 34 58 91 34 100 97 116 97 34 44 34 102 105 101 108 100 34 93 44 34 102 114 111 109 34 58 123 34 118 97 114 34 58 34 105 110 115 116 97 110 99 101 34 125 125 125]}]
{
  "binding1": {
    "@query": {
      "lambda": "document",
      "expr": {
        "select": [ "data", "field" ],
        "from": { "var": "document" }
      }
    }
  }
}
{
  "binding1": {
    "@query": {
      "lambda": "document",
      "expr": {
        "select": [ "data", "field" ],
        "from": { "var": "document" }
      }
    }
  }
}
{
  "binding1": {
    "@query": {
      "lambda": "document",
      "expr": {
        "select": [ "data", "field" ],
        "from": { "var": "document" }
      }
    }
  }
}
{
  "binding1": {
    "@query": {
      "lambda": "document",
      "expr": {
        "select": [ "data", "field" ],
        "from": { "var": "document" }
      }
    }
  }
}
{ binding1: Query("[object Object]") }

Term objects

Term objects describe the fields used to locate entries in the index. If multiple terms are provided, documents that are missing a value emit a Null term in the index for that field.

If no term objects are defined, passing term values to match is not required. The resulting set contains all documents within the source collection.

A value can be from a field in the document or a binding defined by the source object.

Field Type Definition

field

Array

The path of the field within an document to be indexed.

binding

String

The name of a binding from a source object.

Value Objects

Value objects describe the data covered by the index, which are included in query results on the index and control ordering of entries having the same terms. By default, indexes cover only the refs of included documents.

A value can be from a field in the document or a binding defined in a source object.

Field Type Definition

field

Array

The path of the field within an document to be indexed.

binding

String

The name of a binding from a source object.

reverse

Boolean

Whether this field’s value should sort reversed. Defaults to false.

The document’s ref may also appear in before and after cursors when paging through an index with Paginate, even if the ref does not appear as a covered value in the index configuration. These "extra" refs are used to stabilize pagination.

Field Values

Any of a document’s fields may be indexed. The value of field in a Term or Value object indicates the position within a document for a field. For example, the field ref refers to the top-level ref field. The field ["data", "address", "street"] refers to the street field contained in an address object within the document’s data object.

Ordering

Covered values sort lexically according to their type. Strings and Numbers sort using their natural order, while Arrays and Objects sort according to their contents. For example, { "name": "Hen Wen" } appears after { }, and { "age": 110 }.

Documents may have different types of values in the same field, or a document may be missing a field entirely. An index covering that path will sort values in this order: Integers, Strings, Arrays, Objects, Refs, Times, Dates, Booleans, Decimals, Nulls.

The default sort order may be reversed on a per-field basis using the reverse flag in the index configuration.

Partitioning

FaunaDB partitions indexes on the set of terms for each index entry. Therefore, there is no performance difference between an index on the terms [{ "field": ["data", "name"] }, { "field": ["data", "age"] }] and [{ "field": ["data", "age"] }, { "field": ["data", "name"] }].

Term partitions may be sub-divided by the number of partitions in the index configuration. Sub-dividing indexes beyond their term may increase write performance for terms with many entries, but may also increase read latency.

Collection indexes default to 8 sub-partitions; all other indexes default to a single term partition. No index may have more than 8 sub-partitions.

Sets

Sets are sorted groups of tuples. An index derives sets from documents within the collections in its source. As documents are created, modified, and deleted, sets are updated to reflect their documents' current state.

Indexes are groups of sets, each of which has a natural key; a tuple of zero or more terms. The Match query function constructs a set ref to identify a set for a given tuple of terms within an index:

curl https://db.fauna.com/ \
    -u fnACrVIrDDACAFiX3FN4PhwADpl7dtPRhWObP08j: \
    -d '{ "match": { "index": "spells_by_element" }, "terms": "water" }'
client.Query(Match(Index("spells_by_element"), "water"));
System.out.println(
   client.query(
      Match(Index(Value("spells_by_element")), Value("water"))
).get());
result, _ := client.Query(f.MatchTerm(f.Index("spells_by_element"), "water"))

fmt.Println(result)
client.query(Match(Index("spells_by_element"), "water"))
client.query(q.match(q.index("spells_by_element"), "water"))
$client.query do
  match index('spells_by_element'), 'water'
end
client.query(
    Match(index: Index("spells_by_element"), terms: "water")
)
client.query(q.Match(q.Index("spells_by_element"), "water"))
.then((ret) => console.log(ret))
HTTP/1.1 200 OK
{
  "resource": {
    "@set": {
      "match": { "@ref": "indexes/spells_by_element" },
      "terms": "water"
    }
  }
}
{
  "@set": {
    "match": { "@ref": "indexes/spells_by_element" },
    "terms": "water"
  }
}
{
  @set = {
    match: ref(id = "spells_by_element", collection = ref(id = "indexes")),
    terms: "water"
  }
}
{map[match:{spells_by_element 0xc4202b6040 <nil>} terms:water]}
{
  "@set": {
    "match": { "@ref": "indexes/spells_by_element" },
    "terms": "water"
  }
}
{
  "@set": {
    "match": { "@ref": "indexes/spells_by_element" },
    "terms": "water"
  }
}
{
  "@set": {
    "match": { "@ref": "indexes/spells_by_element" },
    "terms": "water"
  }
}
{
  "@set": {
    "match": { "@ref": "indexes/spells_by_element" },
    "terms": "water"
  }
}
SetRef({"match":{"@ref":{"id":"spells_by_element","class":{"@ref":{"id":"indexes"}}}},"terms":"water"})

Set refs are unique according to their structure: Two set refs with the same structure refer to the same set within a database. Query functions such as Union, Intersection, and Join allow the construction of more complex logical set identifiers:

curl https://db.fauna.com/ \
    -u fnACrVIrDDACAFiX3FN4PhwADpl7dtPRhWObP08j: \
    -d '{
          "intersection": [
            {
              "match": { "index": "spells_by_element" },
              "terms": "water"
            },
            {
              "match": { "index": "spells_by_element" },
              "terms": "fire"
            }
          ]
        }'
client.Query(
  Intersection(
    Match(Index("spells_by_element"), "water"),
    Match(Index("spells_by_element"), "fire")));
System.out.println(
    client.query(
       Intersection(
          Match(Index(Value("spells_by_element")), Value("fire")),
          Match(Index(Value("spells_by_element")), Value("water")))
    ).get());
result, _ := client.Query(
    f.Intersection(
        f.MatchTerm(f.Index("spells_by_element"), "water"),
        f.MatchTerm(f.Index("spells_by_element"), "fire"),
    ),
)

fmt.Println(result)
client.query(
  Intersection(
    Match(Index("spells_by_element"), "water"),
    Match(Index("spells_by_element"), "fire")))
client.query(
  q.intersection(
    q.match(q.index("spells_by_element"), "water"),
    q.match(q.index("spells_by_element"), "fire")
  ))
$client.query do
  intersection match(index('spells_by_element'), 'water'),
               match(index('spells_by_element'), 'fire')
end
client.query(
    Intersection(
        Match(
            index: Index("spells_by_element"),
            terms: "water"
        ),
        Match(
            index: Index("spells_by_element"),
            terms: "fire"
        )
    )
)
client.query(
  q.Intersection(
    q.Match(q.Index("spells_by_element"), "water"),
    q.Match(q.Index("spells_by_element"), "fire")))
.then((ret) => console.log(ret))
HTTP/1.1 200 OK
{
  "resource": {
    "@set": {
      "intersection": [
        {
          "@set": {
            "match": { "@ref": "indexes/spells_by_element" },
            "terms": "water"
          }
        },
        {
          "@set": {
            "match": { "@ref": "indexes/spells_by_element" },
            "terms": "fire"
          }
        }
      ]
    }
  }
}
{
  "@set": {
    "intersection": [
      {
        "@set": {
          "match": { "@ref": "indexes/spells_by_element" },
          "terms": "water"
        }
      },
      {
        "@set": {
          "match": { "@ref": "indexes/spells_by_element" },
          "terms": "fire"
        }
      }
    ]
  }
}
{
  @set = {
    intersection: [
    {
      @set = {
        match: ref(id = "spells_by_element", collection = ref(id = "indexes")),
        terms: "fire"
      },
      {
        @set = {
          match: ref(id = "spells_by_element", collection = ref(id = "indexes")),
          terms: "water"
        }
      }
    ]
  }
}
{map[intersection:[{map[match:{spells_by_element 0xc42026c760 <nil>} terms:water]} {map[match:{spells_by_element 0xc42026c920 <nil>} terms:fire]}]]}
{
  "@set": {
    "intersection": [
      {
        "@set": {
          "match": { "@ref": "indexes/spells_by_element" },
          "terms": "water"
        }
      },
      {
        "@set": {
          "match": { "@ref": "indexes/spells_by_element" },
          "terms": "fire"
        }
      }
    ]
  }
}
{
  "@set": {
    "intersection": [
      {
        "@set": {
          "match": { "@ref": "indexes/spells_by_element" },
          "terms": "water"
        }
      },
      {
        "@set": {
          "match": { "@ref": "indexes/spells_by_element" },
          "terms": "fire"
        }
      }
    ]
  }
}
{
  "@set": {
    "intersection": [
      {
        "@set": {
          "match": { "@ref": "indexes/spells_by_element" },
          "terms": "water"
        }
      },
      {
        "@set": {
          "match": { "@ref": "indexes/spells_by_element" },
          "terms": "fire"
        }
      }
    ]
  }
}
{
  "@set": {
    "intersection": [
      {
        "@set": {
          "match": { "@ref": "indexes/spells_by_element" },
          "terms": "water"
        }
      },
      {
        "@set": {
          "match": { "@ref": "indexes/spells_by_element" },
          "terms": "fire"
        }
      }
    ]
  }
}
SetRef({"intersection":[{"@set":{"match":{"@ref":{"id":"spells_by_element","class":{"@ref":{"id":"indexes"}}}},"terms":"water"}},{"@set":{"match":{"@ref":{"id":"spells_by_element","class":{"@ref":{"id":"indexes"}}}},"terms":"fire"}}]})

The Paginate function is used to retrieve the tuples of a set. The Page object returned by Paginate contains an array of tuples and cursors for moving forward or backward within the set.

Field Type Definition and Requirements

data

Array

The elements in the page.

after

Cursor

The cursor for the next page, inclusive. Optional.

before

Cursor

The cursor for the previous page, exclusive. Optional.

Pages can be further manipulated using collection-oriented functions such as Map and Filter, or individual elements can be extracted using select.

Was this article helpful?

We're sorry to hear that.
Tell us how we can improve! documentation@fauna.com

Thank you for your feedback!