# 3.5. Nouveau

Warning

Nouveau is an experimental feature. Future releases might change how the endpoints work and might invalidate existing indexes.

Nouveau indexes enable you to query a database by using the Lucene Query Parser Syntax. A nouveau index uses one, or multiple, fields from your documents. You can use a nouveau index to run queries to find documents based on the content they contain.

Warning

Nouveau cannot function unless it has a functioning Nouveau server. See Nouveau Server Installation for details.

To create a nouveau index, you add a JavaScript function to a design document in the database. An index builds after processing one search request or after the server detects a document update. The `index` function takes the following parameters:

1. Field type - The type of the field, can be `string`, `text`, `double` or `stored`. See Field Types for more information.

2. Field name - The name of the field you want to use when you query the index. If you set this parameter to `default`, then this field is queried if no field is specified in the query syntax.

3. Data that you want to index, for example, `doc.address.country`.

4. (Optional) The third parameter includes the following field: `store`.

By default, a nouveau index response returns 25 rows. The number of hits that are returned can be changed by using the `limit` parameter. Each response includes a `bookmark` field. You can include the value of the `bookmark` field in subsequent queries to fetch results from deeper in the result set.

Example design document that defines a nouveau index:

    {
        "_id": "_design/nouveau_example",
        "nouveau": {
            "animals": {
                "index": "function(doc){ ... }"
            }
        }
    }

A nouveau index will inherit the partitioning type from the `options.partitioned` field of the design document that contains it.

## 3.5.1. Lucene Version Upgrade

Nouveau has been upgraded to use Lucene 10, earlier releases used Lucene 9.

Nouveau can query and update indexes created by Lucene 9 but will not create new ones. The index definition can optionally define a `lucene_version` field (which must be either 9 or 10 expressed as an integer). If not specified when defining a new index the current version (10) will be automatically added to the definition.

As Lucene only supports indexes up to one major release behind the current, it is important to rebuild all indexes to the current release. As Lucene major releases are infrequent, and Nouveau supports 9 and 10 versions simultaneously it is only necessary to rebuild version 9 indexes before Nouveau upgrades to Lucene 11 (when it exists). A `couch_scanner` plugin is available to automate this process, and can be enabled as follows;

    [couch_scanner_plugins]
    nouveau_index_upgrader = true

The plugin will scan all design documents for index definitions either with no `lucene_version` field or one equal to a previous version (lower than 10). The new index will be built by the plugin and, on successful completion, will update the `lucene_version` field in the index definition. Search requests against that index will seamlessly switch from the old index to the new one. Invoking the _nouveau_cleanup will delete the old indexes.

## 3.5.2. Field Types

Nouveau currently supports four field types, each of which has different semantics to the others.

Text

  A text field is the most common field type, the field value is analyzed at index time to permit efficient querying by the individual words within it (and wildcards, and regex, etc). This field type is not appropriate for sorting, range queries and faceting.

String

  A string field indexes the fields value as a single token without analysis (that is, no case-folding, no common suffixes are removed, etc). This field type is recommended for sorting and faceting. You can search on string fields but you must specify the `keyword` analyzer in the index definition for this field to ensure that your queries are not analyzed.

Double

  A double field requires a number value and is appropriate for sorting, range queries and range faceting.

Stored

  A stored field stores the field value into the index without analysis. The value is returned with search results but you cannot search, sort, range or facet over a stored field.

Warning

the type of any specific field is determined by the first index call. Attempts to index a different type into the same field will throw an exception and prevent the index from building.

## 3.5.3. Index functions

Attempting to index by using a data field that does not exist fails. To avoid this problem, use the appropriate guard clause.

Note

Your indexing functions operate in a memory-constrained environment where the document itself forms a part of the memory that is used in that environment. Your code’s stack and document must fit inside this memory. In other words, a document must be loaded in order to be indexed. Documents are limited to a maximum size of 64 MB.

The function that is contained in the index field is a JavaScript function that is called for each document in the database. The function takes the document as a parameter, extracts some data from it, and then calls the function that is defined in the `index` field to index that data.

The `index` function takes four parameters, where the third parameter is optional.

1. The first parameter is the type of the field.

2. The second parameter is the name of the field you intend to use when querying the index, and which is specified in the Lucene syntax portion of subsequent queries. An example appears in the following query:

       q=color:red

   The Lucene field name `color` is the first parameter of the `index` function.

   If the special value `"default"` is used when you define the name, you do not have to specify a field name at query time. The effect is that the query can be simplified:

       q=red

3. The third parameter is the data to be indexed. Keep the following information in mind when you index your data:

   - This data must be only a string, number, or boolean. Other types will cause an error to be thrown by the index function call.

   - If an error is thrown when running your function, for this reason or others, the document will not be added to that search index.

4. The fourth, optional, parameter is a JavaScript object with the following fields:

   Index function (optional parameter)

   - store - If `true`, the value is returned in the search result; otherwise, the value is not returned. Values are `true` or `false`. Default is `false`.

   Note

   If you do not set the `store` parameter, the index data results for the document are not returned in response to a query.

Example search index function:

    function(doc) {
        if (typeof(doc.min_length) == 'number') {
            index("double", "min_length", doc.min_length, {"store": true});
        }
        if (typeof(doc.diet) == 'string') {
            index("string", "diet", doc.diet, {"store": true});
        }
        if (typeof(doc.latin_name) == 'string') {
            index("string", "latin_name", doc.latin_name, {"store": true});
        }
        if (typeof(doc.class) == 'string') {
            index("string", "class", doc.class, {"store": true});
        }
    }

### 3.5.3.1. Index guard clauses

Runtime errors in the index function cause the document not to be indexed at all. The most common runtime errors are described below;

Example of failing to check whether the indexed value exists:

Warning

example of bad code

    index("double", "min_length", doc.min_length, {"store": true});

For documents without a min_length value, this index call will pass `undefined` as the value. This will be rejected by nouveau’s validation function and the document will not be indexed.

Example of failing to check whether the nested indexed value exists:

Warning

example of bad code

    if (doc.foo.bar) {
        index("string", "bar", doc.foo.bar, {"store": true});
    }

This bad example fails in a different way if `doc.foo` doesn’t exist; the evaluation of `doc.foo.bar` throws an exception.

    if (doc.foo && typeof(doc.foo) == 'object' && typeof(doc.foo.bar == 'string')) {
        index("string", "bar", doc.foo.bar, {"store": true});
    }

This example correctly checks that `doc.foo` is an object and its `bar` entry is a string.

Example of checking the index value exists but disallowing valid false values:

Warning

example of bad code

    if (doc.min_length) {
      index("double", "min_length", doc.min_length, {"store": true});
    }

We correct the previous mistake so documents without min_length are indexed (assuming there are other index calls for values that do exist) but we’ve acccidentally prevented the indexing of the `min_length` field if the `doc.min_length` happens to be `0`.

    if (typeof(doc.min_length == 'number')) {
      index("double", "min_length", doc.min_length, {"store": true});
    }

This good example ensures we index any document where `min_length` is a number.

## 3.5.4. Analyzers

Analyzers convert textual input into `tokens` which can be searched on. Analyzers typically have different rules for how they break up input into tokens, they might convert all text to lower case, they might omit whole words (typically words so common they are unlikely to be useful for searching), they might omit parts of words (removing `ing` suffixes in English, for example):

We expose a large number of Lucene’s analyzers. We invent one ourselves (`simple_asciifolding`);

- arabic

- armenian

- basque

- bulgarian

- catalan

- chinese

- cjk

- classic

- czech

- danish

- dutch

- email

- english

- finnish

- french

- galician

- german

- hindi

- hungarian

- indonesian

- irish

- italian

- japanese

- keyword

- latvian

- norwegian

- persian

- polish

- portugese

- romanian

- russian

- simple

- simple_asciifolding

- spanish

- standard

- swedish

- thai

- turkish

- whitespace

Example analyzer document:

    {
        "_id": "_design/analyzer_example",
        "nouveau": {
            "INDEX_NAME": {
                "index": "function (doc) { ... }",
                "default_analyzer": "$ANALYZER_NAME"
            }
        }
    }

### 3.5.4.1. Field analyzers

You may optionally specify a different analyzer for a specific field.

Example of defining different analyzers for different fields:

    {
        "_id": "_design/analyzer_example",
        "nouveau": {
            "INDEX_NAME": {
                "default_analyzer": "english",
                "field_analyzers": {
                    "spanish": "spanish",
                    "german": "german"
                },
                "index": "function (doc) { ... }"
            }
        }
    }

### 3.5.4.2. Testing analyzer tokenization

You can test the results of analyzer tokenization by posting sample data to the `_nouveau_analyze` endpoint.

Example of using HTTP to test the keyword analyzer:

    POST /_nouveau_analyze HTTP/1.1
    Content-Type: application/json
    {"analyzer":"keyword", "text":"ablanks@renovations.com"}

Example of using the command line to test the keyword analyzer:

    curl 'https://$HOST:5984/_nouveau_analyze' -H 'Content-Type: application/json'
        -d '{"analyzer":"keyword", "text":"ablanks@renovations.com"}'

Result of testing the keyword analyzer:

    {
        "tokens": [
            "ablanks@renovations.com"
        ]
    }

Example of using HTTP to test the standard analyzer:

    POST /_nouveau_analyze HTTP/1.1
    Content-Type: application/json
    {"analyzer":"standard", "text":"ablanks@renovations.com"}

Example of using the command line to test the standard analyzer:

    curl 'https://$HOST:5984/_nouveau_analyze' -H 'Content-Type: application/json'
        -d '{"analyzer":"standard", "text":"ablanks@renovations.com"}'

Result of testing the standard analyzer:

    {
        "tokens": [
            "ablanks",
            "renovations.com"
        ]
    }

## 3.5.5. Queries

After you create a search index, you can query it.

- Issue a partition query using: `GET /$DATABASE/_partition/$PARTITION_KEY/_design/$DDOC/_nouveau/$INDEX_NAME`

- Issue a global query using: `GET /$DATABASE/_design/$DDOC/_nouveau/$INDEX_NAME`

Specify your search by using the `q` parameter.

Example of using HTTP to query a partitioned index:

    GET /$DATABASE/_partition/$PARTITION_KEY/_design/$DDOC/_nouveau/$INDEX_NAME?include_docs=true&q=*:*&limit=1 HTTP/1.1
    Content-Type: application/json

Example of using HTTP to query a global index:

    GET /$DATABASE/_design/$DDOC/_nouveau/$INDEX_NAME?include_docs=true&q=*:*&limit=1 HTTP/1.1
    Content-Type: application/json

Example of using the command line to query a partitioned index:

    curl https://$HOST:5984/$DATABASE/_partition/$PARTITION_KEY/_design/$DDOC/
    _nouveau/$INDEX_NAME?include_docs=true\&q=*:*\&limit=1 \

Example of using the command line to query a global index:

    curl https://$HOST:5984/$DATABASE/_design/$DDOC/_nouveau/$INDEX_NAME?
    include_docs=true\&q=*:*\&limit=1 \

### 3.5.5.1. Query Parameters

A full list of query parameters can be found in the API Reference.

Note

Do not combine the `bookmark` and `update` options. These options constrain the choice of shard replicas to use for the response. When used together, the options might cause problems when contact is attempted with replicas that are slow or not available.

### 3.5.5.2. Relevance

When more than one result might be returned, it is possible for them to be sorted. By default, the sorting order is determined by ‘relevance’.

Relevance is measured according to Apache Lucene Scoring. As an example, if you search a simple database for the word `example`, two documents might contain the word. If one document mentions the word `example` 10 times, but the second document mentions it only twice, then the first document is considered to be more ‘relevant’.

If you do not provide a `sort` parameter, relevance is used by default. The highest scoring matches are returned first.

If you provide a `sort` parameter, then matches are returned in that order, ignoring relevance.

If you want to use a `sort` parameter, and also include ordering by relevance in your search results, use the special fields `-<score>` or `<score>` within the `sort` parameter.

### 3.5.5.3. POSTing search queries

Instead of using the `GET` HTTP method, you can also use `POST`. The main advantage of `POST` queries is that they can have a request body, so you can specify the request as a JSON object. Each parameter in the query string of a `GET` request corresponds to a field in the JSON object in the request body.

Example of using HTTP to POST a search request:

    POST /db/_design/ddoc/_nouveau/searchname HTTP/1.1
    Content-Type: application/json

Example of using the command line to POST a search request:

    curl 'https://$HOST:5984/db/_design/ddoc/_nouveau/searchname' -X POST -H 'Content-Type: application/json' -d @search.json

Example JSON document that contains a search request:

    {
        "q": "index:my query",
        "sort": "foo",
        "limit": 3
    }

## 3.5.6. Query syntax

The CouchDB search query syntax is based on the Lucene syntax. Search queries take the form of `name:value` unless the name is omitted, in which case they use the default field, as demonstrated in the following examples:

Example search query expressions:

    // Birds
    class:bird

    // Animals that begin with the letter "l"
    l*

    // Carnivorous birds
    class:bird AND diet:carnivore

    // Herbivores that start with letter "l"
    l* AND diet:herbivore

    // Medium-sized herbivores
    min_length:[1 TO 3] AND diet:herbivore

    // Herbivores that are 2m long or less
    diet:herbivore AND min_length:[* TO 2]

    // Mammals that are at least 1.5m long
    class:mammal AND min_length:[1.5 TO *]

    // Find "Meles meles"
    latin_name:"Meles meles"

    // Mammals who are herbivore or carnivore
    diet:(herbivore OR omnivore) AND class:mammal

    // Return all results
    *:*

Queries over multiple fields can be logically combined, and groups and fields can be further grouped. The available logical operators are case-sensitive and are `AND`, `+`, `OR`, `NOT` and `-`. Range queries can run over strings or numbers.

If you want a fuzzy search, you can run a query with `~` to find terms like the search term. For instance, `look~` finds the terms `book` and `took`.

Note

If the lower and upper bounds of a range query are both strings that contain only numeric digits, the bounds are treated as numbers not as strings. For example, if you search by using the query `mod_date:["20170101" TO "20171231"]`, the results include documents for which `mod_date` is between the numeric values 20170101 and 20171231, not between the strings “20170101” and “20171231”.

You can alter the importance of a search term by adding `^` and a positive number. This alteration makes matches containing the term more or less relevant, proportional to the power of the boost value. The default value is 1, which means no increase or decrease in the strength of the match. A decimal value of 0 - 1 reduces importance. making the match strength weaker. A value greater than one increases importance, making the match strength stronger.

Wildcard searches are supported, for both single (`?`) and multiple (`*`) character searches. For example, `dat?` would match `date` and `data`, whereas `dat*` would match `date`, `data`, `database`, and `dates`. Wildcards must come after the search term.

Use `*:*` to return all results.

The following characters require escaping if you want to search on them:

    + - && || ! ( ) { } [ ] ^ " ~ * ? : \ /

To escape one of these characters, use a preceding backslash character (`\`).

The response to a search query contains an `order` field for each of the results. The `order` field is an array where the first element is the field or fields that are specified in the `sort` parameter. See the sort parameter. If no `sort` parameter is included in the query, then the `order` field contains the Lucene relevance score.

### 3.5.6.1. Faceting

Nouveau Search also supports faceted searching, enabling discovery of aggregate information about matches quickly and easily. You can match all documents by using the special `?q=*:*` query syntax, and use the returned facets to refine your query.

Example of search query:

    function(doc) {
        index("string", "type", doc.type);
        index("double", "price", doc.price);
    }

To use facets, all the documents in the index must include all the fields that have faceting enabled. If your documents do not include all the fields, you receive a `bad_request` error with the following reason, “The `field_name` does not exist.” If each document does not contain all the fields for facets, create separate indexes for each field. If you do not create separate indexes for each field, you must include only documents that contain all the fields. Verify that the fields exist in each document by using a single `if` statement.

The `top_n` query parameter controls how many facets, per grouping, are returned, defaulting to 10, to a maximum of 1000.

Example if statement to verify that the required fields exist in each document:

    if (typeof doc.town == "string" && typeof doc.name == "string") {
        index("string", "town", doc.town);
        index("string", "name", doc.name);
       }

### 3.5.6.2. Counts

Note

The `counts` option is only available when making global queries.

The `counts` facet syntax takes a list of fields, and returns the number of query results for each unique value of each named field.

Note

The `count` operation works only if the indexed values are strings. The indexed values cannot be mixed types. For example, if 100 strings are indexed, and one number, then the index cannot be used for `count` operations. You can check the type by using the `typeof` operator, and convert it by using the `parseInt`, `parseFloat`, or `.toString()` functions.

Example of a query using the counts facet syntax:

    ?q=*:*&counts=["type"]

Example response after using of the counts facet syntax:

    {
        "total_rows":100000,
        "bookmark":"g...",
        "rows":[...],
        "counts":{
            "type":{
                "sofa": 10,
                "chair": 100,
                "lamp": 97
            }
        }
    }

### 3.5.6.3. Ranges

Note

The `ranges` option is only available when making global queries.

The value of the range parameter is a JSON object where the fields names are double fields, and the values of the fields are arrays of JSON objects. The objects must have a `label`, `min` and `max` value (of type string, double, double respectively), and optional `min_inclusive` and `max_inclusive` properties (defaulting to `true` if not specified).

Example of a request that uses faceted search for matching ranges:

    ?q=*:*&ranges={"price":[{"label":"cheap","min":0,"max":"100","max_inclusive":false},{"label":"expensive","min":100}]}

Example results after a ranges check on a faceted search:

    {
        "total_rows":100000,
        "bookmark":"g...",
        "rows":[...],
        "ranges": {
            "price": {
                "expensive": 278682,
                "cheap": 257023
            }
        }
    }