A newer version of this documentation is available.

View Latest

Import Filter Configuration

      +
      Pre-3.0 Legacy Configuration Equivalents

      This content describes configuration for Sync Gateway 3.0 and higher — for legacy configuration, see: Legacy Pre-3.0 Configuration

      Introduction

      Using an import filter will significantly improve efficiency if you’re working with large datasets:

      • Import filters are defined at the collection level and allow you to retrieve only the relevant documents you need rather than the entire dataset. The import filter helps determine which documents can be copied by Sync Gateway. It looks at the application’s needs and applies these criteria to all future changes.

      • By reducing the amount of data that needs to be processed, an import filter will improve the performance of your queries and analysis. It’s worth noting that Sync Gateway imports all documents by default, so it’s generally a good idea to use an import filter unless you have a strong reason not to.

      Function Provision

      Use the Database Configuration Admin Rest API endpoint POST /{db}/_config to provision an import filter for a database using the application/javascript mime type.

      If you are using legacy configuration then, you need to include it in your configuration file, see: import-filter.

      Configuration

      You need Couchbase Lite 3.1+ and Sync Gateway 3.1+ to use custom Scopes and Collections.
      If you’re using Capella App Services or Sync Gateway releases that are older than version 3.1, you won’t be able to access custom Scopes and Collections. To use Couchbase Lite 3.1+ with these older versions, you can use the default Collection as a backup option.

      The configuration settings described here are provisioned through the Database Configuration endpoints.

      {
        scopes: {
            {scopename...}: {
               collections: {
                  {collectionname...}: {
                     import_filter: "function(doc) { if (doc.type != 'mobile') { return false; } return true; }",
                  }
               }
            }
         },
         // other configuration
      }

      For more information, see Sync Gateway Configuration Schema.

      Property Description

      scopename

      Represents the name of each scope

      collections

      Contains different collections within each scope

      collectionname

      Represents the name of each collection within a scope.

      import_filter

      Used to decide if a document should be imported. It checks the type property of the document. If it is not 'mobile', the function returns false, otherwise, it returns true.

      • API

      • Legacy

      
      curl -X PUT "http://localhost:4985/froglist/_config/import_filter" \
      -H "accept: application/json" \
      -H "Content-Type: application/javascript" \
      -H "Authorization: Basic dXNlcm5hbWU6cGFzc3dvcmQ=" \
      -d "\"function(doc) {\ if (doc.type != 'mobile') {\ return false\ }\ return true\}\\\""
      
        {
          "databases": {
            "getting-started-db": {
              "bucket": "getting-started-bucket",
              "import_docs": true,
              "num_index_replicas": 0,
              // ... other config as required
              "import_filter": `
              function(doc) {
                if (doc.type != "mobile") {
                  return false
                }
                return true
                }`,  
              }
            }
        }

      Upsert Import Filter Function

      https:://{sgw-uri}/{db}/_config/import_filter

      This will allow you to update the database's import filter.

      Required Sync Gateway RBAC roles:

      • Sync Gateway Architect

      See: Import filter for more.

      Parameters

      Path Parameters

      Name Description Schema

      keyspace
      required

      The keyspace to run the operation against.

      A keyspace is a dot-separated string, comprised of a database name, and optionally a named scope and collection.

      String

      Body Parameter

      Name Description Schema

      Body
      optional

      The import filter to use

      Example: body_example

      String

      Header Parameters

      Name Description Schema

      If-Match
      optional

      If set to a configuration's Etag value, enables optimistic concurrency control for the request. Returns HTTP 412 if another update happened underneath this one.

      String

      Query Parameters

      Name Description Schema

      disable_oidc_validation
      optional

      If set, will not attempt to validate the configured OpenID Connect providers are reachable.

      Default: false

      Boolean

      Responses

      HTTP Code Description Schema

      200

      Updated import filter successfully

      400

      There was a problem with your request

      HTTP_Error

      404

      Resource could not be found

      HTTP_Error

      412

      Precondition Failed The supplied If-Match header did not match the current version of the configuration. Returned when optimistic concurrency control is used, and there has been an update to the configuration in between this update.

      HTTP_Error

      Schema

      This section shows Sync Gateway’s import control configuration settings in schema format for convenience in constructing JSON models for use in the Admin REST API.

      The configuration settings described here are provisioned through Access Control endpoints.

      Import_filter_model

      The import_filter controls whether a document written to the Couchbase Server bucket should be made available to Couchbase Mobile clients (that is, whether it ought to be imported).

      You should provision the filter as a Javascript function in the request body of a call to the Admin Rest API endpoint put {db}/_config/import_filter.

      Set the header’s content type to content-Type: application/javascript.

      The function takes the document body as parameter and is expected to return a boolean to indicate whether the document should be imported.

      If you do not provide a filter function then no filter will be applied and ALL documents will be imported.

      Type : string