Document Expiry

The example requires a document to be created in the 'source' collection with a key of SampleDocument2, a value of {'a_key': 'a_value'}, and most importantly that the document’s expiration (or TTL) set to 600 seconds or 10 minutes).

UPSERT INTO `bulk`.`data`.`source` (KEY, VALUE) VALUES ("SampleDocument2", {"a_key":"a_value"}, {"expiration":600});

/opt/couchbase/bin/cbc \
    create SampleDocument2 -V '{"a_key": "a_value"}' \
    -U couchbase://localhost/source \
    --scope=data --collection=source \
    -u Administrator -P password \
    --expiry=600

/Applications/Couchbase\ Server.app/Contents/Resources/couchbase-core/bin/cbc \
    create SampleDocument2 -V '{"a_key": "a_value"}' -U couchbase://localhost/source \
    --scope=data --collection=source \
    -u Administrator -P password \
    --expiry=600

"C:\Program Files\Couchbase\Server\bin\cbc" ^
    create SampleDocument2 -V "{'a_key': 'a_value'}" -U couchbase://localhost/source ^
    --scope=data --collection=source ^
    -u Administrator -P password ^
    --expiry=600

#!/usr/bin/python3
import sys
import couchbase.collection
import couchbase.subdocument as SD
from couchbase.cluster import Cluster, ClusterOptions
from couchbase_core.cluster import PasswordAuthenticator
from couchbase.durability import ServerDurability, Durability
from datetime import timedelta

pa = PasswordAuthenticator('Administrator', 'password')
cluster = Cluster('couchbase://127.0.0.1', ClusterOptions(pa))
bucket = cluster.bucket('bulk')
collection = bucket.scope('data').collection('source')

try:
  document = dict( a_key="a_value" )
  result = collection.upsert(
    'SampleDocument2',
    document,
    expiry=timedelta(minutes=10)
  )
  print("UPSERT SUCCESS")
  print("cas result:", result.cas)
except:
  print("exception:", sys.exc_info())

// Must use the Collections API
package com.jonstrabala;
import java.time.Duration;
import com.couchbase.client.java.*;
import com.couchbase.client.java.json.JsonObject;
import static com.couchbase.client.java.kv.UpsertOptions.upsertOptions;
public class DocExpiryTestCC {
    public static void main(String... args) throws Exception {
    	// Note, if not on the server you need to change "localhost" to your DNS name or IP
    	Cluster cluster = Cluster.connect("localhost", "Administrator", "password");
    	Bucket bucket = cluster.bucket("bulk");
    	// Collection collection = bucket.defaultCollection();
    	Collection collection = bucket.scope("data").collection("source");
    	String docID = "SampleDocument2";
    	Duration dura = Duration.ofMinutes(10);
    	try {
    		collection.upsert(
    			docID, JsonObject.create().put("a_key", "a_value"),
    			upsertOptions().expiry(dura) );
    		System.out.println("docID: " + docID + " expires in " + dura.getSeconds());
    	} catch (Exception e) {
    		System.out.println("upsert error for docID: " + docID + " " + e);
    	}
        bucket = null;
        collection = null;
    	cluster.disconnect(Duration.ofSeconds(2000));
    }
}

You now have a document in collection 'source' (keyspace bulk.data.source) with an expiration set.

To verify that your new document was created, access the Couchbase Web Console > Documents page and click the Documents then select the keyspace bulk.data.source. The new document gets displayed automatically (as this page will attempt to list the first few items). You will see one (1) document in the bulk.data.source keyspace (this will disappear on the document’s expiry of 10 minutes).

[Optional Step] Click on the document’s id, SampleDocument2 to view the documents Data and also the documents Metadata information. Note that the "expiration" field in the Metadata is non-zero (set to a Unix timestamp in seconds since epoch).

From the Couchbase Web Console > Eventing page, click ADD FUNCTION, to add a new Function. The ADD FUNCTION dialog appears.

In the ADD FUNCTION dialog, for individual Function elements provide the below information:

After providing all the required information in the ADD FUNCTION dialog, click Next: Add Code. The add_timer_before_expiry dialog appears.

From the Eventing screen, click the add_timer_before_expiry function to select it, then click Deploy.

The Eventing function is deployed and starts running within a few seconds. From this point, the defined Function is executed on all existing documents and on subsequent mutations.

Look at the Log for add_timer_before_expiry once it deploys (the "Log" link will appear once the function is deployed)

2022-04-17T15:34:21.034-07:00 [INFO] "OnUpdate add Timer 2 min. prior to TTL to DocId:" "SampleDocument2"

Now look at the Buckets in the UI the rr100.eventing.metadata keyspace will have 1280 documents related to the Eventing function and three (3) additional documents related to the timer

If you select Scopes & Collections (on the right) from the bucket bulk, then click on data (on the left) you should see one (1) document in the bulk.data.source collection (that you poked in via N1QL, cbc, Python or Java).

Wait a several minutes, return to Eventing in the UI and Look at the Log data again for add_timer_before_expiry at two minutes before the TTL was scheduled the timer will have fired and executed DocTimerCallback (note the logs display by the "Log" link are in reverse time order). Note the document this is based on had a 10 minute expiration and the timer will fire 2 minutes prior to that expiration (thus you might wait almost 8 minutes).

2022-04-17T15:42:03.481-07:00 [INFO] "DocTimerCallback 3 tgt archive via Key:" "SampleDocument2"
2022-04-17T15:42:03.481-07:00 [INFO] "DocTimerCallback 2 src expiry:" "2022-04-17T22:44:00.000Z"
2022-04-17T15:42:03.478-07:00 [INFO] "DocTimerCallback 1 on DocId:" "SampleDocument2"
2022-04-17T15:34:21.034-07:00 [INFO] "OnUpdate add Timer 2 min. prior to TTL to DocId:" "SampleDocument2"

Now look at the Documents in the UI again you will see one (1) document in the bulk.data.source keyspace (this will disappear on the document’s expiry of 10 minutes).

Now look at the Documents in the UI again you will see one (1) document in the bulk.data.target keyspace (this will persist)

Wait a few more minutes (actual just bit more than two minutes) past the 120 second window, then check the document in the bulk.data.source keyspace', you will find that it is missing and will not be accessible as it has expired due to the defined TTL on the document.