Logging

how-to

Configuring logging; working with the event bus; and log redaction for data security.

The Couchbase Java SDK logs events and also provides an event bus that transmits information about the behavior of your database system, including system and metric events.

Logging

The Couchbase Java SDK uses SLF4J, a logging façade that lets you use any logging framework that has an SLF4J binding. This includes popular Java logging frameworks like Log4j, Logback, and java.util.logging (JUL).

To see log messages from the Couchbase SDK, add an SLF4J binding as a dependency of your project.

SLF4J API versions

At the time of writing, there are two different versions of the SLF4J API:

Version 2 is the modern version of SLF4J. It is actively maintained, and recommended for most users.

Version 1.7 is no longer maintained, but you can still use it if your preferred SLF4J binding does not support version 2.

The Couchbase SDK is compatible with both versions of the SLF4J API. The SDK’s Maven POM has a dependency on version 1.7, but you can override this by using version 2 in your project.

Using Log4j 2

Log4j 2 is a popular and flexible logging framework. This section shows how to configure your project to use Log4j 2 with the Couchbase SDK.

First, add an SLF4J binding for Log4j 2 as a dependency of your project. The following example uses the binding for SLF4J API version 2.

Maven
Gradle

Add these as children of the dependencies element.

pom.xml

<dependency>
  <groupId>org.apache.logging.log4j</groupId>
  <artifactId>log4j-slf4j2-impl</artifactId>
  <version>2.22.0</version>
</dependency>

<!-- If your SLF4J binding requires API version 2
     (like log4j-slf4j2-impl in this example!),
     add this dependency to your project to ensure
     Maven uses the correct SLF4J API version. -->
<dependency>
  <groupId>org.slf4j</groupId>
  <artifactId>slf4j-api</artifactId>
  <version>2.0.9</version>
</dependency>

An alternate way to ensure Maven uses the correct version of the SLF4J API is to declare the dependency on log4j-slf4j2-impl before the dependency on the Couchbase SDK. See the Maven documentation on Transitive Dependencies to learn more about how Maven resolves transitive dependency version conflicts.

build.gradle

// Add this to the `dependencies` section:
implementation("org.apache.logging.log4j:log4j-slf4j2-impl:2.22.0")

Gradle automatically uses the correct SLF4J API 2.x dependency required by log4j-slf4j2-impl, even though the Couchbase SDK declares a dependency on SLF4J API 1.7.

Configuring Log4j 2 output

Log4j 2 needs a configuration file to tell it which messages to log, where to write them, and how each message should be formatted.

Here’s an example log4j2.xml configuration file you can use to get started. It tells Log4j 2 to log messages to the console, and sets some reasonable logging levels.

If your project uses the Maven Standard Directory Layout, this file should live in the src/main/resources directory. This makes it available at runtime as a class path resource.

src/main/resources/log4j2.xml

<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN">
  <Appenders>
    <Console name="Console" target="SYSTEM_OUT">
      <PatternLayout pattern="%d{ISO8601_OFFSET_DATE_TIME_HHCMM} %-5p [%c:%L] %m%n"/>
    </Console>
  </Appenders>
  <Loggers>
    <!-- Trace/debug/info messages from the Couchbase SDK's repackaged Netty
         are of little interest, unless you're debugging a network issue. -->
    <Logger name="com.couchbase.client.core.deps.io.netty" level="warn"/>

    <!-- Uncomment if using the 'io.captureTraffic' client setting. -->
    <!-- <Logger name="com.couchbase.io" level="trace"/> -->

    <!-- Most messages from the Couchbase SDK are logged under
         this prefix. Change the level to "debug" to see more
         details about SDK activity, or "warn" to see less.
         In production environments, we recommend "info". -->
    <Logger name="com.couchbase" level="info"/>

    <!-- The default level for everything else. -->
    <Root level="info">
      <AppenderRef ref="Console"/>
    </Root>
  </Loggers>
</Configuration>

Consult the Log4J 2 configuration documentation for more information and advanced configuration options.

Using `java.util.logging` (JUL)

If java.util.logging (JUL) is your preferred logging framework, add the slf4j-jdk14 SLF4J binding as dependency of your project.

Maven
Gradle

Add these as children of the dependencies element.

pom.xml

<dependency>
  <groupId>org.slf4j</groupId>
  <artifactId>slf4j-jdk14</artifactId>
  <version>2.0.9</version>
</dependency>

<!-- If your SLF4J binding requires API version 2
     (like slf4j-jdk14 in this example!),
     add this dependency to your project to ensure
     Maven uses the correct SLF4J API version. -->
<dependency>
  <groupId>org.slf4j</groupId>
  <artifactId>slf4j-api</artifactId>
  <version>2.0.9</version>
</dependency>

An alternate way to ensure Maven uses the correct version of the SLF4J API is to declare the dependency on slf4j-jdk14 before the dependency on the Couchbase SDK. See the Maven documentation on Transitive Dependencies to learn more about how Maven resolves transitive dependency version conflicts.

build.gradle

// Add this to your `dependencies` section:
implementation("org.slf4j:slf4j-jdk14:2.0.9")

Gradle automatically uses the correct SLF4J API 2.x dependency required by slf4j-jdk14, even though the Couchbase SDK declares a dependency on SLF4J API 1.7.

Configuring a JUL Logger

By default, JUL logs INFO level and above. If you want to set it to DEBUG (or the JUL equivalent: Fine) you can do it like this programmatically before initializing the Cluster object (or creating a custom ClusterEnvironment):

Logger logger = Logger.getLogger("com.couchbase.client");
logger.setLevel(Level.FINE);
for (Handler h : logger.getParent().getHandlers()) {
  if (h instanceof ConsoleHandler) {
    h.setLevel(Level.FINE);
  }
}

We do not recommend using JUL in production. Dedicated logging frameworks like Log4j 2 and Logback are more configurable, and tend to perform better than JUL.

The Event Bus

Event Bus Stability

While the event bus functionality itself is considered stable, the events themselves may not be. Please only consume the events you are interested in, and add error handling code in case of unexpected behavior.

Log files are neither fun to wade through, nor do they have any kind of real-time aspect. To make them usable, normally their content is piped into systems such as Graphite or Logstash. Since most setups interleave all different kinds of log messages, it makes it very hard to see whats going on, let alone perform post-disaster analysis.

To make the situation better and ultimately improve supportability, the Java SDK provides you with the ability to tap into all events before they get logged and consume them in "real-time".

You can subscribe to the event bus, and receive and react to events as they are happening; not when someone parses the logs, sends them into another system where an alarm is triggered, and eventually a sysadmin checks what is going on. The time delta between an event happening and reacting to it can thus be substantially decreased.

The following code subscribes to the event bus and prints out all events that are published on it with INFO or WARN level:

ClusterEnvironment environment = ClusterEnvironment.builder().build();

environment.eventBus().subscribe(event -> {
  // handle events as they arrive
  if (event.severity() == Event.Severity.INFO || event.severity() == Event.Severity.WARN) {
    System.out.println(event);
  }
});

Cluster cluster = Cluster.connect(
    connectionString,
    ClusterOptions.clusterOptions(username, password).environment(environment)
);

Bucket bucket = cluster.bucket(bucketName);

This leads to output similar to this:

CoreCreatedEvent{severity=INFO, category=com.couchbase.core, duration=PT0S, createdAt=43700573062858, description={"clientVersion":"3.0.0","clientGitHash":"a3d7a770","coreVersion":"2.0.0","coreGitHash":"a3d7a770","userAgent":"couchbase-java/3.0.0 (Mac OS X 10.14.6 x86_64; OpenJDK 64-Bit Server VM 1.8.0_202-b08)","maxNumRequestsInRetry":32768,"ioEnvironment":{"nativeIoEnabled":true,"eventLoopThreadCount":6,"eventLoopGroups":["KQueueEventLoopGroup"]},"ioConfig":{"captureTraffic":[],"mutationTokensEnabled":true,"networkResolution":"auto","dnsSrvEnabled":true,"tcpKeepAlivesEnabled":true,"tcpKeepAliveTimeMs":60000,"configPollIntervalMs":2500,"kvCircuitBreakerConfig":"disabled","queryCircuitBreakerConfig":"disabled","viewCircuitBreakerConfig":"disabled","searchCircuitBreakerConfig":"disabled","analyticsCircuitBreakerConfig":"disabled","managerCircuitBreakerConfig":"disabled","numKvConnections":1,"maxHttpConnections":12,"idleHttpConnectionTimeoutMs":30000,"configIdleRedialTimeoutMs":300000},"compressionConfig":{"enabled":true,"minRatio":0.83,"minSize":32},"securityConfig":{"tlsEnabled":false,"nativeTlsEnabled":true,"hasTrustCertificates":false,"trustManagerFactory":null},"timeoutConfig":{"kvMs":2500,"kvDurableMs":10000,"managementMs":75000,"queryMs":75000,"viewMs":75000,"searchMs":75000,"analyticsMs":75000,"connectMs":10000,"disconnectMs":10000},"loggerConfig":{"customLogger":null,"fallbackToConsole":false,"disableSlf4j":false,"loggerName":"CouchbaseLogger","diagnosticContextEnabled":false},"orphanReporterConfig":{"emitIntervalMs":10000,"sampleSize":10,"queueLength":1024},"retryStrategy":"BestEffortRetryStrategy","requestTracer":"OwnedSupplier"}, context=CoreContext{coreId=1}, cause=null}

NodeConnectedEvent{severity=INFO, category=com.couchbase.node, duration=PT0S, createdAt=43700609755560, description=Node connected, context=NodeContext{coreId=1, managerPort=8091, remote=127.0.0.1}, cause=null}

BucketOpenedEvent{severity=INFO, category=com.couchbase.core, duration=PT0.281625729S, createdAt=43701036027888, description=Opened bucket "travel-sample", context=CoreContext{coreId=1}, cause=null}

We recommend filtering on the specific events you are interested in, since most of the time only a subset of the published ones will be of use to you. Also, there are new events added between releases so make sure these new events do not break your functionality.

Blocking Warning

If you consume the EventBus you MUST NOT block inside the consumer callback. It will stall all other consumers. If you must write into a blocking sink like a blocking HTTP API you MUST write it onto a different thread with a non-blocking queue first.

Log Redaction

Redacting logs is a two-stage process. If you want to redact client logs (for example before handing them off to the Couchbase Support team) you first need to enable log redaction in your application.

LogRedaction.setRedactionLevel(RedactionLevel.FULL);

Different redaction levels are supported — please see the RedactionLevel enum description for more information.

Note that you need to run this command before any of the SDK code is initialized so all of the logs are captured properly. Once the SDK writes the logs with the tags to a file, you can then use the cblogredaction tool to obfuscate the log.

You may wish to read more on Log Redaction in the Server docs.

Logging

Logging

Using Log4j 2

Configuring Log4j 2 output

Using java.util.logging (JUL)

Configuring a JUL Logger

The Event Bus

Log Redaction

Using `java.util.logging` (JUL)