CCDAK Confluent Certified Developer for Apache Kafka Certification Examination Question and Answers

The offset in the RecordMetadata object returned by the producer represents the position of the record in the partition — i.e., the sequential ID assigned by Kafka once the message is committed.

From Kafka Producer API Documentation:

“The offset is the position of the record in the partition. This is a unique, sequential number assigned by the broker.”

D refers to metadata.partition(), not offset().

B and C are unrelated to how Kafka handles committed offsets.

A 409 Conflict response in Kafka Connect REST API usually means:

The connector name provided already exists (i.e., a connector with that name is already deployed).

The Kafka Connect cluster is temporarily unavailable to accept new tasks due to a rebalance in progress.

From the Confluent Developer documentation and Kafka Connect REST API spec:

“409 – Conflict. The connector name is already in use or the worker group is in the process of rebalancing and cannot complete the request at this time.”

Option A is incorrect because invalid configurations return 400 Bad Request, not 409.

Option B is invalid because Kafka Connect does not have a strict concept of "capacity" that prevents connector creation with a 409.

According to the official Apache Kafka documentation, log compaction is the mechanism used to retain only the latest record for each key in a topic. By setting cleanup.policy=compact, Kafka ensures that older records with the same key are eventually removed, leaving only the most recent value for each key.

This behavior is exactly suited for use cases such as tracking the current state of an entity, including GPS tracker locations, user profiles, or configuration data. In this scenario, the key (trackerId) uniquely identifies a tracker, and the value represents its most recent latitude and longitude.

Option B (retention.ms=infinite) retains all historical data, which contradicts the requirement. Option C (min.cleanable.dirty.ratio) controls when compaction runs, not what data is retained. Option D (retention.ms=0) would immediately delete all data and is unsafe.

Therefore, setting cleanup.policy=compact is the correct and officially documented solution to always retain the latest location per tracker.

=========================

To change the replication factor of an existing topic, you must perform a partition reassignment, because the replication factor is not directly modifiable via kafka-topics.sh. The kafka-reassign-partitions.sh tool is the official command-line utility to handle partition reassignments, including changing the replication factor.

This tool allows you to generate a reassignment JSON specifying new replica assignments per partition and then apply it.

From official documentation:

“To change the replication factor, use the kafka-reassign-partitions tool to assign the partitions to more or fewer brokers.”

Neither kafka-topics.sh nor kafka-reassign-topics.sh exists for this purpose, and kafka-recreate-topic.sh is not an official Kafka tool.

The Apache Kafka consumer documentation explains that when a RecordDeserializationException occurs, the consumer cannot continue polling until the problematic offset is skipped. Simply logging the error is insufficient, because the consumer will repeatedly fail on the same record.

The recommended pattern is to log the malformed record, extract its topic, partition, and offset from the exception, and then call consumer.seek() to move the consumer position to the next offset (offset + 1). This allows the application to skip the poison pill and resume processing subsequent records.

Option B is invalid because Kafka does not provide a consumer.skip() API. Option C is unnecessary if the application is designed to tolerate malformed records. Option D results in an infinite failure loop.

Therefore, seeking past the bad record after logging it is the correct and officially documented way to handle poison pill records while maintaining consumer liveness and resilience.

According to the Apache Kafka producer documentation, serialization is configured independently for message keys and values. The key and value can use different serializers depending on their data types.

In this scenario, the message key is a string, so the producer must be configured with org.apache.kafka.common.serialization.StringSerializer for the key. The value contains JSON, so a suitable JSON serializer must be configured for the value, such as a custom JSON serializer, a library-based serializer, or a byte-array serializer after converting the JSON to bytes.

Option B correctly reflects this separation by specifying a string serializer for the key and a JSON serializer for the value. Kafka does not automatically infer serializers, and setting a serializer to null is invalid. Additionally, there is no built-in “JSON” serializer for keys, making Option D incorrect.

Therefore, the correct approach—fully aligned with Kafka’s producer configuration model—is to explicitly configure a StringSerializer for the key and a JSON-capable serializer for the value.

=========================

To ensure exactly-once delivery and avoid duplicates even during retries:

enable.idempotence=true ensures deduplication on the broker

retries=2147483647 allows unlimited retries on retriable errors

max.in.flight.requests.per.connection=5 is the maximum value that preserves message order with idempotence

From Kafka Producer Config Docs:

“To achieve exactly-once semantics, set enable.idempotence=true, and max.in.flight.requests.per.connection ≤ 5.”

A is unrelated (consumer-side)

C disables retries

D disables idempotence, leading to duplicates

Session windows are ideal for tracking periods of activity separated by inactivity, such as user sessions.

From Kafka Streams Documentation > Windowing:

“A session window captures streams of events that are intermittent and separated by a gap of inactivity.”

Tumbling/Hopping/Sliding windows are fixed in size

Session windows are dynamic and close after inactivity timeout

This makes them perfect for identifying gaps in user interaction.

According to the official Apache Kafka documentation for Kafka Streams, exactly-once semantics (EOS) are controlled using the processing.guarantee configuration. To ensure that each input record is processed once and only once, and that the results are written atomically to the output topic, Kafka Streams provides built-in EOS support.

The correct and recommended configuration is processing.guarantee=exactly_once_v2, which is the default EOS implementation starting from Kafka 3.0. This mode uses Kafka’s transactional APIs to atomically commit consumer offsets and produce output records within a single transaction. As documented, this prevents duplicates even in the presence of failures, retries, or application restarts.

Option A is incorrect because log compaction does not guarantee exactly-once processing; it only retains the latest record per key. Option B is insufficient because enable.idempotence alone does not handle consumer offset commits or state store consistency in Kafka Streams. Option C is invalid because no such configuration exists in Kafka Streams.

Therefore, setting processing.guarantee=exactly_once_v2 is the only fully correct and officially supported solution for achieving EOS in Kafka Streams applications.

These three metrics are critical for cluster health:

OfflinePartitionsCount: Indicates partitions without active leaders — a sign of broker failure.

ActiveControllerCount: There should be exactly one active controller. A count ≠ 1 signals controller failure.

UncleanLeaderElectionsPerSec: Tracks leader elections where out-of-sync replicas were selected — risky for data loss.

From Kafka Monitoring Documentation:

“Offline partitions and unclean leader elections should trigger alerts. Also, ensure a single active controller is running.”

A is about topics pending deletion — not critical.

E is a per-topic record metric, not broker-level.

To send failed records to a dead letter queue (DLQ), you must configure:

errors.tolerance=all: Tells the connector to not fail on errors but handle them (e.g., send to DLQ).

errors.deadletterqueue.topic.name=DLQ-Topic: Specifies the DLQ topic.

errors.deadletterqueue.context.headers.enable=true: Includes error context in message headers.

From Kafka Connect Error Handling Docs:

“Kafka Connect supports directing problematic records to a separate topic (DLQ) using errors.* configs. Headers can include failure metadata.”

Options D, E, F are related to logging, not DLQ behavior.

Apache Kafka Connect stores source connector offsets in a Kafka topic defined by the offset.storage.topic configuration. Since this property is explicitly set to connect-offsets, Kafka Connect will commit offsets to that topic, making Option C correct.

The offset.storage.replication.factor controls how many replicas the offsets topic will have. When this value is set to -1, Kafka Connect uses the broker’s default replication factor, as documented in the Kafka Connect worker configuration reference. Therefore, Option A is also correct.

Option B is incorrect because Kafka Connect does not use Kafka’s internal consumer offsets topic (__consumer_offsets). It always uses the configured offsets topic. Option D is incorrect because offset.flush.timeout.ms defines how long Connect will wait for offset commits to complete, not how long it waits before attempting a commit. The commit interval itself is controlled by offset.flush.interval.ms (60 seconds in this case).

Thus, the correct statements that match the expected behavior are A and C.

Unit Testing → MockProducer

Integration Testing → Testcontainers

Performance Testing → Trogdor

Mock Data Generation → Connect Datagen

MockProducer: Simulates a Kafka producer in unit tests (no real broker interaction).

Testcontainers: Spawns Kafka in Docker for real environment testing.

Trogdor: Kafka’s built-in performance load testing framework.

Connect Datagen: Creates sample source records for test and demo purposes.

From Kafka Developer Tools Guide:

“Kafka developers commonly use MockProducer for unit tests, Testcontainers for integration, and Trogdor for performance tests.”

According to the official Apache Kafka consumer documentation, the auto.offset.reset configuration is only applied when a consumer has no valid committed offset for a partition. In this scenario, the consumer last committed offset 217, but the topic’s earliest available offset is now 318. This means the previously committed offset is out of range, most likely due to topic retention cleanup.

When a committed offset is out of range, Kafka treats it as invalid and applies the auto.offset.reset policy. Since the consumer is configured with auto.offset.reset=latest, it will start reading from the end of the log, not from the earliest available offset.

Kafka defines the “latest” offset as the log end offset, which is the offset of the next message to be written. Given that the topic’s offsets end at 588, the next offset is 589.

Therefore, the consumer will start reading from offset 589, skipping all existing records in the topic, which aligns with the documented Kafka consumer behavior.

=========================

The Apache Kafka Streams documentation defines a co-partitioning requirement for KStream–KTable and KStream–KStream joins. Both input topics must have the same number of partitions and the same key partitioning strategy.

One valid solution is to use a GlobalKTable (Option A). A GlobalKTable is fully replicated to every Kafka Streams instance, removing the co-partitioning requirement. This approach is recommended when the reference data is relatively small and changes infrequently.

Another valid solution is to repartition one topic so that both topics have the same number of partitions (Option B). Kafka Streams provides repartition topics specifically for this purpose, allowing proper KStream–KTable joins.

Option C does not resolve the partition mismatch, as increasing instances does not change partitioning. Option D is incorrect because Kafka Streams does not automatically repartition both topics for joins; repartitioning must be explicitly configured.

Therefore, the correct and officially supported solutions are using a GlobalKTable and explicitly repartitioning one topic.

=========================

According to the official Apache Kafka Connect REST API documentation, restarting a connector can be done at different granularities: connector-level, task-level, or both. By default, calling POST /connectors/{name}/restart only restarts the connector instance, not its tasks.

To restart the connector and all of its tasks, the request must explicitly include the query parameter includeTasks=true. This instructs Kafka Connect to restart the connector and all associated tasks, regardless of whether they are currently running or failed.

Option B (onlyFailed=true) limits the restart to only failed tasks, which does not meet the requirement to restart all tasks. Option C restarts only the connector instance, leaving tasks untouched. Option D restarts a single specific task and does not restart the connector itself.

Therefore, the correct and officially documented REST request to restart the connector instance and all tasks is:

POST /connectors/s3-connector/restart?includeTasks=true

This ensures a full restart of the connector and its execution context across the Kafka Connect cluster.

Comprehensive and Detailed Explanation From Exact Extract:

✅ C. Consumers can consume both committed and uncommitted transactions.By default, Kafka consumers only read committed messages if they are configured with isolation.level=read_committed. However, if configured as read_uncommitted, they can also consume uncommitted (potentially aborted) transactional messages.

From Kafka Documentation:

"The isolation.level setting controls whether the consumer will read only committed messages or all messages, including uncommitted messages from ongoing or aborted transactions."

✅ D. Information about producers and their transactions is stored in the _transaction_state topic.Kafka uses an internal topic named __transaction_state to maintain metadata about producer transactions. This topic is essential for tracking the transaction lifecycle, fencing, and recovery.

From Kafka Internals:

“Kafka stores the state of active and completed transactions in an internal topic called __transaction_state.”

This syntax is a clear match to Protocol Buffers (Protobuf). It defines a schema with fields, types, and tags, which is a format supported by Kafka when using Protobuf-based schema registry serialization.

From Confluent Schema Registry Docs:

“Kafka supports Protobuf serialization, where schemas are written in .proto files and include fields with tags.”

int32, double, and string are standard Protobuf types.

Avro uses JSON-style schema.

JSON Schema uses JSON object structure, not .proto.

YAML is unrelated.

To maintain message ordering per bank account, the message key must be the account number. Kafka guarantees ordering per partition, and the partition is selected based on the hash of the key.

From Kafka Producer Docs:

“All records with the same key will be sent to the same partition, and Kafka preserves the order of records in a partition.”

Using account number ensures:

All messages for the same account go to the same partition

Order is maintained

Workload is evenly distributed if account keys are varied

Other options break ordering or introduce unnecessary randomness.

According to the official Apache Kafka Connect and Confluent JDBC Source Connector documentation, the correct connector class for a JDBC source connector is io.confluent.connect.jdbc.JdbcSourceConnector. This connector is used to read data from relational databases and publish each table as a Kafka topic.

To read all rows from a specific table, the configuration must include table.whitelist (or table.include.list in newer versions) with the table name, and a topic.prefix to determine the Kafka topic name. In this case, using topic.prefix=dbl- with table.whitelist=orders results in records being written to the dbl-orders topic, which matches the requirement.

Options A, B, and C are invalid because they reference a non-existent connector class (DdbcSourceConnector), contain unsupported properties such as topic.whitelist for a source connector, or include malformed/incorrect JDBC parameters. Additionally, blacklisting tables does not ensure that only the orders table is read.

Therefore, option D is the only configuration that is syntactically correct, uses the proper connector class, and aligns with the official Kafka Connect JDBC Source Connector documentation.

To isolate logs for a specific connector, you can configure a separate logger and appender in the Connect worker’s log4j.properties file, using the connector’s name as the logging context.

From Kafka Connect Logging Docs:

“Kafka Connect loggers use hierarchical logger names. You can configure per-connector log levels and output files by extending log4j.properties.”

A and C change verbosity but don’t separate logs.

D is false; targeted logging is possible.

Table → Product, Customers

Stream → Orders_Placed, Shipment_Of_Orders

Tables represent slowly changing, reference data — e.g., customer profiles, product info.

Streams represent real-time event data — e.g., orders placed, shipments processed.

From Kafka Streams Documentation:

“Use KTable for reference datasets, and KStream for event-based processing such as orders or logs.”

According to the official Apache Kafka Streams documentation, Kafka Streams relies on internal topics (such as changelog topics and repartition topics) to store state, task assignments, and application metadata. These topics are essential for fault tolerance, rebalancing, and state recovery.

When auto-topic creation is disabled, Kafka Streams cannot automatically create the required internal topics unless they already exist. In this situation, the Streams application will fail during startup with a fatal, non-retriable exception, indicating that the necessary internal topics could not be created.

Kafka Streams does not pause or wait for manual topic creation, nor does it operate without storing metadata. Instead, it explicitly requires these topics to exist or be creatable. While administrators can manually pre-create the required internal topics with the correct configuration, failure to do so causes the application to terminate.

Therefore, the correct and documented behavior is that the Kafka Streams application terminates with a non-retriable exception when auto-topic creation is disabled and required internal topics are missing.

If the application is CPU-bound and lagging, adding more consumers to the group will allow better parallel processing, especially since the topic has 4 partitions, allowing up to 4 active consumers.

From Kafka Consumer Group Docs:

“Kafka achieves parallelism by distributing partitions across consumers in a group. Adding consumers helps reduce lag if partitions are underutilized.”

B may help but requires repartitioning and coordination.

C or D affects how much data is polled, not how fast it’s processed.