Command platformKept for this browser session. No learning progress is saved.

Station 3 of 3 · 60 minutes · S05 / S06 · Exercise D

Acknowledgement, retry and dead letters

Explain what is confirmed, what can be delivered again, and how failed work is recovered deliberately.

A shipping worker receives an order and fails before confirming it. Another order contains invalid business data. Recover deliberately without treating a receive as a completed business effect.

Order queueTransacted worker · rollback / commitAfter 3 failed attempts → DLQ → repair / replay

A transacted worker commits healthy orders or rolls back failed deliveries. After three failed total attempts, invalid work moves to a dead letter queue for inspection and deliberate repaired replay.

Prerequisites: Java foundations and the common setup, with Terminal 1 showing BROKER_READY. Use your own local broker. Keep a fresh namespace consistent across cooperating clients.

1. Predict

If a worker receives an order and stops before JMS commit, can that same business ID return? Will waiting and retrying repair an invalid order?

Write your prediction in the worksheet before commands, hints or AI. Keep the sent and received business-ID sets as evidence.

2. Run

Terminal 3 — producer

Send one identifiable order

Lab root: the folder containing mvnw.cmd (source) or RUN.md (runtime ZIP).

The local broker remains in Terminal 1. Use a fresh namespace for this rollback experiment.

.\scripts\lab.cmd s05 produce --count 1 --id ack-demo --namespace pilot-ack

Observe: One SEND. Record its business ID and JMSMessageID separately.

Terminal 2 — worker

Receive, then roll back

Lab root: the folder containing mvnw.cmd (source) or RUN.md (runtime ZIP).

This worker intentionally rolls back one delivery and exits. It does not commit a successful business result.

.\scripts\lab.cmd s05 consume --count 1 --rollback --namespace pilot-ack

Observe: RECEIVE followed by ROLLBACK for the same business ID.

Terminal 2 — recovery worker

Receive the rolled-back work again

Lab root: the folder containing mvnw.cmd (source) or RUN.md (runtime ZIP).

Use the same namespace. This run commits the recovered delivery.

.\scripts\lab.cmd s05 consume --count 1 --timeout 10000 --namespace pilot-ack

Observe: Same business ID, JMSRedelivered=true and attempt=2 in this controlled run; then COMMIT. An unavailable delivery count is reported as unknown, not invented.

3. Break

Terminal 3 — producer

Prepare a fresh crash experiment

Lab root: the folder containing mvnw.cmd (source) or RUN.md (runtime ZIP).

Choose a separate namespace so prior delivery attempts do not affect this comparison.

.\scripts\lab.cmd s05 produce --count 1 --id crash-demo --namespace pilot-crash

Observe: One fresh business ID to compare across the crash.

Terminal 2 — worker

Crash before JMS commit

Lab root: the folder containing mvnw.cmd (source) or RUN.md (runtime ZIP).

The --crash option deliberately terminates this client process. Exit code 17 is expected; the broker remains running.

.\scripts\lab.cmd s05 consume --count 1 --crash --namespace pilot-crash

Observe: RECEIVE, CRASH_BEFORE_JMS_COMMIT and exit 17. There is no successful JMS COMMIT.

Terminal 2 — recovery worker

Wait for broker detection and redelivery

Lab root: the folder containing mvnw.cmd (source) or RUN.md (runtime ZIP).

The broker may need several seconds to detect the dead connection. Use the bounded ten-second recovery timeout.

.\scripts\lab.cmd s05 consume --count 1 --timeout 10000 --namespace pilot-crash

Observe: The same business ID returns with redelivery metadata, then commits. Receive alone did not confirm completion.

Terminal 3 — producer

Send healthy work and one invalid order

Lab root: the folder containing mvnw.cmd (source) or RUN.md (runtime ZIP).

S06 makes order-2 invalid by setting priorityClass=invalid. Use a fresh namespace for this three-order experiment.

.\scripts\lab.cmd s06 produce --count 3 --namespace pilot-retry

Observe: Three SEND business IDs. Identify the business ID associated with order-2.

Terminal 2 — worker

Observe the attempt limit

Lab root: the folder containing mvnw.cmd (source) or RUN.md (runtime ZIP).

count counts ALL deliveries, including rollbacks. Two healthy deliveries plus three attempts at the invalid order means five receives.

.\scripts\lab.cmd s06 consume --count 5 --namespace pilot-retry

Observe: Two healthy COMMITs, three ROLLBACKs for the invalid business ID, approximately 500 ms redelivery delay. Three total attempts are two additional retries. Healthy work continues.

Terminal 3 — inspection

Inspect before repairing

Lab root: the folder containing mvnw.cmd (source) or RUN.md (runtime ZIP).

Wait for the worker to exit. DLQ means dead letter queue: failed work retained for investigation after the attempt limit.

.\scripts\lab.cmd s06 browse --target dlq --namespace pilot-retry

Observe: Exactly one failed business ID, matching order-2. Browsing does not consume it.

Terminal 3 — deliberate replay

Repair and replay the known failed order

Lab root: the folder containing mvnw.cmd (source) or RUN.md (runtime ZIP).

The demonstration repairs priorityClass in both JSON payload and JMS properties, then commits its DLQ receive and new send together. Replaying without fixing invalid data would fail again.

.\scripts\lab.cmd s06 replay --count 1 --namespace pilot-retry

Observe: REPLAY preserves the business ID and produces a new technical JMS send. Business identity and JMSMessageID are different concepts.

Terminal 2 — worker

Process the repaired work

Lab root: the folder containing mvnw.cmd (source) or RUN.md (runtime ZIP).

Consume the deliberate replay from the same namespace.

.\scripts\lab.cmd s06 consume --count 1 --namespace pilot-retry

Observe: One successful COMMIT for the repaired business ID.

4. Explain

In a transacted session, commit confirms receives and publishes sends against the same broker; rollback reverses both. Find these calls in the private Java code.
AUTO_ACKNOWLEDGE confirms a synchronous delivery when receive returns successfully, or a listener delivery after onMessage returns successfully. Later business failure is not automatically protected.
CLIENT_ACKNOWLEDGE acknowledges all messages consumed so far in that session, not just the individual message object. These lab workers instead use explicit transacted sessions.
Compare the business ID and delivery metadata before and after rollback/crash. Explain three total attempts versus two retries and why invalid business data needs repair.
At-least-once delivery permits duplicates. A JMS commit does not prove an external payment or email happened once. S09’s atomic local H2 effect is a later extension, not an end-to-end exactly-once guarantee.

Staged hints

Use one hint at a time after recording your prediction.

Hint 1

Locate the transaction boundary. A receive and an external business effect are different events. An uncommitted delivery can be delivered again.

Hint 2

Rollback priorityClass=invalid; commit healthy work. With three produced messages, count=5 includes two healthy deliveries plus three poison attempts. Repair both payload and property on deliberate replay, preserving the business ID.

Check your understanding

Choose an answer, then check it for feedback. These questions do not affect quiz exam scoring.

5. Adapt

In private DExercise.java, implement the invalid-data failure branch: rollback when priorityClass=invalid, otherwise commit after successful work. Identify receive/work/commit in the code, rebuild with -Ptraining, then repeat in a fresh namespace and predict the five-delivery result. If time remains, implement inspection and repaired replay yourself; the full exercise can continue after the pilot. Keep complete worked solutions in the private solutions module.

Reflect on the change

What did you predict, what changed, and which IDs or state support your explanation?

Which suggestion did you accept or reject, and what evidence informed that decision? If you used no assistance, explain the decision you made from the supplied material.

Completion checklist

This is a local self-assessment. Nothing is submitted or saved.

I compared one business ID and redelivery metadata across rollback and crash.I distinguished three attempts from two retries and showed the invalid order in the DLQ.I verified repaired replay, empty input/DLQ, and explained the commit boundary without claiming exactly-once.

Clean up this station

Terminal 3 — inspection

Verify empty input and DLQ

Lab root: the folder containing mvnw.cmd (source) or RUN.md (runtime ZIP).

Wait for the repaired worker to exit. These two snapshots should agree with your recorded commits.

.\scripts\lab.cmd s06 browse --namespace pilot-retry
.\scripts\lab.cmd s06 browse --target dlq --namespace pilot-retry

Observe: Both show visible=0. If either differs, investigate your business-ID evidence before removing anything.

Terminal 3 — inspection

Check the acknowledgement experiments

Lab root: the folder containing mvnw.cmd (source) or RUN.md (runtime ZIP).

Both recovery workers should already have exited. Drain any adaptation namespace separately using its known count; stop remaining workers.

.\scripts\lab.cmd s05 browse --namespace pilot-ack
.\scripts\lab.cmd s05 browse --namespace pilot-crash

Observe: Both show visible=0. Continue to the lab recap to stop your owned local broker.