Connect

Discussions

Members

Resources

Events

 View Only
  • 1.  Connect CDC queue mismatch errors and why fixing the data doesn’t fix the problem

    Employee
    Posted 2 days ago
    What I ran into
    I hit a queue mismatch error recently where everything looked fine on the target, but replication still would not move forward. The issue was not the data itself. It was how CDC tracks transactions internally.

    How CDC actually tracks transactions
    CDC does not just store rows in a queue. It tracks transactions in two separate places:
    • rp_log - stores individual row operations
    • rp_trans - stores transaction metadata like expected row count
    In simple terms:
    rp_trans = "this transaction has N rows"
    rp_log   = actual rows captured
    What “queue mismatch” really means
    During apply, CDC compares the two:

    expected rows (rp_trans) vs actual rows (rp_log)
    If they do not match, CDC stops.
    It will not apply a partial transaction.
    This is intentional - it protects consistency.

    Why this happens even after fixing data
    This is the part that caught me.
    Fixing the target table does not fix the queue.
    The mismatch lives inside CDC’s transaction tracking, not in the actual table data.

    Common ways the mismatch happens
    • Manual row deletion from queues
      • Rows removed from rp_log
      • rp_trans still expects the original count
    • Partial cleanup during troubleshooting
      • Some rows removed, others left behind
      • Transaction boundary breaks
    • Skipping rows using ignore/delete options
      • Row removed
      • Transaction metadata unchanged
    • Retry after failures
      • Old transactions stay in queue
      • Counts drift out of sync
    • Edge cases with transaction bundling
      • Multiple transactions grouped
      • Interrupted rollback can leave inconsistencies
    Why the system refuses to continue
    CDC enforces a strict rule:
    A transaction must be applied as a whole or not at all.
    If even one row is missing, it stops instead of risking inconsistent data.
    What actually fixes it
    From what I have seen, safe resolution usually means:
    • Stop replication
    • Clean the affected queue properly (not manual deletes)
    • Recopy affected tables or the full set
    • Restart replication
    Trying to patch rows or adjust only the target does not help.
    What changed for me
    I used to think of CDC queues as a list of rows.
    This made it clear they are actually transaction contracts.
    • rp_trans defines what should exist
    • rp_log must match it exactly
    If that contract breaks, replication stops.
    Takeaway
    Queue mismatch is not a data issue. It is a transaction integrity issue inside CDC.
    Once I looked at it that way, the behavior made a lot more sense and troubleshooting became more straightforward.

    Have you run into queue-mismatch errors where everything looked fixed, but CDC still refused to move?


    ------------------------------
    Adhitya Maya
    *Precisely Software Inc.
    ------------------------------


  • 2.  RE: Connect CDC queue mismatch errors and why fixing the data doesn't fix the problem

    Employee
    Posted 5 hours ago

    Understanding that a queue mismatch is about transaction integrity rather than just the target data is an important distinction and can help avoid troubleshooting in the wrong place. Good Explanation!



    ------------------------------
    Sathya LJK
    *Precisely Software Inc.
    ------------------------------