Connect

Discussions

Members

Resources

Events

 View Only

Connect CDC and why database logging settings directly impact data correctness

  • 1.  Connect CDC and why database logging settings directly impact data correctness

    Employee
    Posted an hour ago
    While working with Connect CDC, I ran into something that changed how I look at capture. CDC itself is not the full story. Its correctness depends heavily on how the database records changes.

    What CDC really depends on
    CDC does not invent change data. It reads what the database logs.
    That means the quality of replication depends on the quality of logging.
    A good example is IBM i / DB2 journaling. For CDC to work correctly, tables need to be journaled with BOTH images enabled. That means both before and after values are recorded.
    What happens when logging changes
    If journaling is changed to AFTER-only:
    • Update operations no longer have before images
    • CDC loses part of the change information
    • Capture can become incomplete or inconsistent
    At that point, the issue is not in CDC configuration. It is in how the database is recording history.
    Why this matters more than it seems
    With proper logging:
    • CDC has full visibility into changes
    • Updates and deletes can be handled accurately
    • Downstream systems stay consistent
    Without it, even a perfectly configured CDC pipeline can produce unexpected results.
    What this unlocked for me
    Once I understood this, a lot of behavior started to make sense.
    CDC is not just moving data.
    It is replaying what the database recorded.
    That means:
    • Good logging → reliable replication
    • Incomplete logging → gaps in behavior
    Where this shows up in practice
    This becomes critical when using:
    • Before-value capture
    • Auditing or history tracking
    • Complex transformations
    • High integrity systems
    All of these rely on having complete change information from the source.

    The takeaway
    CDC is not just configured in the replication tool. It is also shaped by database logging settings.
    It behaves like a log-powered pipeline, not just a data mover.
    Once I started looking at it that way, it became much easier to debug issues and design more reliable models.

    Have you ever traced a CDC issue back to database logging settings rather than the replication model itself?


    ------------------------------
    Adhitya Maya
    *Precisely Software Inc.
    ------------------------------