Connect

Discussions

Members

Resources

Events

 View Only
  • 1.  Connect CDC keep-alive queries and why a simple SELECT keeps your pipeline alive

    Employee
    Posted 08-10-2026 19:55
    What I noticed
    I was looking into how Connect CDC keeps database connections alive, and the implementation is surprisingly simple. It uses a single query that runs across all supported databases, but that small detail explains a lot of behavior in long-running systems.

    What CDC actually runs
    For keep-alive, CDC executes:

    SELECT * FROM <metabase>.rp_process 
    WHERE rp_process = 'SCHEMA';
    That query always returns one row.
    Why? Because rp_process is a kernel-managed table. It is always present and maintained by CDC itself, so the result is predictable and safe.
    Why this design works
    • Lightweight
      It does not scan user tables or large datasets
    • Consistent
      The table is always there, so the query never “fails” due to missing data
    • Safe
      It avoids touching application data
    It is basically the simplest possible query that still proves the database connection is alive.
    What’s different for PostgreSQL
    There is one extra step.
    • CDC runs the SELECT to keep the reader alive
    • It also performs a COMMIT to keep the writer alive
    This is because PostgreSQL separates reader and writer activity, and both need to stay active.
    Why this matters in production
    This small detail explains a few things I have seen:
    • Idle connections staying open even when there is no data movement
    • Periodic low-impact queries showing up in monitoring tools
    • No noticeable load even when keep-alive is active
    If you ever see this query in logs, it is not random. It is CDC doing exactly what it should.
    What changed for me
    I used to assume CDC keep-alive was complex or DB-specific.
    It is actually very minimal and consistent across systems.
    That simplicity is intentional. It reduces risk and keeps connections stable without interfering with real workloads.
    Takeaway
    CDC does not need complex logic to stay alive.
    A single predictable query against a kernel-managed table is enough to keep the pipeline connected and ready.

    Have you noticed these keep-alive queries in your DB logs and wondered what was generating them?


    ------------------------------
    Adhitya Maya
    *Precisely Software Inc.
    ------------------------------


  • 2.  RE: Connect CDC keep-alive queries and why a simple SELECT keeps your pipeline alive

    Employee
    Posted 08-12-2026 01:10

    This is useful to know, especially during troubleshooting. Seeing periodic queries against rp_process could easily be mistaken for unexpected CDC activity when there is no actual data movement. Knowing that it can be part of the connection keep alive mechanism helps distinguish normal background activity from actual replication processing



    ------------------------------
    Sathya LJK
    *Precisely Software Inc.
    ------------------------------



  • 3.  RE: Connect CDC keep-alive queries and why a simple SELECT keeps your pipeline alive

    Employee
    Posted 4 days ago
    Great insight, Adhitya. Understanding the keep-alive mechanism helps avoid unnecessary investigation when these queries appear in database monitoring tools. It is particularly useful during troubleshooting to distinguish normal CDC connection maintenance from actual replication activity. Thanks for sharing the underlying behaviour and PostgreSQL-specific considerations.


    ------------------------------
    Neha Goel
    *Precisely Software Inc.
    ------------------------------