I’m building an open-source data engineering tool (SQLpipe), and am now trying to incorporate WAL CDC into the product.

It seems possible to detect when schema changes happen indirectly via the WAL. For example, anytime I run a create table or some other DDL command, there is a begin and commit message in the log, but no data on what actually happened.

Are there any other operations (other than schema changing DDL statements) that create such “empty” logs? If not, then I could just re-scan the column names and types of the tables that are being synced for changes.

Source link