GoldenGate PostgreSQL Replicat Stall: Check the ODBC Driver

Table of Contents

My GoldenGate PostgreSQL Replicat stopped stalling after I changed the DSN’s ODBC driver path to ggpsql25.so. The 1TB initial load then completed without another restart, and CDC continued normally. I narrowed the investigation to the driver layer using process stacks, PostgreSQL session state, and socket snapshots captured before restarting. I did not establish the driver’s exact protocol failure.

The load was moving a table with CLOB data from Oracle to PostgreSQL. Replicat still reported RUNNING, but its CPU usage was 0%, its checkpoint stopped moving, and SEND stopped responding. PostgreSQL showed idle in transaction / ClientRead. Neither side logged an error for the stall.

Capture the stall before restarting Replicat

My first useful step was to collect the process stack, database session state, and socket information while the process was still stuck. Restarting got past the current position, but the stall later returned elsewhere. Loading the same row by itself succeeded.

Classic initial-load Replicat was applying a table with a CLOB column, mapped to PostgreSQL text.
The process was RUNNING, with CPU at 0% and no checkpoint progress.
SEND REPLICAT ... STATUS timed out with OGG-15163.
The target session showed idle in transaction and ClientRead. Its last recorded query was RELEASE <sp>; SAVEPOINT <sp>.
The Replicat report, ggserr.log, and PostgreSQL log contained no error explaining the stall.

This differed from my earlier Extract checkpoint-lag investigation (Korean), where a long-running source transaction delayed progress. Here, the target Replicat stopped making progress.

Identify the libraries between Replicat and PostgreSQL

The stack pointed through two ODBC layers: the driver manager and the DataDirect driver selected by the DSN. In this deployment, Replicat connected directly to PostgreSQL over local TCP on the same target host.

Component	Recorded version or setting
GoldenGate for PostgreSQL	21.12.0.0.4 · Classic · OL7 build
Replicat	Classic · NODBCHECKPOINT · EXTTRAIL
PostgreSQL	16.15 · Patroni HA · asynchronous replication
Operating system	Rocky Linux 8.10
Loaded ODBC driver	GGpsql25.so · 07.16.0445

GoldenGate PostgreSQL Replicat initial-load path. Replicat calls pass through the ODBC driver manager and the selected driver. Arrows show data and call progress; responses return in the opposite direction. This is not a packet capture. — Figure 1. Replicat calls pass through the ODBC driver manager and the selected driver. Arrows show data and call progress; responses return in the opposite direction. This is not a packet capture. Open full-size diagram

Match the stack, session, and socket snapshots

I collected pstack, pg_stat_activity, and ss -tnp three times, approximately 15 seconds apart. The process IDs and client port let me match the observations rather than treating unrelated sessions as evidence.

The main thread remained inside the driver’s recv()

The three stacks matched, including their addresses. The samples spanned more than 31 seconds and repeatedly showed the same wait; they did not trace every instruction between captures. Four auxiliary threads used for logging and file caching were idle. The relevant main-thread frames were:

#0  recv ()                                   from libpthread.so.0
#1~#10  ?? ()                                 from lib/GGpsql25.so
#11 SQLPrepareW ()                             from lib/GGpsql25.so
#12~#13 ?? ()                                 from lib/libodbc.so
#14 SQLPrepareW ()                             from lib/libodbc.so
#15 DBODBC_prepare_statement (...)
#16 ODBCUTIL_stmt_t::Prepare (...)
#17 execute_statement (...)
#18 perform_sql_statements (...)
#19 ggs::er::ReplicatContext::replicate_io (...)
#20 ggs::er::ReplicatContext::processRecord (...)
#21 ggs::er::ReplicatContext::processReplicatLoop ()
#22~#27 ... main ()

Read this stack from the higher-numbered frames toward frame 0: the Replicat loop processes a record, enters statement handling, calls SQLPrepareW through libodbc.so, and reaches a socket read inside GGpsql25.so. The two SQLPrepareW frames belong to different libraries.

The name execute_statement describes the broader statement-handling path; it does not show that the final SQL execution had already completed. Preparing a statement, binding values, and executing it are useful conceptual stages. The PostgreSQL wire protocol also defines Parse, Bind, and Execute messages, but an ODBC call does not necessarily map one-to-one to a wire message.

The captured wait was in the preparation path, not a trail-read or an OGG internal-lock wait. The stack alone did not prove that a complete Parse message had been sent, or establish the full history of binding, execution, or LOB transfer.

PostgreSQL was waiting for client data

pid            | 3486801
client_port    | 39275
xact_start     | 11:36:47.059
state          | idle in transaction
wait_event     | Client / ClientRead
query          | RELEASE T5612C7B61CB0;SAVEPOINT T5612C7B61CB0

All three session snapshots were the same. PostgreSQL documents ClientRead as waiting to read data from the client. Together with idle in transaction, this was a client wait, not evidence that PostgreSQL was currently executing the last query shown. See the session and wait-event documentation.

The recorded RELEASE and SAVEPOINT activity was consistent with DataDirect’s TransactionErrorBehavior=2 behavior. In manual commit mode, that setting establishes savepoints around statement processing. Oracle’s OGG-30082 message explains that GoldenGate sets this DSN attribute internally to 2 for its supported error-handling features. That explains why setting it to 0 in this DSN did not take effect.

The connection was local and both queues were empty

ESTAB  0  0  <host>:39275  <host>:5432  users:(("replicat",pid=3486796,fd=9))

Port 39275 matched pg_stat_activity.client_port, linking the socket to the observed session. At the sampled times, Send-Q 0 showed no outstanding unacknowledged data in the send queue, and Recv-Q 0 showed no received data waiting for the application to read.

The local direct connection reduced the priority of explanations involving an external proxy or network appliance. Empty queues did not establish message completeness, reveal data still buffered inside the application, or rule out every network or HA failure. I used them as one piece of evidence alongside the stack and session state. ss socket inspection manual

Figure 2. The sampled Replicat thread waited in recv(), while PostgreSQL showed ClientRead. The queues were empty. The snapshots do not establish which message, if any, was missing. The dashed line represents the connection. Open full-size diagram

Check the GoldenGate PostgreSQL Replicat driver path

I checked the file that appeared in the stack. The Replicat report identified GGpsql25.so, version 07.16.0445. A newer driver was already installed beside it.

Oracle’s GoldenGate 21c release notes describe DataDirect 08.02.2122 under Bug 35031894 in release 21.9.0.0.3. Using it requires updating existing DSNs to the lowercase ggpsql25.so path. That release-note entry supports the configuration change; it does not establish that this stall was the exact defect described by that bug number. Oracle 21c release notes

$ ls -la $OGG_HOME/lib/*psql25*
GGpsql25.so    2021-06-01   (old driver, 7.16)   ← selected by odbc.ini
GGpsql25r.so   2021-06-01
ggpsql25.so    2023-02-01   (new driver, 8.02)   ← driver named in release notes

[pgtarget]
Driver=/opt/ogg/lib/GGpsql25.so
HostName=...
PortNumber=5432
Database=...
TransactionErrorBehavior=0

The file dates in this record were June 1, 2021, for the uppercase driver and February 1, 2023, for the lowercase driver. Installing GoldenGate 21.12.0.0.4 had not changed the existing DSN. The driver manager continued loading the file named by Driver=. These file names are case-sensitive in this Linux environment.

Figure 3. The original DSN selected GGpsql25.so. The newer ggpsql25.so beside it was not used until the Driver= path changed. Open full-size diagram

Changing the driver path was followed by a complete load

The main corrective change was the driver path. I also removed an ineffective attribute and aligned the DSN description with Oracle’s sample, so this was not a strict single-variable experiment.

Back up the existing DSN and confirm the intended GoldenGate installation path.
Change Driver= to the installed $OGG_HOME/lib/ggpsql25.so file. Use the actual absolute path in the DSN; do not assume that odbc.ini expands shell variables.
Remove TransactionErrorBehavior=0, which GoldenGate overrode in this case.
Align [ODBC Data Sources] and Description= with the Oracle PostgreSQL DSN example.
After restarting, verify ggpsql25.so and the 08.02.xxxx version in the Replicat report’s Driver Information section.

The stall did not recur after the change. The separately observed OGG-00551 / SQLSTATE 08S01 Socket closed abends also stopped. The 1TB LOB-table initial load, which previously stalled within minutes, completed without another restart. CDC then continued normally.

I had considered whether a timeout was closing sessions left open by the stall. The disappearance of both symptoms was consistent with that hypothesis, but I did not identify which component closed the connection during the earlier 08S01 failures. I did not collect an ODBC trace after the system recovered.

What the record establishes

The main thread waited inside the driver’s recv() under SQLPrepareW.
The matched PostgreSQL session waited for client data; the sampled socket queues were empty.
The loaded driver was 07.16.0445, selected through the uppercase DSN path.
After the path correction and DSN cleanup, the stalls and 08S01 failures did not recur, the load completed, and CDC ran normally.

What remained a hypothesis or was not checked

A protocol-state mismatch, an unflushed or incomplete message, or missing Flush/Sync processing was not confirmed by a trace.
The suggestion that larger LOBs increased the opportunity for the stall remained a hypothesis.
The component responsible for the earlier connection closures was not identified.
I did not verify certification of the OL7 build on Rocky Linux 8.10 with PostgreSQL 16. Successful operation is not a certification check.

What I would check first next time

Save the stack, session state, and socket information before recovery changes remove the evidence. Automate collection if the stall is intermittent.
Read the stack from the outer call path to the current wait, and match the process and port to the database session.
Use queue values with topology and repeated samples; do not treat an empty queue as a complete network diagnosis.
Check the exact loaded library, its version, and the DSN path. A newer file beside it does not mean the process uses it.
Include configuration changes required by release notes in the upgrade checklist, and confirm the loaded driver in the report.

If immediate recovery is necessary, collect what the incident allows before restarting and follow the site’s recovery procedure. My conditional-restart runbook (Korean) covers a separate recovery case. The outcome here supports checking the selected driver early, while the exact internal mechanism remains unproven.

Frequently asked questions

What should I check when Replicat is RUNNING but its checkpoint and SEND response stop?

Capture several process stacks and match them with pg_stat_activity and ss -tnp from the same period. This identifies the observed wait and its connection before a restart removes the evidence. OGG-15163 indicates that the status request timed out; it does not establish the underlying cause.

Does idle in transaction / ClientRead prove a driver fault?

No. It describes PostgreSQL waiting for client data while a transaction remains open. In this case, the matching driver-side socket wait made driver behavior worth investigating. A protocol trace would be needed to establish the exact message-level failure.

How do GGpsql25.so and ggpsql25.so differ in this case?

The uppercase file was the older 7.x driver; the lowercase file was the newer 8.02 driver identified in Oracle’s release notes. Check the installed build and report rather than treating the file name alone as proof of a version. The DSN’s Driver= path selected which file was loaded.

References and scope

The versions, 31-second wait, and 1TB completion result come from the supplied incident record. I did not rerun this incident or verify product certification while preparing this article. Commands and paths are anonymized examples; connection credentials and identifying host details are omitted.